The synergetic effect on hardenability by combining boron with other microalloying elements (such as Nb, Mo and Nb + Mo) is widely known for high-strength medium carbon steels produced by direct quenching and subsequent tempering treatment. The improvement of mechanical properties could be reached through optimization of different mechanisms, such as solid solution hardening, unit size refinement, strain hardening, fine precipitation hardening and the effect of carbon in solid solution. The current study proposes a procedure for evaluating the contribution of different microstructural aspects on Charpy impact toughness. First, the effect that austenite conditioning has on low-temperature transformation unit sizes and microstructural homogeneity was analysed for the different microalloying element combinations. A detailed crystallographic characterization of the tempered martensite was carried out using electron backscattered diffraction (EBSD) in order to quantify the effect of unit size refinement and dislocation density. The impact of heterogeneity and presence of carbides was also evaluated. The existing equations for impact transition temperature (ITT50%) predictions were extended from ferrite-pearlite and bainitic microstructures to tempered martensite microstructures. The results show that microstructural refinement is most beneficial to strength and toughness while unit size heterogeneity has a particularly negative effect on ductile-to-brittle transition behaviour. By properly balancing alloy concept and processing, steel having a yield strength above 900 MPa and low impact transition temperature could be obtained by direct quenching and tempering.