Progress in detection of and correction for low-energy contamination

Abstract

Contamination with low-energy radiation leads to an increased number of weighted residuals being larger in absolute terms than three standard uncertainties. For a Gaussian distribution, these rare events occur only in 0.27% of all cases, which is a small number for small- to medium-sized data sets. The correct detection of rare events – and an adequate correction procedure – thus relies crucially on correct standard uncertainties, which are often not available [Henn (2019), Crystallogr. Rev. 25, 83–156]. It is therefore advisable to use additional, more robust, metrics to complement the established ones. These metrics are developed here and applied to reference data sets from two different publications about low-energy contamination. Other systematic errors were found in the reference data sets. These errors compromise the correction procedures and may lead to under- or overcompensation. This can be demonstrated clearly with the new metrics. Empirical correction procedures generally may be compromised or bound to fail in the presence of other systematic errors. The following systematic errors, which were found in the reference data sets, need to be corrected for prior to application of the low-energy contamination correction procedure: signals of 2λ contamination, extinction, disorder, twinning, and too-large or too-low standard uncertainties (this list may not be complete). All five reference data sets of one publication show a common resolution-dependent systematic error of unknown origin. How this affects the correction procedure can be stated only after elimination of this error. The methodological improvements are verified with data published by other authors.