SYSTEM Research note on: Initial observations of diagnostic accuracy concerning quantitative testing for selection bias in RCTs - II

CONTEXT: Selection bias interferes with the internal validity of clinical trials and leads to favoring one clinical outcome over another. Random sequence generation and allocation concealment of such sequence have been proposed to limit the risk of selection bias. However, selection bias can be introduced based on knowledge of the directly observed random sequence when allocation concealment is subverted. Such subversion may statistically be detected in randomised control trials with dichotomous outcomes through regression analysis of the reversed propensity score (RPS) sequence together with the sequence of the observed dichotomous outcome per patient.

PROBLEM: Preliminary investigations have shown promising accuracy of RPS based selection bias testing in simulated randomised control trials (S-RCTs). However, these results were not presented in form of summary receiver operating characteristics (SROC) curves that may have eased the graphical recognition of the achieved test accuracy.

SUGGESTED SOLUTION: The previously established total numbers of true negative/false positive (TN/FP) and true positive/false negative (TP/FN) results, per set alpha level, were entered into Meta-DiSc Version 1.4 statistical software and SROC curves per alpha level were generated. Initially, the SROC curve for alpha level 5% suggested higher test accuracy than for alpha 1%. However, the result may have been due to one data point outlier. Correction by removal of the outlier suggested test accuracy for alpha 1% > alpha 5% > alpha 20%. Further investigations to this topic are needed through well-designed statistical simulation studies.