SYSTEM Research note on: Initial observations of diagnostic accuracy concerning quantitative testing for selection bias in RCTs

CONTEXT: Selection bias interferes with the internal validity of clinical trials and leads to favouring one clinical outcome over another. In order to limit the influence of selection bias on clinical trials, the methodological interventions: random sequence generation and allocation concealment of such sequence have been proposed. Subsequently, authors of systematic reviews judge risk of selection bias in trials according to the reported details concerning how random sequence generation and allocation concealment of such sequence was conducted.

PROBLEM: The problem with the current approach is that it only can judge bias risk on the reported attempt of adequacy of random sequence generation and allocation concealment but not whether such reported attempt (even if judged as adequate from the trial report) was indeed successful. High risk of selection bias may be prevalent and thus limit the validity of trial results even when adequate methods for bias control were reported. Without any further evidence as to whether any as adequate reported attempt was successful there is no guarantee that the reported trial results are not overestimations of the true intervention effect.

SUGGESTED SOLUTION: One possible option could be based on the level of correlation (established by use of linear regression analysis) between the (i) probability values for each subject to be allocated to an intervention group with (ii) the observed dichotomous intervention outcome per subject. The probability for each subject to be allocated to an intervention group can be expressed through the reverse propensity score (RPS). Without selection bias, the RPS should not be associated with the intervention outcome of participants in a RCT beyond play of chance. Based on the RPS and by following a previously described methodology the summary measures of potential diagnostic test accuracy were established at different block sizes and subject numbers. Reasonably high evidence in support for test accuracy with values >0.80 in terms of specificity and sensitivity, >5.0 in terms of positive likelihood ratios and