Critical Appraisal

Critical appraisal is the process of systematically reviewing a study to identify its strengths and weaknesses, key findings, and broader implications. It is an essential clinical skill because the volume and variable quality of medical literature requires clinicians to judge which findings are trustworthy and applicable.

Internal validity asks how accurately the study findings reflect the true relationship in the study population. It is affected by three principal threats: chance (random error), bias (systematic error from design or conduct), and confounding (distortion by a third variable related to both exposure and outcome).

Chance (random error) produces confidence intervals around estimates — narrower CIs reflect greater precision. Assumptions of statistical methods must be met (e.g., independence, approximate normality of the sampling distribution). When multiple statistical tests are performed, the probability of at least one false positive increases — this multiple testing problem requires adjustment.

Bias arises from systematic errors. Selection bias occurs when the selection process creates a systematic difference between the sample and the source population — caused by non-response, the healthy worker effect, or differential referral. Information bias is systematic error in how information is collected, interpreted, or recorded — caused by the observer, the study individual (e.g., recall bias, non-adherence), instruments used, or missing data (loss to follow-up). In analytic studies, information bias is particularly problematic when there are systematic differences in data quality between exposure groups.

Confounding: a confounding variable is associated with both the exposure and the outcome, independently of each other, and is not on the causal pathway. Positive confounding makes an association look stronger or creates a spurious association; negative confounding makes an association look weaker or in the opposite direction. Common confounders include age and sex. Control methods: restriction, matching, and randomisation in design; multivariable analysis in the analysis phase.

Experimental studies (RCTs) provide stronger tests of causal hypotheses than observational studies because they allow direct testing by manipulating exposure allocation. Observational studies — cohort and case-control — cannot isolate the effects of exposure because participants self-select into exposure groups.

External validity (generalisability) asks whether results can be applied to populations beyond the study sample, considering differences in participant characteristics, disease prevalence, clinical context, and co-interventions.

Causation is difficult to establish from a single observational study because of susceptibility to bias and confounding. An association is more likely to be causal when (BEST CDS): the association is very strong (B — big effect), there is evidence of a Dose-response, results from Several studies are Consistent, and there is a known biological mechanism (Dose-response, Specificity).