Scientific methods · Chapter 04.03
Confidence in a biological explanation and confidence in a clinical effect are related questions, but they require different evidence.
Certainty belongs to an estimate for a specific comparison, population, outcome and follow-up period, not to a compound in the abstract. GRADE distinguishes high, moderate, low and very low certainty. An intervention can have different certainty for symptom relief, serious adverse events and survival. Certainty assessment and a recommendation are separate tasks: recommendations also require values, feasibility and benefit–harm considerations.
Risk of bias, inconsistency, indirectness, imprecision and publication bias address different routes to uncertainty. Record concerns and their reasons rather than adding points into an automatic score. Study design informs the assessment but does not settle it: rigorous randomization cannot prevent selective reporting or loss to follow-up. Reviewers should avoid counting the same underlying problem twice across overlapping domains.
Compare the evidence with the intended population, intervention, comparator and outcome. A cell-culture change in signaling is not a demonstrated improvement in a person’s symptoms or survival. Species, tissue, achievable exposure, formulation and concurrent therapy affect applicability. Surrogates need justification for the intended setting; mechanistic coherence can support plausibility without removing the need for direct clinical outcome evidence.
Before interpreting an interval, define which absolute differences would count as important benefit or harm over a stated period. If the interval crosses those thresholds, substantially different interpretations remain plausible. Consider sample size and event counts as well as interval width. In the illustrative explorer, changing baseline risk changes absolute effects even when the relative risk is fixed; it does not change evidence certainty automatically.
Check whether effects differ in direction or magnitude across studies and whether prespecified population, exposure or methodological differences explain them. I² describes a proportion of observed variability under its assumptions; it is not an effect size or a universal quality threshold. A pooled average can hide clinically important variation, and post-hoc subgroup explanations require independent confirmation rather than immediate acceptance.
Even internally rigorous studies can provide a misleading body of evidence if publication depends on favorable results. Search for unreported studies, compare prespecified and published outcomes, and retain uncertainty about missing results. An attractive dose-response pattern or large association does not excuse major biases. Any certainty upgrade requires a defensible explanation of why plausible confounding or selective availability is unlikely to account for the finding.
Compare three fictional evidence profiles. The first two assume a body of randomized trials with high starting certainty. These are reasoned examples, not an automated GRADE calculator; no study quality score is added up.
High certainty for the narrowly specified hypothetical clinical outcome
Fixed RR = 0.80 (95% CI 0.60–1.10). Difference = baseline risk × (RR−1); a negative number means fewer events. The interval spans benefit and harm. No real population or time horizon is represented.
Full GRADE requires an explicit question, a body of evidence, outcome-specific judgments and documented reasons. This example omits upgrading and alternative GRADE approaches and is not a certification of any Curetanics finding.
Methods behind the example: [2]
These sources support the scientific methods discussed. They do not validate Curetanics or establish efficacy for a compound. Reference titles remain in their original language.