Scientific methods · Chapter 04.01
From a searchable question to an auditable synthesis: how research is located, appraised and interpreted.
Define population, intervention or exposure, comparator, outcome and observation period before reading results. A prevention question, a supportive-care question and a recurrence question require different eligibility criteria. Combine controlled vocabulary with free-text synonyms, search more than one appropriate database, and include trial registries. Preserve the exact query, dates and selection decisions so another reviewer can reproduce the evidence set.
One trial can generate a protocol, abstract, primary report and follow-up papers. Link these records rather than counting them as independent experiments. PRISMA documents how a systematic review was reported, including exclusions and the flow of records. It is not a certificate of sound methods, and peer review does not guarantee reproducibility, complete reporting or a correct conclusion.
For a randomized trial, RoB 2 examines the randomization process, deviations from intended interventions, missing outcomes, outcome measurement and selective reporting. Specify which result and treatment-effect estimand are being judged. A blinded laboratory endpoint and a subjective symptom endpoint can have different vulnerabilities within the same trial. Do not replace this domain-based judgement with journal prestige or a numerical quality total.
ROBINS-I addresses non-randomized intervention studies, not every observational question. Begin with the hypothetical target trial and plausible confounders: disease severity, co-medication and health-seeking behavior can influence both supplement use and outcomes. Adjustment requires causal reasoning; controlling a mediator or collider can introduce bias. For exposure, diagnostic or prediction questions, select a framework designed for that study type.
Published findings can be distorted when unfavorable studies or prespecified outcomes remain unreported. Compare reports with registries and protocols; seek corrections, retractions and additional results. Funnel-plot asymmetry is a signal requiring explanation, not proof of publication bias: heterogeneity and small-study effects can also produce it. A large number of favorable articles does not resolve selective missingness.
A P value does not state the size of an effect, its clinical importance or the probability that a hypothesis is true. Report an appropriate effect measure, its confidence interval and the underlying event counts or scale. For binary outcomes, translate relative effects into absolute differences using a justified baseline risk and time horizon. Distinguish an imprecise estimate from evidence of no meaningful difference.
Four fictional studies report the same binary outcome. Pooling uses log risk ratios and inverse-variance weights. Nothing here represents an actual trial.
| Study | Risk ratio | Standard error of log(RR) |
|---|---|---|
| A | 0.80 | 0.12 |
| B | 0.92 | 0.18 |
| C | 0.76 | 0.22 |
| D | 1.30 | 0.30 |
Each line is exp(log(RR) ± 1.96·SE); marker size is constant. The diamond is the pooled estimate. The vertical line at RR = 1 marks no relative difference.
Q = 2.71 · I² = 0.0% · τ² = 0.000
Common-effect: wᵢ = 1/SEᵢ². Random-effects: wᵢ = 1/(SEᵢ² + τ²).DerSimonian–Laird estimates τ²; the interval uses a normal approximation. With only 3–4 studies, heterogeneity estimates and coverage are unstable. I² is not a measure of bias or clinical importance. This lab cannot decide which studies should be combined.
Methods behind the example: [7]
These sources support the scientific methods discussed. They do not validate Curetanics or establish efficacy for a compound. Reference titles remain in their original language.