Where each number comes from, and how far to trust it.

Every number is cited to the paper it came from

Each figure on BedsideDx — every likelihood ratio, sensitivity and specificity — is one that a named, peer-reviewed study or review reported for this finding in this diagnosis, or one calculated from that source’s own printed figures, in which case we say so. The citation sits beside the figure, so you can open the paper and read the numbers where we read them.

Every pooled figure comes from a published review; we do not pool studies to produce one. No figure is borrowed from a related condition or stretched from one study across several diagnoses. Where more than one source qualifies, we show the one that earns the higher research tier, and we never choose a source for what its numbers say. Preprints, conference abstracts, theses, narrative reviews and textbooks that compile other people’s figures can lead us to studies, but no research tier rests on them.

The research tier judges the research, not the size of the result

A striking number from one small study is not the same as one pooled across several studies of the same question. So every finding shows a research tier, which reflects how well the research behind its numbers was designed and done, and whether it was repeated — not how large the numbers are. There are four:

• Strong evidence — a systematic review that pools this finding’s accuracy for this diagnosis across at least three studies with a statistical model, rather than by adding up their counts; that formally appraised every study it included and judged no more than half of them at high risk of bias in any respect; that measured the finding against a diagnostic standard which did not itself depend on the finding; and that did so mainly in patients being evaluated for the diagnosis or, for a screening finding, in the people screened.
• Moderate evidence — a single well-designed study, one study drawn from a larger review, a review that summarised its studies without statistically pooling them or that pooled only two, or a review that pooled the finding but fell short of one of the standards above. Real evidence, but a narrower foundation.
• Weak evidence — figures that come mainly from a design known to make a finding look more accurate than it is: patients with the diagnosis compared against separately chosen healthy or other people (a case-control design), or a diagnostic standard that leaned on the finding itself. Lijmer and colleagues found that case-control designs inflate the diagnostic odds ratio about threefold on average (JAMA, 1999). We still show these numbers, and their note says so first; a finding is placed here only where the papers show such a design, never on suspicion.
• Insufficient evidence — the tier we give a finding when no published study we could find meets our bar. Instead of a number, its note names the closest study found and why it fell short, or says none was found. You still see the finding, because it is classically taught — just without a statistic we can’t stand behind.

What earns a tier is design, never the journal a paper appeared in. A large likelihood ratio from one small study still rests on one study, and we label it that way.

We search before we call the evidence insufficient

For every finding, in every diagnosis, we work down the same order:

1. Systematic reviews first. We start with systematic reviews and meta-analyses that report where and how they searched and which studies they admitted. When one has statistically pooled this finding across at least three studies and appraised each of them, we use its pooled results as published, or figures calculated from them.
2. Then single studies of sound design. If no such review earns a higher research tier, a single cohort or cross-sectional study can qualify, measured against a diagnostic standard independent of the finding. A case-control study, or one whose standard leaned on the finding, qualifies only where nothing better supports the finding, and only at Weak evidence.
3. Otherwise, we say so. If nothing meets that bar, the finding is placed at Insufficient evidence — but only after a documented PubMed search for that finding in that diagnosis, with no date, language or publication-type limit, that also follows the references and citations of the closest studies. We repeat the search every quarter, and each entry shows the date of its latest one.

What decides is the research tier a source earns, not the kind of paper it is: a single cohort study at Moderate evidence is shown ahead of a review built mainly on case-control studies, at Weak evidence.

Ratings follow the field’s own thresholds

We lead with the likelihood ratio because it says how far a result should move the probability of the diagnosis from wherever you start, and, unlike sensitivity and specificity, it can be applied to a starting probability directly. Some findings show the ratios without sensitivity and specificity: either the paper never published those two, or it reached them by a different calculation from the ratios we show. In that case the note behind the research tier names both sets; we do not print side by side two sets of numbers that were never meant to be read together.

Each ratio is translated into a plain rating using the thresholds of Jaeschke, Guyatt and Sackett (JAMA, 1994) — 2, 5 and 10, and their reciprocals 0.5, 0.2 and 0.1. We divide their lowest band at 1.4 and 0.7, close to its midpoints on a logarithmic scale:

• Very helpful — a large, often decisive shift
• Helpful — a moderate, frequently useful shift
• Somewhat helpful — a small shift that occasionally matters
• Minimally helpful — a shift too small to act on by itself
• Not helpful — no meaningful shift
• Inconclusive — the headline number looks useful, but the range of values the study is compatible with includes no effect at all and extends beyond 0.5 to 2. The evidence cannot settle the question either way, so we withhold the rating and say so. The number stays on the page, so you can see exactly what the study reported.

Some findings work in reverse: a positive result argues against the diagnosis. The classic example is reproducible chest-wall tenderness in suspected acute coronary syndrome, where finding it lowers the probability of a cardiac cause. We label these by their true clinical meaning, so there is no ambiguity.

Uncertainty is shown, and it can change the rating

How far a finding shifts probability, and how sure anyone can be of that shift, are two different questions, and we answer them separately. The rating comes from the study’s central result; beside it we look at the confidence interval (CI), the range of values the study is actually compatible with. A range that includes no effect makes a useful-looking result Inconclusive — unless the whole range lies within roughly 0.5 to 2.0, the band Jaeschke, Guyatt and Sackett described as altering probability to a small and rarely important degree. Then Not helpful is a verdict the evidence has earned, and we give it.

Where a rating needs a qualification, a short note sits directly beneath it. There are four:

• No CI — the paper gave no usable range around its result, and none could be worked out from its figures. The rating reflects the single number reported and nothing more.
• Wide CI — there is a range, but it runs from no useful effect up past a clinically important one. The study was not precise enough to say whether the finding matters.
• No CI, small study — no range, and fewer than fifty patients, in one study or several combined. One number from a handful of patients, with no measure of its uncertainty, is not a rating we can stand behind, so the finding reads Inconclusive and keeps its number and its research tier.
• Derived CI — the source published no usable range for the likelihood ratio but reported enough for one to be calculated from its own figures. The range you see is ours, not the paper’s, and we say so rather than let our arithmetic pass for theirs.

The calculator computes only where the evidence allows

For subscribers, the calculator does the arithmetic: enter your starting (pre-test) probability and the result you observed, and it returns the updated (post-test) probability by Bayes’ rule. It computes only where a likelihood ratio has been published for your result. It will not multiply by a number the study could not settle, or by one from fewer than fifty patients with no measure of its uncertainty. Where it does compute but the answer needs a caveat, such as no usable range behind the ratio, a wide one, or Weak evidence, a line beneath the answer says so.

Four limits no research tier removes

Diagnostic evidence is never perfect, and we would rather you know its limits than discover them later:

• The studied patients may not be yours. Accuracy studies are often done in particular settings — a referral clinic, an emergency department — and may not translate to a different population. The note behind each research tier says who was studied; such differences are disclosed, not adjusted for.
• Some findings are hard to reproduce. A number of physical signs are elicited or interpreted differently from one examiner to the next, which limits even a strong result in everyday practice. Our figures do not account for this.
• Yardsticks change. What counted as the diagnostic gold standard decades ago has sometimes been surpassed — ultrasound replacing the chest X-ray for some questions, for instance — so an older number may not show how an exam compares against today’s standard.
• Flattering results get published more. Studies showing a test works are likelier to appear in print than those showing it doesn’t, which can make a sign look better than it really is. A research tier does not weigh this risk, and no source removes it entirely.

Every figure on BedsideDx can be checked against its source. If one does not match, or you know of a study we should have found, tell us with the Suggest an edit button at the bottom right of every page.

← Back to about