How Confidence Intervals Inform a Decision
Use a confidence interval to understand sampling uncertainty, not as the probability that the true value sits inside one computed interval.
Audit before applying the interval
- Check sampling and measurement design.
- Identify the estimate and population.
- Review model assumptions.
- Test decisions across the interval endpoints.
Intervals answer a method-specific question
A frequentist confidence interval describes how an interval procedure performs across repeated samples under assumptions. It does not correct biased recruitment, poor measurement, or a sample unrelated to the target population.
Decision relevance differs from significance
A precise difference can be too small to matter, while an uncertain estimate may still rule out an unacceptable downside.
Example: preference survey
A narrow interval around a volunteer-site poll does not make it representative. A smaller probability sample may have a wider interval but a defensible population claim.
The decision rule
What to record before acting
Use intervals to test decision sensitivity only after the data collection process supports the intended claim.
Apply the rule in context with audit who the sample represents. Keep the evidence and assumptions with the decision so an outcome review can distinguish reasoning from luck.
Continue within evidence and uncertainty
Compare Evidence From Small Samples
Evaluate small-sample evidence through case selection, measurement quality, effect size, consistency, and how much the decision depends on it.
Compare Evidence From Small SamplesLeading Indicators for a Decision
Choose leading indicators that appear before the outcome, connect through a credible mechanism, and can trigger a specific decision or correction.
Leading Indicators for a Decision