Confidence Intervals V. Hypothesis Testing
Suppose you read the following statement:
The mean value for the intervention group was 29 points lower than for the control group (p-value < 0.05). This might correspond to either of the following 95% confidence intervals:
- Treatment difference: 29.3 (22.4, 36.2)
- Treatment difference: 29.3 (11.8, 46.8)
If exact p-value is reported, then the relationship between confidence intervals and hypothesis testing is very close. However, the objective of the two methods is different:
- Hypothesis testing relates to a single conclusion of statistical significance vs. no statistical significance.
- Confidence intervals provide a range of plausible values for your population.
- Use hypothesis testing when you want to do a strict comparison with a pre-specified hypothesis and significance level.
- Use confidence intervals to describe the magnitude of an effect (e.g., mean difference, odds ratio, etc.) or when you want to describe a single sample.