A small gap between groups can be just noise
- The idea
- Predict
- See it
- Why
- Check
The idea
“If one group's average is higher, even a little, the treatment made a difference.”
Numbers look exact, so 41 beating 40 feels like a result. Repeat any measurement, though, and the number wobbles.
The short version
The mistake: If one group's average is higher, even a little, the treatment made a difference.
What's actually true: Whether a gap between group averages is real depends on how it compares with the spread among repeats in each group.
- Repeat measurements always vary. Seeds differ, dishes differ, and every measurement has some error. That scatter is noise.
- So compare the gap between group means with the spread of repeats within each group.
- A gap much bigger than that spread likely reflects a real effect. A smaller one could just be chance.
On the exam: Check any gap between groups against the spread within each group.