Well-Designed Experiments (College Board AP® Statistics)

Confounding variables

What are confounding variables?

• A confounding variable is a variable that

  • you are not interested in

  • but that can affect the results of your experiment

    • e.g. the levels of background noise when trying to conduct a memory test

• As the explanatory variable changes, the confounding variable also changes

  • These two changes both influence the response variable

    • This makes it hard to draw conclusions

• For example, it may look like increasing coffee drinking increases rates of heart disease

  • but actually increasing coffee drinking increases tendency to smoke which increases rates of heart disease

    • The level of smoking is the confounding variable

What should I do if there are confounding variables?

• In an experiment, it is important to

  • identify possible confounding variables before beginning an experiment

  • control (minimize or eliminate) the effect of any confounding variables

    • e.g. conduct memory tests in a quiet room

• In an observational study, you cannot control any confounding variables

  • This makes it harder to know what is causing what

Well-designed experiments

What is a well-designed experiment?

• A well-designed experiment consists of the following:

  • At least two treatments groups (comparing one group to another group)

    • A control group counts as a treatment group

  • Treatment groups are formed by randomly assigning treatments to the experimental units

    • This keeps the groups as similar as possible before the experiment

    • This makes it easier to distinguish responses to treatments only

    • This process is called randomization

  • Treatment groups have more than one experimental unit each

    • This reduces the effects of any natural variations

    • This is called replication (the more the better!)

  • Confounding variables are identified and controlled

    • This ensures they stay the same across all treatment groups

Statistically significant experiment results

What are statistically significant experiment results?

• The results of an experiment are called statistically significant if the changes in the response variable (or the differences between treatment groups) are so large that they are unlikely to be down to chance

  • It suggests that there is a relationship between the treatment and the response

When can I use the word "cause" in my conclusions?

• You can conclude that the treatment causes the response if the following two conditions are met:

  • Treatments were randomly assigned to experimental units

  • The experimental results are statistically significant

When can I generalize the results from an experiment to the population?

• You can generalize the results from an experiment to a population if

  • the sample of experimental units used in the experiment was randomly selected from the population

    • random selection reduces bias in the sample and makes it more representative of the population