Statistical Testing (AQA A Level Psychology)

Probability & significance

• Psychologists are interested in finding out if the results of their studies show real differences or correlations, or if the results are due to chance factors

• To determine whether results are significant and not due to chance factors, researchers use a measure of the level of significance 

• Researchers must decide how large an effect or relationship is required to conclude that the observed result is unlikely to be due to chance

  • This decision is reflected in the level of significance applied to the data

• The level of significance is expressed as a decimal value where 'p' stands for the probability that chance factors are responsible for the results

• For most purposes in psychology, the 5% level of significance is appropriate which is expressed as p < 0.05 (i.e. the probability of chance factors producing the observed result is less than or equal to 5%)

• Psychologists may use a more stringent level of significance, e.g. p < 0.01 (i.e. the probability that the results have occurred by chance is less than or equal to 1%) in cases where:

  • there is a human cost, e.g. drug trials

  • where existing research evidence is contradictory

• The researcher will then use statistical tables to find the critical value which will determine whether or not they can reject the null hypothesis

Use of statistical tables & the critical value

• Once the researcher has conducted their research and carried out a statistical test, they then have an observed value, which is used to determine whether the results of their study are significant  

• The observed value needs to be compared to the critical value in the statistical table

• Each statistical test has its own critical values table

  • The critical values table for the Mann-Whitney test is different to the table for the Wilcoxon test

• To find out whether or not the observed value is significant, the researcher must ask the following questions, which will help them to use the critical values table properly:

  • Is it a test of difference (i.e. a lab experiment) or a test of correlation?

  • Is the experimental design independent measures, repeated measures or matched pairs (tests of difference only)?

  • Is the data nominal, ordinal or interval?

• Once the above questions have been answered, the researcher consults the critical values tables to find the value at which a significant result can be claimed, e.g.

  • using the probability level of 0.05, if the observed value is 15 and the critical value is 17 then (for some of the statistical tests) the researcher can claim a significant result

    • This is due to the observed value being lower than the critical value at the 0.05 level (for some tests, the observed value must be higher than the critical value)

• If significance is found, then the researcher can reject the null hypothesis and accept their alternative hypothesis; the results are significant, not due to chance

Type I & Type II Errors

• A Type I Error occurs when the null hypothesis is rejected when it should have been accepted

  • The researcher claims that the results are significant when in fact they are not (also known as a ‘false positive’)

• A Type I Error is more likely to happen when the researcher uses a probability value that is too stringent e.g.

  • 0.1 rather than 0.05 

  • 0.06 rather than 0.05

• A Type II Error occurs when the null hypothesis is accepted when it should have been rejected

  • The researcher claims that the results are not significant when in fact they are (also known as a ‘false negative’)

• A Type II Error is more likely to happen when the researcher uses a probability value that is too lenient e.g.

  • 0.01 instead of 0.05

  • 0.03 instead of 0.05

• Using a 0.05 significance level guards against making either a Type I or a Type II Error