Size & Power of Test (Edexcel A Level Further Maths): Revision Note

Size

What is the size of a test?

• The size of a test is the probability of rejecting H₀ when it was in fact true

  • P(in critical region | H₀ is true) 

  • The situation being described is not a good outcome 

    • Something has been rejected when it was actually true!

  • A better test has a smaller size

    • You want to minimise this error happening

• Size is related to the significance level, α%

  • A better test has a smaller significance level (e.g. 1%)

  • For continuous distributions (e.g. normal)

    • Size = significance level, α

    • You can often write this down with no calculation

  • For discrete distributions (e.g. binomial, Poisson, geometric)

    • Size = actual significance level (≤α)

    • As close to α% as a discrete variable can get, whilst still being critical

How does size relate to Type I errors?

• The size is exactly the same as the probability of a Type I error

  • Both want to know the probability of rejecting H₀ when it was in fact true

Power

What is the power of a test?

• The power of a test is the probability of rejecting H₀ when it was false

  • P(in critical region | H₀ is false) 

  • The situation being described has a good outcome 

    • The null hypothesis was false and it rightly got rejected

  • A better test has a higher power

    • You want to maximise this happening

• In practice, you need to be given the actual population parameter to calculate the power

  • For example, H₀ assumed  but actually 

    • This is more helpful than just saying  

  • Power is P(in the critical region | actual population parameter)

How does power relate to Type II errors?

• The power of a test is 1 - P(Type II error) 

  • Power is when H₀ is false and it gets rejected

    • That's a good outcome

  • A Type II error is when H₀ is false and it does not get rejected

    • That's a bad outcome

• You ideally want the power of a test to be greater than 0.5 

  • That way it's less likely to produce a Type II error

    • And more likely to reach the correct conclusion

Power Functions

What is a power function?

• The power function is the power of a test written algebraically

  • In terms of or 

  • For when you're not given the actual population parameter in the question

• In reality, it's very unlikely you'll know the actual population parameter anyway

  • Otherwise you wouldn't be doing a hypothesis test on it!

  • Power functions don't need this information

• The power function is P(in the critical region | population parameter is )

  • Or, for Poisson 

    • P(in the critical region | population parameter is )

How do I find power functions?

• It's easier to show in an example

  • If the critical region is for a binomial hypothesis test with  

  • Then the power function is P(in the critical region | population parameter is )

    • Let 

    • 

  • Simplify

    • 

  • Factorise and collect like terms

    • The power function is

What can I do with power functions?

• You can plot them against  (or ) 

  • You can then see where the power is biggest

• You can input different values of  (or ) 

  • To compare two (or more) different hypothesis tests

    • The better test is the one with the higher power

  • To check if the power of a test is greater than 0.5

    • So that it's more likely to reach the correct conclusion

    • And less likely to produce a Type II error

    • Because Power = 1 - P(Type II error)