Reliability of Qualitative & Quantitative Methods (AQA GCSE Psychology)

Quantitative methods are those which generate quantitative data, for example:

• Lab experiments

• Closed questions on a questionnaire

• Observations which use a tally chart to record the frequency of specific behaviours

Reliability is achieved if a study has been set up so that the IV can be seen to affect the DV:

• If the procedure is replicated, the study should show similar results

• Replicating a study and finding similar results shows that the measure is consistent and not liable to fluctuate to any significant degree

If a study is completed using a standardised procedure and obtains the same results, it can be said that it is reliable 

Lab experiments are the most reliable method used in psychology as they:

• take place in a neutral space under controlled conditions

• follow a standardised procedure

• use random allocation to conditions

• tend to use a control group as a comparison to the experimental group

• generate quantitative data which is easy to compare and analyse

Field experiments generate quantitative data and implement an IV but they are subject to an array of extraneous variables over which the research has no control, making them less reliable than lab experiments

Natural experiments generate quantitative data but they use a naturally occurring IV over which the research has no control making them less reliable than lab experiments

Questionnaires are a reliable self-report method as they:

• Tend to use all (or mainly) closed questions

• Closed questions generate quantitative data which can be analysed and translated to graphs, percentages etc.

• Use a standardised set of questions

• Are likely to show consistency i.e. people are unlikely to radically change their answers from one testing session to the next

• The reliability of a questionnaire can be assessed using two methods:

  • The test-retest method measures external reliability: 

    • The same participants are given the same questionnaire at separate time intervals (e.g. with a 6-month gap between testing sessions) 

    • If the same result is found per participant then external reliability is established

  • The split-half method measures internal reliability:

    • The researcher splits the test in half and analyses the responses given to the first half of the questionnaire compared to the second half of the questionnaire

    • If similar responses are given in both halves then internal reliability is established

For an observational study the researchers will have previously agreed on which specific behaviours should be recorded so that all observers are in agreement before the observation 

Researchers can test the reliability of their observations by comparing them with another researcher's recording of their behaviours: the level of consistency between the two records is then compared

Inter-observer reliability is the level of consistency between two or more trained observers when they conduct the same observation, as follows:

• The observers compare the two independent data sets (often designed as a tally chart)

• They then test the correlation between the two sets

• If there is a strong positive correlation between the sets then this shows that there is good inter-observer reliability and that the behaviour categories are reliable

• All observers must agree on the behaviour categories and how they are going to record them before the observation begins

• The observation is conducted separately by each observer to avoid conformity (i.e. one observer may be influenced by one or more other observers)

• After the observational period:

Establishing good inter-observer reliability means that there is less chance that researcher bias has interfered with the observation