Evaluation of Quantitative Research Methods (DP IB Psychology)

Evaluation of experiments

Lab experiments

Strengths

• It is easier to establish a cause-effect relationship between the IV and the DV than for other methods used in psychology

  • This is due to the use of controls and the objective nature of the research

    • Therefore lab experiments are high in internal validity

• The use of a standardised procedure means that the research is replicable which increases the reliability

Limitations

• The use of artificial tasks means that lab experiments lack ecological validity

  • If participants are performing tasks in an unfamiliar, 'sterile' setting this does not reflect how they might behave in real conditions

  • This makes the findings difficult to generalise beyond the lab setting

• Demand characteristics may limit the generalisability of the findings

  • As participants know they are in a study they may alter their behaviour (e.g., they may feel shy, self-conscious or they may try too hard)

  • This would lower the external validity of the study

Field experiments

Strengths

• As the research is conducted in real settings the degree of artificiality is reduced

  • If participants feel more comfortable in the setting this means that their behaviour is likely to reflect their behaviour outside of the research parameters

  • This makes field experiments high in external validity

• Participants are less likely to experience demand characteristics due to the more relaxed and 'natural' environment of a field experiment which heightens the validity of the findings

Limitations

• Extraneous variables are much more likely to interfere with the findings of the study

  • The researcher cannot control all extraneous variables due to the lessening of control possible with field experiments

  • This means that reliability is reduced

• It is more difficult to replicate field experiments due to the nature of the method

  • This in turn reduces the ability to check for consistency - and hence reliability - of the results

Quasi experiments

Strengths

• Due to the lack of manipulation of the IV, the results could be said to be higher in external validity e.g.

  • Comparing the performance of young versus older people on a memory test gives insight into the effect of age on recall

  • Comparing the ability to identify emotion based on empathy training gives insight as to how this training might benefit other groups or professions

• Quasi-experiments follow a true experimental design which means that they could be replicated with participants that match the original sample in terms of demographics e.g.

  • The effect of age on recall could use the same procedure over and over again

Limitations

• As the participants cannot be randomly allocated to condition this can lead to participant variables, making it difficult to determine causality e.g.

  • A study which investigates the effect of age on recall might include a group of participants (in either the younger or the older group) who naturally have a much better memory than is representative of their population

    • This means that quasi-experiments are less reliable than true experiments

• Quasi-experiments lack internal validity, as there may be other factors which could explain the results

  • The teachers who have been trained in empathy may work in a school in which emotional intelligence is valued

    • This means that they would already be at an advantage in an emotion-recognition task

Natural experiments

Strengths

• Natural experiments allow the researcher to investigate topics which would otherwise be unethical to study using a traditional lab experiment e.g., experiencing a mental illness or a natural disaster

  • This means that natural experiments are high in ethical validity

• Natural experiments are high in ecological validity

  • The participants report on events and experiences that they have personal, first-hand knowledge of

  • The researcher does not attempt to control the procedure

    • This elevates mundane realism

Limitations

• Causal relationships are difficult to determine due to the array of variables at play

  • This is a key limitation of research which imposes no controls on the procedure

  • The researcher cannot be sure as to what effect the phenomena have had on the participants - they have to rely on the accounts of the participants themselves

    • This reduces the reliability of natural experiments

• Natural experiments may suffer from several types of bias which would lower the validity of the study e.g.

  • social desirability bias

    • The participant wishes to present themselves and their experiences in the best light possible

  • confirmation bias

    • The researcher looks for evidence that will back up any pre-existing ideas they have brought to the research process

  • sample bias

    • The sample is determined by experience which means that only a very limited group of people can be used in the research

Evaluation of correlational research

Strengths

• The data may be easily available for researchers to quickly analyse 

  • This is a strength as it enables the researcher to access large amounts of data that would otherwise be impossible to gather if they tried to amass this from scratch

    • Large amounts of quantitative data mean that the research is high in reliability

• Correlations allow researchers to make predictions as to the relationship between co-variables e.g.

  • Knowing that there is a relationship between school absence and GCSE results could be used to identify students at risk and to implement interventions to help them achieve their potential

Limitations

• Extraneous factors connected to one or both co-variables may affect the result and lead to invalid conclusions being made e.g.

  • number of days absence from school may be due to illness rather than to choice 

  • a low GCSE score may be due to a high turnover of teachers in one school rather than to student absence

• Correlations work well for linear relationships e.g., height and shoe size

  • They are less successful when dealing with non-linear relationships e.g. number of hours worked and level of happiness

• This limits the type of data that can be analysed and conclusions drawn

Evaluation of questionnaires

Strengths

• Questionnaires are a quick, easy and convenient method of gathering data

  • Large samples can be reached via the use of electronic survey tools

  • Large samples produce reliable results as any anomalous results are averaged by the overall trend of the data

• Questionnaires use standardised questions which means that they can be replicated to check for reliability

  • The test-retest method can check for external reliability

  • The split-half method can check for internal reliability

Limitations

• There is a tendency for people to under-report negative and over-report positive aspects of themselves when completing a questionnaire

  • This means that questionnaires can lead to participants succumbing to social desirability bias

  • Any form of bias in research impairs the validity of the findings

• Questionnaires tend to under-utilise open questions which limits their usefulness

  • This means that they can show the 'what' of behaviour (e.g., people become more cautious with age) but not the 'why' of that behaviour (e.g., why do people become more cautious with age?)