Experiments represent a highly effective methodology for data collection, particularly for researchers seeking to determine the effects of specific actions or stimuli in creating reactions or changes in other stimuli. In other words, experiments are particularly suitable when the research purpose is explanatory (see chapter 3 for a review of the various purposes of social science research). Predominantly classified as a quantitative research method, experiments are employed by scholars in fields such as criminal justice, psychology, and various other social science disciplines to explore a diverse array of research inquiries. Even for individuals who do not intend to engage in experimental research themselves, acquiring an understanding of the nature and execution of experiments is invaluable: it enhances one’s ability to critically assess the experimental studies encountered in academic journal articles or mainstream media reports.

It is noteworthy that students enrolled in research methods courses frequently utilize the term “experiment” to encompass a broad spectrum of empirical research endeavours. However, within social scientific inquiry, this term possesses a distinct definition and should not be indiscriminately applied to all methodological approaches. Furthermore, it is essential to acknowledge the contributions and perspectives of Indigenous Peoples in discussions surrounding research methodologies, particularly given their historical experiences with research practices that have often marginalized their voices and knowledge systems.

In this chapter, we will explore experimental methodologies within criminological research, focusing on their frameworks, applications, and implications. We will begin by defining what constitutes an experiment, laying the groundwork for our discussion. Following this, we will distinguish between true experiments and quasi-experiments, highlighting the critical differences that affect the validity and reliability of research findings. We will also clarify the roles of experimental groups and control groups in research design, as these concepts are foundational to understanding how experiments are structured. Furthermore, we will delve into various types of true experimental designs, such as randomized controlled trials and field experiments, before examining different quasi-experimental designs that researchers may employ when randomization is not feasible. A significant aspect of our discussion will focus on the implications of each type of experimental research design in studies centred on Indigenous communities, where cultural sensitivity and ethical considerations are paramount. Lastly, we will critically assess the strengths and weaknesses of experimental methodologies in criminology. This multifaceted approach aims to equip students with a nuanced understanding of experimental methodologies in criminological research.

What is an Experiment?

An experiment can be defined as a systematic method of data collection aimed at evaluating hypotheses within controlled environments. This form of empirical research may be employed in either laboratory or field settings. Laboratory experiments occur within artificially constructed environments established by the research team, allowing for precise control over variables. Conversely, field experiments are conducted in naturalistic contexts, such as within actual agencies or organizations, thereby facilitating the observation of phenomena in real-world scenarios. Regardless of the setting in which an experiment is performed, certain fundamental terminology is universally applicable across all experimental methodologies.

In the context of experimental research, participants are typically divided into two distinct groups: one exposed to an experimental stimulus, sometimes referred to as a treatment, and another that serves as a control group, which does not receive such exposure. Social science researchers employ a diverse array of experimental stimuli, which may include written passages, visual imagery, audiovisual materials, and even olfactory (i.e., smelling) or auditory (i.e., hearing) cues. The group subjected to the experimental stimulus is referred to as the experimental group, while the group that remains unexposed is designated as the control group. In this context, researchers use various methodologies, including surveys and interviews, to quantify the impact of the stimulus. These assessments are conducted at two distinct points: prior to the introduction of the stimulus, termed the pre-test, and after its introduction, referred to as the post-test.

When conducting experimental research, scholars need to carefully consider the implications of both random selection and random assignment in their methodological frameworks. Both of these instances occur at different stages of the research process.   Random selection refers to the process of selecting participants from a population through a technique of probabilistic random sampling, as elaborated upon in chapter 7b. Following this initial phase of sampling, researchers engaged in experimental designs should strive for random assignment whenever feasible. That is, once a sample has been selected, random assignment involves the systematic allocation of participants in the sample into either experimental or control groups through a randomization process. This methodological approach significantly enhances the likelihood that the experimental and control groups are similar prior to the administration of any stimuli or interventions.

Types of Experimental Designs in Criminological Research

Criminology scholars employ various experimental designs to rigorously test their hypotheses. These designs can be categorized into two primary types: “true experiments” and “quasi-experiments.” Each of these categories encompasses a combination of essential components, including independent and dependent variables, pre-testing measures, stimuli presentation, post-testing assessments as well as the establishment of experimental and control groups in true experiments or comparison groups in quasi-experiments. A critical distinction between these two types lies in the methodology employed for group formation: true experiments uniquely incorporate random selection and random assignment to create their experimental and control groups. This is necessary to ensure the internal validity of findings (review chapter 6b and chapter 6c for the various forms of validity relevant for social science research). These two designs are described in more detail below.

True Experiments

True experiments are characterized by the systematic manipulation of the independent variable and dependent variable as well as the implementation of pre-testing and post-testing measures. These studies typically involve the establishment of experimental and control groups, which are selected and assigned through rigorous random selection and assignment methodologies. Within the realm of true experimental designs, there are three prevalent types: the classic experimental design, the Solomon four-group design, and the post-test-only control group design.

Classic experiments serve as a foundational methodology for examining the impact of specific stimuli on behavioural outcomes in criminological research; in other words, these experiments are key to determining cause-and-effect relationships. In such studies, there are two distinct groups: one subjected to the stimulus under investigation (the experimental group) and another that remains unexposed to the stimulus (the control group). This methodological approach allows us to determine how an independent variable influences a dependent variable. Given that the primary focus of the researcher is to ascertain the effects of the independent variable, you must conduct measurements on participants regarding the dependent variable both before and after the administration of the independent variable or stimulus. Consequently, pre-testing and post-testing protocols are critical components within the framework of the classic experimental design.

One notable example of a criminology-related study that employs pre-testing and post-testing protocols, particularly involving Indigenous participants, is the research conducted by Morin et al. (2022). This study evaluated the effectiveness of the Indigenous Healing and Seeking Safety (IHSS) intervention – an integrated, culturally grounded program that combined Indigenous healing practices with a trauma-informed, harm-reduction model – within a residential treatment program in northern Ontario. The researchers used a pre-intervention cohort to assess baseline outcomes such as treatment completion rates and health service utilization and compared these results to a post-intervention cohort following the implementation of IHSS. The findings revealed that program completion rates nearly doubled in the post-intervention group, suggesting that culturally relevant, trauma-informed approaches can significantly enhance engagement and outcomes among Indigenous clients. This study highlights the importance of integrating Indigenous worldviews into treatment and intervention frameworks, especially when aiming to reduce recidivism and promote healing in justice-involved populations (Morin et al., 2022).

Table 12.1 presents an example of a classic experimental design. The designation “R” preceding each group signifies that the researcher randomly assigns participants into groups. Group 1 functions as the experimental group, wherein all participants undergo a pre-test, are exposed to a stimulus, and subsequently complete a post-test. In contrast, Group 2 serves as the control group, with participants receiving only the pre-test and post-test assessments. They are not administered the stimulus; in other words, they are not exposed to the “thing” we want to know the effects of.

An illustrative example of experimental research is presented in the study conducted by McCoy and Major (2003), which investigates individuals’ perceptions of prejudice. In one segment of this research, all participants underwent a pre-test designed to evaluate their levels of depression. The findings from this pre-test indicated no statistically significant differences in depression levels between the experimental group and the control group. Subsequently, participants assigned to the experimental group were exposed to an article that argued that prejudice against their own racial group is both severe and pervasive. Conversely, those in the control group read an article asserting that prejudice against a different racial group is similarly severe and pervasive. Upon conducting a post-test assessment of depression scores, the researchers found that individuals who had engaged with the material highlighting prejudice against their own racial group reported higher levels of depression compared to those who had engaged with material highlighting prejudice against a different racial group.

This research contained all three features of a true experiment and is in fact a classic experiment. It tests the effects of an independent variable/stimulus (the reading) on a dependent variable/outcome (depression) by using pre-tests and post-tests of one experimental group and one control group.

You may be wondering how the findings of this study might apply to Indigenous Peoples. In examining the unique influences of systemic bias and historical trauma on mental health outcomes among Indigenous populations, it is essential to recognize that these communities often encounter compounded challenges related to identity, cultural dislocation, and societal marginalization. Such factors can intensify feelings of depression, particularly when narratives about prejudice resonate with their lived experiences.

To enhance our understanding of these challenges, it is crucial to integrate Indigenous methodologies and perspectives into the research design. This approach not only acknowledges the specific cultural contexts of Indigenous Peoples but also allows for a more nuanced exploration of how perceptions of prejudice impact mental health. By employing methods that prioritize community engagement, storytelling, and culturally relevant frameworks, researchers can gain deeper insights into the mental health experiences of Indigenous populations. This integration can lead to more effective interventions and support systems tailored to their unique needs.

The Solomon four-group design, which is characterized by its inclusion of both control and experimental groups, represents a sophisticated variant of a true experimental methodology. This design is particularly noteworthy due to its incorporation of two additional groups: one that is exposed to the experimental stimulus prior to undergoing a post-test and another that does not receive the stimulus but still participates in the post-test assessment. This design allows for a more nuanced understanding of the effects of the stimulus while controlling for potential confounding variables. In other words, it still allows us to say that no outside factors caused the effect we are trying to measure for.

As illustrated in Table 12.2, random assignment remains a critical component of this design, as it ensures that groups are formed without bias. Specifically, Groups 1 and 2 mirror those found in traditional experimental designs, while Group 3 is subjected to both the stimulus and subsequent post-test evaluation, and Group 4 engages only in the post-test without prior exposure to the stimulus.

Lastly, the post-test-only control group design is recognized as a valid experimental design despite the absence of pre-test measurements. In this framework, researchers engage in the random assignment of participants to both experimental and control groups. Then, the researcher administers a specific stimulus and subsequently evaluates the participants based on the dependent variable. This approach skips the pre-test phase under the premise that randomization effectively mitigates potential biases, thereby rendering pre-testing unnecessary. An appropriate example of a scenario where the post-test-only control group design is suitable – and potential biases are mitigated through randomization – would be a study examining the effectiveness of a new educational intervention on student performance. In this case, researchers might randomly assign students from various classrooms to either an experimental group that receives the new teaching method or a control group that continues with the standard curriculum. Because the assignment is random, any pre-existing differences in student abilities, motivation levels, or classroom environments are likely to be evenly distributed across both groups. Thus, when measuring the dependent variable, such as test scores after the intervention, the researchers can attribute any significant differences in outcomes directly to the educational intervention rather than confounding variables.

A post-test-only control group design is illustrated in Table 12.3. It is important to observe that neither group is administered the pre-test. The first group, designated as the experimental group, is subjected to the stimulus followed by the administration of the post-test. Conversely, the second group, referred to as the control group, is only administered the post-test.

Quasi-Experiments

Quasi-experimental designs share significant similarities with true experimental designs, particularly in their structured approach to investigating causal relationships. However, a defining characteristic that distinguishes quasi-experimental designs is the absence of the random assignment of participants to experimental and control groups. This lack of randomization can stem from various constraints faced by researchers, including limited funding, time restrictions, participant availability, field conditions, organizational constraints, and inherent limitations associated with the specific research topic or question at hand.

For example, in a collaborative effort to evaluate the impact of a revised curriculum within a local police training program, researchers encountered organizational constraints (one of which was the fact that the classes had already started and students had already formed study groups) that precluded the possibility of randomly assigning students to either experimental or control groups.  Consequently, they opted to implement the new curriculum across entire training cohorts rather than individual participants. Following this implementation, they conducted a comparative analysis of post-test results between these cohorts and those that had undergone the traditional curriculum.

The absence of random assignment in quasi-experimental research significantly heightens the likelihood that the group that receives the stimulus – and the group that does not – will be a non-equivalent comparison group. This non-equivalence refers to the presence of critical differences between the groups that may influence the outcomes of the study, and because of this, the term comparison group is typically used instead of control group (Maxfield & Babbie, 2018). For instance, in their investigation into the efficacy of a curriculum modification, it is plausible that the cohorts selected for the experimental group possessed a greater degree of prior knowledge related to the subject matter under examination than their counterparts in the comparison group. This could be due to chance or some bias in group selection.  The implications of such non-equivalence are manifold, potentially compromising the internal validity of experimental studies (i.e., something other than the stimulus could have caused the change in the dependent variable in the post-test). Nevertheless, researchers may have to use a quasi-experimental design when random assignment is not feasible.

Numerous true experimental designs can be transformed into quasi-experimental designs through the exclusion of random assignment, as seen in the non-equivalent groups design exemplified in Table 12.4. When comparing this to the classic experimental design shown in Table 12.1, there is only one clear difference: the designation “R” preceding the groups has been substituted with an “N.” This modification signifies the absence of random assignment among the groups involved in the study. Notwithstanding this alteration, all other methodological components remain consistent.

A distinct variant of quasi-experimental design is the post-test-only control group design, which has been previously discussed. Specifically, in the post-test-only non-equivalent groups design, exemplified in Table 12.5, the researcher implemented a stimulus to the experimental group and subsequently conducted only post-tests on both the experimental and comparison groups to assess the impact of the administered stimulus.

A comparative analysis of this table with the post-test-only control group experimental design shown in Table 12.3 reveals that the sole distinction lies in the notation “N” preceding the groups, signifying non-random assignment.

The research conducted on the effects of a curriculum change in police training employed a quasi-experimental design. In this study, several cohorts were exposed to the newly implemented curriculum, and subsequent post-test results were obtained. These results were then compared with those from other cohorts that had not undergone the new curriculum. The absence of random assignment of participants to groups necessitated the classification of this research as quasi-experimental.

The various forms of experimental and quasi-experimental designs covered so far in this chapter can be adapted by Western scholars when conducting research on Indigenous communities to respect cultural contexts and community leaders and members throughout the research process. For instance, ethical considerations and culturally responsive methodologies can – and should be – integrated into both the selection and assignment processes to ensure respect for Indigenous knowledge systems and community engagement. Indigenous methodologies emphasize community involvement and culturally relevant practices that honor traditional knowledge systems. By incorporating Indigenous frameworks into experimental research, Western scholars can enhance the relevance and applicability of their findings while fostering respectful partnerships with Indigenous communities. This integration allows for a more holistic understanding of crime and justice issues as experienced by Indigenous communities, thereby enriching the overall discourse within criminology.

Experimental Research Conducted by (and with) Indigenous Peoples

While Western scholars have been known to conduct experimental research on Indigenous communities, it is also important to note that Indigenous Peoples have a long-standing tradition of conducting their own forms of experimentation and investigation within their cultural frameworks, frequently drawing upon traditional knowledge systems and practices that enhance our comprehension of human behaviour and social interactions. This approach often contrasts with conventional Western scientific methods, which can overlook or misinterpret Indigenous knowledge systems, but they are nonetheless valid and significant within their own frameworks. In their approach, Indigenous researchers prioritize community involvement and ethical considerations, ensuring that the research is not only beneficial to the community but also conducted in a manner that respects traditional practices and values. For instance, many Indigenous communities employ participatory action research (PAR) as a means of integrating local knowledge with scientific inquiry. As discussed in chapter 9, albeit in the context of field research, this method allows community members to actively participate in the design implementation and analysis of experiments, thereby fostering a sense of ownership over the research process and its outcomes.

As discussed in the introductory chapter, Two-Eyed Seeing is an integrative approach in PAR and community collaboration that brings together Indigenous knowledge and Western science, including experimental research. This approach helps different groups understand each other better and learn from one another. Peltier (2018) has explored the concept of Two-Eyed Seeing (see Figure 12.3 below). The figure illustrates how Two-Eyed Seeing can serve as a framework for conducting research within Indigenous communities. The diagram outlines the conventional research processes associated with a Western paradigm, including planning, implementation, knowledge production, and action. These processes can be transformed to align with an Indigenous paradigm that emphasizes community engagement, capacity building, empowerment, and self-determination. By employing Two-Eyed Seeing and fostering genuine collaboration with Indigenous communities, researchers can effectively validate Indigenous perspectives and adapt the four outlined processes beyond traditional frameworks.

Moreover, Indigenous experimental research often incorporates traditional ecological knowledge (TEK) into scientific frameworks. TEK encompasses the understanding and insights gained from generations of living in harmony with the land and its resources. By blending TEK with experimental designs, Indigenous researchers can address pressing issues, such as climate change, resource management, and health disparities within their communities. For example, studies on sustainable agriculture practices have utilized both quantitative data from controlled experiments and qualitative insights from Indigenous farmers to develop more effective strategies for food security. This integrative approach not only enhances the validity of the findings but also ensures that they are culturally relevant and applicable to the community’s needs.  This is unique from a laboratory approach, which often seeks to isolate variables rather than examine multiple interconnections between variables.

Finally, there is a growing recognition within academic circles of the importance of decolonizing research methodologies to include Indigenous perspectives. This shift has led to increased funding opportunities for Indigenous-led experimental studies and collaborations between universities and Indigenous communities. Such partnerships aim to create a more equitable research landscape where Indigenous voices are amplified, allowing for innovative experimental designs that respect cultural protocols while addressing critical social issues. As these collaborative efforts continue to evolve, they promise to reshape how experimental research is conducted in ways that honour both scientific rigour and Indigenous wisdom.

Strengths and Limitations of Experimental Research

 Strengths

 In the realm of criminological research, experiments present a unique set of advantages and challenges that scholars must meticulously evaluate during the design phase. A notable strength of experimental methodologies, particularly laboratory experiments, lies in the significant degree of control they afford researchers over the variables and conditions to which participants are exposed. This level of control enables researchers to isolate specific factors and assess their effects on behaviours or attitudes, thereby enhancing the internal validity of the findings. In simpler terms, it enables researchers to confidently state that the effect they observe is directly caused by the specific factor they are studying and not influenced by anything else. Furthermore, experiments are typically more amenable to replication compared to other research methodologies, such as observational studies or qualitative interviews. This means that the entire study can be re-created by a future researcher to determine if the same results will be found. The capacity for replication is crucial in establishing the reliability and generalizability of experimental results across diverse populations and contexts.

However, it is important to acknowledge that traditional experimental designs may not fully capture the complexities inherent in studying human behaviour within various cultural frameworks, particularly those of Indigenous Peoples. Indigenous perspectives often emphasize holistic approaches that consider community context, historical trauma, and cultural values. Therefore, when designing experiments involving Indigenous populations, researchers should strive to incorporate culturally relevant methodologies that respect Indigenous knowledge systems and prioritize community engagement, as seen in PAR. This approach not only enhances the ethical integrity of the research but also enriches the findings by ensuring they resonate with the lived experiences of Indigenous individuals.

Limitations

The utilization of experimental methodologies also presents inherent limitations that merit critical examination. One significant critique pertains to the artificiality of experimental settings. This concern is particularly pronounced in laboratory experiments where the controlled environment may not accurately replicate the complexities and nuances of real-world scenarios. Even field experiments, which are ostensibly conducted in naturalistic settings, often fall short of capturing the multifaceted dynamics that characterize everyday life.

A unique challenge associated with field experiments is the diminished control researchers have over various stimuli and contextual factors that may influence participants’ behaviours. This lack of control can lead to variability in responses that complicates the interpretation of results. Furthermore, when the conditions established within an experimental framework diverge from those encountered in broader societal contexts, researchers face significant obstacles regarding the generalizability of their findings. This issue is particularly salient when considering Indigenous populations and perspectives: traditional experimental designs may overlook or misrepresent cultural contexts and values intrinsic to these communities. Consequently, it becomes imperative for researchers to adopt methodologies that are sensitive to Indigenous epistemologies and practices to ensure that their findings are both relevant and applicable within these distinct sociocultural frameworks.

To enhance the rigour and applicability of experimental research in criminology, scholars should engage with Indigenous perspectives and incorporate culturally responsive methodologies that honour the lived experiences and knowledge systems of Indigenous Peoples. By doing so, researchers can better navigate the complexities inherent in human behaviour while fostering a more inclusive understanding of crime and justice.

Another significant concern arises regarding the extent to which researchers can confidently attribute observed effects to the experimental stimulus, as opposed to extraneous variables or contextual factors that may inadvertently influence outcomes. This issue is particularly salient when considering the diverse backgrounds and experiences of participants, including Indigenous Peoples, whose unique cultural perspectives may interact with experimental conditions in unforeseen ways. Moreover, researchers must remain vigilant about potential confounding variables that could distort findings. These may include unanticipated conditions within the experimental framework or longitudinal changes in participant behaviour and attitudes over time. Such dynamics underscore the necessity for a nuanced understanding of how various factors can converge to produce specific outcomes in experimental settings.

While experimental research designs afford researchers a degree of control over conditions and facilitate replication by other scholars, they are not without limitations. The artificiality inherent in controlled settings may compromise ecological validity (i.e., the extent to which research findings can be generalized to real-world settings, meaning that the results of a study are applicable and relevant outside of the controlled experimental environment), raising questions about the generalizability of findings across different populations and contexts. Furthermore, researchers must grapple with the challenge of establishing causal relationships with confidence, particularly when engaging with diverse groups such as Indigenous communities whose lived experiences may diverge significantly from those represented in traditional experimental paradigms.

Conclusion

In this chapter, we reviewed both true experiments and quasi-experiments. True experiments are characterized by random selection of participants from a larger population and random assignment to either experimental or control groups. This methodological rigour enhances the internal validity of findings by minimizing biases that could affect results. The experimental design typically includes several key components: pre-tests, which assess participants’ baseline characteristics or behaviours before any treatment; a stimulus, which is the intervention or condition applied to the experimental group; post-tests, which evaluate the impact of the stimulus after its application; and the establishment of both experimental and control groups. Quasi-experiments, on the other hand, are research designs that aim to evaluate the effect of an intervention or treatment, without random assignment, to experimental (or treatment) and comparison groups, often utilizing existing groups or conditions for comparison.

Integrating Indigenous perspectives into criminological experiments requires thoughtful consideration of cultural contexts throughout all stages – from hypothesis formulation through data collection and analysis – to ensure that research outcomes are meaningful and beneficial for Indigenous populations. This necessitates an understanding of how traditional knowledge systems and community values can influence research design and interpretation. Indigenous methodologies often emphasize relationality, respect for cultural contexts, and community engagement in research processes. Further, the implications of integrating Indigenous perspectives into criminological research are profound. When done thoughtfully, such integration can lead to more accurate representations of crime within Indigenous contexts – and perhaps may inform broader contexts as well – and contribute to more effective interventions tailored to community needs. Moreover, it empowers Indigenous Peoples by validating their knowledge systems and fostering resilience through community-driven solutions. However, if these perspectives are not integrated appropriately, there remains a risk of perpetuating colonial attitudes within criminology, whereby Indigenous voices are sidelined or misrepresented. Thus, it is essential for researchers to remain vigilant about power dynamics in their work and strive for equitable partnerships with Indigenous communities.

Further Reading

• Individual perceptions of climate anomalies and collective action: Evidence from an artefactual field experiment in Malaysian Borneo
• Art in the city reduces the feeling of anxiety, stress, and negative mood: A field study examining the impact of artistic intervention in urban public space on well-being
• Is there ethnic discrimination in Roma children’s access to sports clubs in Hungary? Evidence from field experiments in basketball, volleyball, and soccer
• Revisiting broken windows: The role of neighborhood and individual characteristics in reaction to disorder cues
• Vehicle trajectory at curved sections of two-lane mountain roads: a field study under natural driving conditions

References

Maxfield, M. G., & Babbie, E. R. (2018). Research methods for criminal justice and criminology (8th ed.). Cengage Learning.

McCoy, S. K., & Major, B. (2003). Group identification moderates emotional response to perceived prejudice. Personality and Social Psychology Bulletin, 29, 1005–1017. https://doi.org/10.1177/0146167203253466

Morin, K. A., Marsh, T. N., Eshakakogan, C., Eibl, J. K., Spence, M., Gauthier, G., Walker, J. D., Sayers, D., Ozawanimke, A., Bissaillion, B., & Marsh, D. C. (2022). Community trial evaluating the integration of Indigenous healing practices and a harm reduction approach with principles of Seeking Safety in an Indigenous residential treatment program in northern Ontario. BMC Health Services Research, 22, Article 1045. https://doi.org/10.1186/s12913-022-08406-3

Peltier, C. (2018). An application of two-eyed seeing: Indigenous research methods with participatory action research. International Journal of Qualitative Methods, 17(1), https://doi.org/10.1177/1609406918812346

AI Declaration Statement

Artificial Intelligence Tool: Unknown; Writing – Review & Editing: AI (unknown) was used for proofreading and editing of this chapter. Final proofreading and editing was done by humans.