Decisions about research design are layered and connected to one another. Our approach and research purpose, as well as our research question and the data available to answer that question, will guide many of these decisions. In the previous chapter, we introduced you to the measurement process and how conceptualization and operationalization require us to become very specific about what we mean by the concepts we are studying and how exactly we plan to measure and observe those concepts. What a variable is, how different variables are related to one another, and the level of measurement of our variables were reviewed.

In this chapter, we will elaborate on the relationships between variables and how we can determine if one variable did indeed cause the other. We will also introduce the time dimension and describe various time-connected research designs. This discussion will also explain how the time dimension in our study impacts our ability as researchers to say whether our findings indicate that there is a causal relationship between our variables of interest. We will end this chapter with a discussion of validity, with a specific focus on the types of validity relevant to our discussion on research design and causation.

Causality

One important part of the measurement process involves attending to issues of cause and effect. Causality refers to the idea that one event, behaviour, or belief will result in the occurrence of another subsequent event, behaviour, or belief. To assess causality, we must first categorize our variables as independent or dependent.  An independent variable is a variable the researcher thinks might be useful in predicting or explaining this dependent variable.  A dependent variable is the outcome variable the researcher has chosen to focus on. Often, researchers use more than one independent variable to collectively help us understand the dependent variable.

Knowing whether your variable is independent or dependent is important as we want to determine if one has an impact on the other. For instance, let us suppose you are researching whether the age of the defendant influences juror decisions of guilt. You develop a scenario in which you describe a fictitious robbery case, changing the age to create four otherwise identical scenarios (ages 15, 30, 45, and 60). You present the scenarios to participants and ask them to assess whether the person is guilty or not guilty based on the facts of the case. In this research, your independent variable is the age of the defendant, and the dependent variable is the assessment of guilt because your hypothesis is that the age of the defendant influences the assessment of guilt (for more on hypotheses, review chapter 3 and chapter 4).

After identifying our independent and dependent variables, we must establish that there is a relationship – or correlation – between the two variables. This assessment can be done statistically, such as through a correlation analysis, which simply determines if and how two variables are related to each other: when one is present, the other is as well. Establishing a relationship can also be done theoretically or based on previous research. For the hypothetical study above, we would need to connect the variables of defendant age and assessment of guilt; existing research has already established this relationship. Research studies using mock jurors have shown that demographic variables of the defendant, such as their age, gender, and ethnicity, do influence the assessment of guilt (e.g., Barager et al., 2025; Pica et al., 2023).

Establishing a relationship between two variables also involves speaking to the directionality  of that relationship. More specifically, we must be able to say that when one variable increases or decreases, the other variable increases or decreases. A positive correlation exists when variables change in the same direction (i.e., both increase or both decrease). Conversely, a negative correlation exists when variables move in opposite directions. For instance, a study conducted by Heydon (2018) is a great example of a finding of a negative correlation. They found that increased meaningful consultation with First Nation communities regarding oil sands projects resulted in less environmental harm and social unrest. In other words, as consultation went up, environmental harm and social unrest went down.

Specific criteria need to be satisfied to say with confidence that the relationship between two variables moves beyond mere correlation (or relationship) toward causation. Establishing causation is much more straightforward in the physical sciences. For instance, we know that if we heat water to 100 degrees Celsius, it will boil each and every time, and that heating water to this temperature must occur before boiling will occur. But because the social sciences deal with complex human behaviours, identifying a relationship between two variables is rarely this clear-cut. One can say, for example, that a traumatic childhood is believed to be related to a person’s substance abuse in adulthood, but we can’t say that the trauma causes substance abuse in every instance. There are many other variables at play, and not everyone responds to trauma in the same way.

The main criteria for establishing causality include plausibility, temporality, and spuriousness. Plausibility refers to the degree that a claim that one event, behaviour, or belief causes another is reasonable and makes sense. For example, if a researcher observes a series of courtroom interactions during which a Crown prosecutor routinely talks over a criminal defense attorney, the researcher might begin to wonder whether prosecutors who have a propensity to speak loudly are more likely to have a propensity to interrupt other lawyers. However, the fact that there might be a relationship between the volume of a person’s voice and their propensity for talking over other people does not mean that a prosecutor’s loud voice causes them to talk over defense attorneys. In other words, just because there might be some correlation between two variables does not mean that a causal relationship between the two is really plausible. On the surface, it simply doesn’t make sense. This criterion is two-pronged: not only must the two variables be related, but that correlation must be plausible.

The criterion of temporality means that a independent variable must precede the dependent variable in time. Let’s return to the study by Kohm et al. (2012) discussed in chapter 6a, wherein the researchers examined exposure to news media and fear of crime via a survey of both Canadian and American undergraduate students. They identified their independent variable as the news media source participants engaged with most for crime news and how much news they consumed. Their dependent variable was the fear of being the victim of crime, broken into 10 specific crimes. While Kohm et al. (2012) were able to state that there was a connection between their variables, specifically between viewing local news media about crime and higher fear of crime, they could not claim causality as they could not isolate the time element; there was no way to determine whether high levels of fear of crime came before or after exposure to crime news. The criterion of temporality is challenging to meet and should be considered in the methodological design of studies focusing on causality, as seen in studies whose purpose is explanation.

Finally, a spurious relationship is one in which an association between two variables appears to be causal but can in fact be explained by some third variable. Let’s consider a real-world example of spuriousness. Did you know, for example, that high rates of ice cream sales have been shown to cause drowning? Of course, that’s not really true, but there is a positive relationship between the two. In this case, a third variable, time of year, causes both high ice cream sales and increased deaths by drowning because the summer season brings increases in both. In other words, the presence of a third variable explains the apparent relationship between the two original variables. For more (often humorous) examples of spurious relationships, see Spurious Correlations.

In sum, the following criteria must be met for a correlation to be considered causal:

1. There must be a correlation (positive or negative), and that correlation must be plausible.
2. The cause must precede the effect in time.
3. The relationship must be nonspurious.

The Time Dimension

As indicated above, temporality is one of the criteria for determining causality. Indeed, the decisions we make as researchers about how to incorporate time into our research design are relevant to this discussion about causality: do we want to examine the social phenomenon that is the focus of our research question at one point in time, or do we want to examine it over a long period? This decision will impact our ability to say whether there is a change to be observed and whether our independent variable resulted in that change in the dependent variable. There are four main types of time-related studies: cross-sectional, longitudinal, retrospective, and prospective. Each type of study design can utilize a variety of data collection methods, again depending on the research question.

The most basic of these time-related study designs is the cross-sectional research design. Cross-sectional studies are focused on a single point in time, and they offer researchers a snapshot of respondents’ lives, opinions, and behaviours at the time when the data were collected. Exploratory and descriptive studies tend to be cross-sectional in design, and common data collection methods include interviews, focus groups, surveys, and observations.

Let’s consider an example of this type of research design to illustrate how it might work. Stevenson et al. (2018) used a survey to collect data from domestic violence shelter staff about programs and services available through the shelter for the pets of survivors of intimate partner violence. The survey focused on what was available at the time the survey was completed, capturing data at a moment in time. The study found that while there was an awareness of the challenges that survivors with pets faced (e.g., delays in leaving the relationship, fear of harm to the pets), there were few established pet-related programs in place.

To illustrate this study design further, let’s take a look at another example, this time one that’s hypothetical. Let’s say we wanted to conduct interviews in 2019 that asked about people’s perceptions of the police. In the summer of 2020, the death of George Floyd at the hands of police officer Derek Chauvin sparked national (and even international) protests. Imagine how responses to the same set of questions might have been different if people had been interviewed during or after that summer. This difference connects to one of the challenges of cross-sectional studies in that the events, opinions, behaviours, and other phenomena that cross-sectional studies are designed to assess do not generally remain stagnant. In addition, such studies often do not include any follow-up with participants to see if or how anything has changed. While cross-sectional surveys have many important uses, researchers must remember that a cross-sectional survey captures a snapshot of life and opinions as they were at the time that the survey was administered. When employing a cross-sectional design, we do not make any attempt to measure changes in perceptions over a period of time or to determine what might be the causes for those changes. We are simply describing what “is” at a particular time and in a particular place. As such, the cross-sectional design is most appropriate when our research purpose is exploratory or descriptive.

The longitudinal research design tries to overcome this problematic aspect of cross-sectional studies in enabling the collection of data over an extended period of time. Data collection methods – whether they are observations, interviews, surveys, etc. – are administered multiple times. The differences between cross-sectional and longitudinal research designs are shown in Figure 6b.1 below.

There are three types of longitudinal studies: trend, panel, and cohort studies.

Researchers conducting trend design studies are interested in how people’s inclinations change over time. A distinct feature of trend studies is that the same people may not be answering the researcher’s questions each time. For example, the Canadian General Social Survey (GSS) – Canadians’ Safety has been conducted every five years since 1988, focusing on experiences of victimization as well as perceptions of crime and the justice system (Statistics Canada, 2022). In the 2019 GSS, over 22,000 respondents over the age of 15 completed the survey. It is unlikely that the respondents who completed the survey in 2019 also completed the survey in 2014 or in 2024. Rather than focusing on change in an individual’s specific experiences or what may be the reasons or causes for that change, the GSS instead is interested in the overall trends over time. Because of this more overall focus, it is not important that the same people participate in trend studies each time.

The second type of longitudinal study design offers a middle ground between trend and panel studies. In a cohort design study, a researcher identifies a category of people of interest and then regularly includes people who fall into that category. For example, researchers may identify people of specific generations (e.g., all born in the 1960s) or graduating classes, people who began work in a given industry at the same time, people who started university the same year, or perhaps people who have some specific life experience in common. Similar to a trend study, the same people don’t necessarily participate from year to year, but all participants must meet the categorical criteria for inclusion in the study.

Lastly, in a panel design study, the exact same people participate in the study each time data are collected. For this reason, panel studies can be difficult and costly. Imagine trying to conduct interviews with the same 100 people every year for, say, five years in a row. Keeping track of where people live, when they move, and when they die takes resources that researchers often don’t have. When the time and resources are available, however, the results can be quite powerful. In fact, if we observe that a person has changed their views on the topic that is the focus of our study from time point 1 to time point 2, we can ask them why they changed. In other words, the panel design is the only one that allows us to talk about causality to some degree. If the purpose of your research moves beyond mere exploration and description and you are focused on actually explaining something, then this design would be appropriate. Again, we cannot reiterate this enough: all of these research design decisions depend on what your goals are and what your research question is!

The University of Minnesota’s Youth Development Study (YDS) offers an excellent example of a panel study (Mortimer, 2023). Since 1988, YDS researchers have administered an annual survey to the same 1,000 people. Study participants were in ninth grade when the study began, and they are now in their thirties. Several hundred papers, articles, and books have been written using data from the YDS. These data not only outline changes over this time period, but they also allow us to observe changes in specific participants and ask questions to determine the causes for the observed changes.

All three types of longitudinal studies share the strength of allowing a researcher to make observations over time. This means that if the behaviour or other phenomenon of interest changes over time, either because of some world event or because people age, the researcher will be able to capture those changes. However, participant attrition is a key disadvantage of a longitudinal study design. Participant attrition (also called participant mortality) occurs when participants drop out of the study, either because they wish to, they move and are no longer able to be contacted by the researcher, or they die (Carr et al., 2020). Generally, researchers attempt to account for this in their initial sample size, aiming to recruit more participants than is necessary at the beginning to compensate for a degree of attrition.

Longitudinal study designs are prospective in nature: they allow us to measure our variables of interest at one time and then follow up with respondents, either the same people or different ones, depending on the type of research design, at a later point in time. The final time-connected study design is a retrospective design study, where you ask people about their past experiences (Maxfield & Babbie, 2018). Let’s take a look at an example to illustrate the difference between these two research designs. Let’s say your goal is to determine how many current offenders who struggle with substance abuse witnessed substance abuse as children. You could ask them in an interview, a focus group or a survey to share childhood experiences that speak to their exposure to substance abuse in their formative years. This study would take a retrospective approach.  Alternatively, if you wanted to examine the same issue of the impacts of substance use exposure on children, you could instead identify a subgroup of children in families in which substance use has been established, then follow those children throughout their lives and identify how many of them later also use or abuse illicit substances. This study is an example of a prospective approach in a longitudinal panel study design.

The challenge with retrospective studies is that memories are imperfect, and there may be missing or misremembered details. The passage of time and new experiences also colour memories. The General Social Survey (GSS) also has a retrospective element to it. For example, in the survey question block dealing with intimate partner abuse, respondents are asked to think about whether they have experienced various forms of victimization in the past five years (Statistics Canada, 2020). Answers could be inaccurate, as respondents may report on events that happened longer than five years ago, which could call into question the validity of the responses.

The challenge with prospective studies, as mentioned earlier, is that participant attrition is a primary concern. There are also a number of intervening variables that can be difficult to control over time within the research design. In the above example exploring the relationship between substance abuse exposure and later substance abuse, being exposed to sports or other prosocial extracurricular activities may change the trajectory of some children, or perhaps they have wonderful teachers and mentors along the way who offer support and encouragement that influences the choice to avoid substance use. Our ability to make causal inferences is hindered by such intervening variables.

The Connection Between Validity and Research Design

This discussion about whether we are correct when we say that one variable indeed causes another, or whether they are simply correlated, touches on an important aspect of research design: validity. Essentially, validity addresses the issue of whether the claims we make about our measures are accurate and whether our statements about causality are indeed correct. In other words, validity addresses the truthfulness of our claims.

In upcoming chapter 6c, we discuss evaluating the quality of our measures – i.e. their validity and reliability. There are two specific types of validity that relate to the design of our research: internal validity and external validity.

Internal validity speaks to the criterion of causality that addresses spuriousness (criterion 3), concentrating on whether a variable (or variables) other than our independent variable is actually causing the change in the dependent variable. In other words, is some other variable coming “into” our study that we had not intended and is that “thing” actually causing the observed change?

Certain types of research methods increase the internal validity of our research, such as the experiment. In an experiment conducted in a laboratory under precise conditions, researchers have a great deal of control over what factors come into play while conducting the study. For example, participants confined to a laboratory for a full day with no access to the outside world would not have any outside influences. But, there are other factors that may come into play, such as one participant feeling ill, or another participant who is distracted by a lack of sleep the night before. While we can design our research to increase internal validity, we can never be absolutely 100% positive in a causal relationship.

External validity, on the other hand, is concerned with how generalizable our research findings are to the outside world. The question we are asking when we are dealing with external validity is: Can our findings about some variable causing change in another variable apply to some other situation or location? You may recall the rice alcohol study discussed in chapter 3 of this text. This study examined the use of rice alcohol in the Downtown Eastside of Vancouver in the late 1990s and early 2000s. The issue of external validity is relevant in this study: Would the findings about the impacts of the policy change also apply if the consumption of rice alcohol became an issue in Toronto? What about Calgary? What about if it happened again, now, in Vancouver? Studies like this that are conducted in a real-world setting, and where variables are at play much like in other real-world settings, typically have high external validity, especially if the study is conducted in another city with similar demographic and geographic characteristics. They can be generalized to other situations with similar conditions. But when a study is conducted in a highly controlled laboratory experiment, on the other hand, external validity is low as it is unlikely that what we learn in a laboratory would actually apply in real life.

As you can see, some methods have high external validity but low internal validity, while others have high internal validity but low external validity. It is not the case that one is always good and the other is not. These are all things to simply keep in mind when choosing your research method to ensure that the method you choose is the most appropriate to fulfill your research purpose and answer your research question.

Conclusion

In this chapter, we elaborated on aspects of measurement that pertain specifically to causality. We discussed what needs to be established to say that our variables of interest are more than just co-related and that one actually causes the other. The type of research design we choose and how we incorporate the time element will impact our ability to say whether one variable in our study causes the other. Cross-sectional studies provide a snapshot of data at one time and in one place, and they are an appropriate design if the purpose of our research is to describe or explore some phenomenon, but not if we want to prove a cause-and-effect relationship conclusively. Longitudinal studies, on the other hand, collect data over multiple time periods. The panel design allows us to explore changes over time, with a particular eye to explaining cause-and-effect relationships.

Lastly, we touched on two different types of validity: internal and external validity. Some other types of validity will be reviewed in the next chapter along with ways to evaluate the quality of our chosen measures.

References

Barager, R. T., Thompson, L. E., & Pozzulo, J. (2025). Perceptions in a sexual assault trial: The influence of age and race on Canadian mock-jurors. Journal of Police and Criminal Psychology, 40(1), 128-140. https://doi.org/10.1007/s11896-024-09721-7

Kohm, S. A., Waid-Lindberg, C. A., Weinrath, M., Shelley, T. O. C., & Dobbs, R. R. (2012). The impact of media on fear of crime among university students: A cross-national comparison. Canadian Journal of Criminology and Criminal Justice, 54(1), 67-100. https://doi.org/10.3138/cjccj.2011.E.01

Maxfield, M. G., & Babbie, E. R. (2018).  Research methods for criminal justice and criminology (8th ed.). Cengage Learning.

Mortimer, J. T. (2023, September 28). Youth Development Study, 1988-2020 (ICPSR 24881, version 5) [Project description]. ICPSR. https://doi.org/10.3886/ICPSR24881.v5

Pica, E., Hildenbrand, A., Fraser, L., & Pozzulo, J. (2023). Juror decision-making in a child trafficking case: The impact of defendant and victim gender, defendant age, and defendant status. Journal of Interpersonal Violence, 38(17-18), 10031-10054. https://doi.org/10.1177/08862605231169760

Statistics Canada. (2020, November 26). General Social Survey, 2019: Canadians’ Safety [Questionnaire]. https://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=getInstrumentList&Item_Id=1236284&UL=1V

Statistics Canada. (2022). 2019 General Social Survey: Canadians’ Safety: Technical Report. https://www.statcan.gc.ca/en/statistical-programs/document/4504_D1_V1#

Stevenson, R., Fitzgerald, A., & Barrett, B. J. (2018). Keeping pets safe in the context of intimate partner violence: Insights from domestic violence shelter staff in Canada. Affilia: Journal of Women and Social Work, 33(2), 236-252. https://doi.org/10.1177/0886109917747613