Research Methodology Group UOPX Research Community Causal-Comparative Research

Research Methodology Group

UOPX Research Community

Causal-Comparative Research

April 20, 2023

Dr. Frederick P. Lawrence

Topics in Research Designs

• Correlational Research

– March 2, 2023

• Causal-Comparative Research

– April 20, 2023

• Repeated Measures Research

– July 13, 2023

• Regression Research

– August 3, 2023

Agenda

• Definitions of Causal-Comparative Research

• Comparisons with Correlational Research

• Comparisons with Experimental Research

• Causal-Comparative Research Purpose Statements

• Causal-Comparative Research Questions and Hypotheses

• Examples of Causal-Comparative Research

• References

• Discussion and Q&A

Definition of Causal-Comparative

Research

“Causal-comparative research is a methodology used to identify cause-effect

relationships between independent and dependent variables.

Researchers can study cause and effect in retrospect. This can help determine the

consequences or causes of differences already existing among or between

different groups of people.

When you think of Casual Comparative Research, it will almost always consist of

the following:

• A method or set of methods to identify cause/effect relationships

• A set of individuals (or entities) that are NOT selected randomly – they were

intended to participate in this specific study

• Variables are represented in two or more groups (cannot be less than two,

otherwise there is no differentiation between them)

• Non-manipulated independent variables – *typically, it’s a suggested

relationship (since we can’t control the independent variable completely)”

(https://www.questionpro.com/blog/causal-comparative-research/)

Definition of Causal-Comparative

Research

(cont.)

“Casual Comparative Research is broken down into two types:

Retrospective Comparative Research: Involves investigating a particular

question…. after the effects have occurred. As an attempt to see if a specific

variable does influence another variable.

Prospective Comparative Research: This type of Casual Comparative Research is

characterized by being initiated by the researcher and starting with the causes and

determined to analyze the effects of a given condition. This type of investigation is

much less common than the Retrospective type of investigation.”

(https://www.questionpro.com/blog/causal-comparative-research/)

“A causal-comparative design is a research design that seeks to find relationships

between independent and dependent variables after an action or event has already

occurred. The researcher's goal is to determine whether the independent variable

affected the outcome, or dependent variable, by comparing two or more groups of

individuals … causal-comparative research [is] also referred to as ex post facto

research …”

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

Comparisons with Correlational

Research

• Similarities:

• Both methods are useful when experimental research has been deemed

impossible or unethical as the research design for a particular question.

• Both designs attempt to determine relationships among variables, but

neither allows for the actual manipulation of these variables.

• Thus, neither can definitively state that a true cause-and-effect relationship

occurred between these variables.

• Finally, neither type of design randomly places subjects into control and

experimental groups, which limits the generalizability of the results.

• Differences:

• In causal-comparative research, the researcher investigates the effect of an

independent variable on a dependent variable by comparing two or more

groups of individuals.

• In correlational research, the researcher works with only one group of

individuals. Instead of comparing two groups, the correlational researcher

examines the effect of one or more independent variables on the dependent

variable within the same group of subjects.

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

Comparisons with Experimental

Research

• Similarities:

• Unlike correlational research, both experimental research and causal-

comparative research typically compare two or more groups of subjects.

Research subjects are generally split into groups on the basis of the

independent variable that is the focus of the study.

• Goal of both types of research is to determine what effect the independent

variable may or may not have on the dependent variable or variables.

• Differences:

• In true experimental research designs, the researcher manipulates the

independent variable in the experimental group. Because the researcher has

more control over the variables in an experimental research study, the

argument that the independent variable caused the change in the dependent

variable is much stronger.

• In causal-comparative research, the research subjects are already in groups

because the action or event has already occurred, whereas subjects in

experimental research designs are randomly selected prior to the manipulation

of the variables. Random sampling allows for wider generalizations to be

made from the results of an experimental research study.

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

Comparison of Causal-Comparative,

Correlational, and Experimental Research

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

Causal-Comparative Research

Purpose Statements

Elements of Purpose Statements:

• Type of quantitative research (e.g., quasi-experimental, ex post facto, …)

• Research variables

• Location of study [e.g., business or organization (sector or specific), place, …)

Frequently used wording:

The purpose of this quantitative causal-comparative [or, ex post facto] research

study is to determine if variable A affects variable B within the business [or,

organizational] sector.

The purpose of this quantitative causal-comparative [or, ex post facto] research

study is to identify the cause-effect relationship between independent variable A

and dependent variable B for individuals in the business [or, organization].

The purpose of this quantitative causal-comparative [or, ex post facto] research

study is to determine if treatment variable A will lead to the outcome [response]

variable B at location [or, place].

Causal-Comparative Research

Questions (RQ) and Hypotheses

Frequently used wording:

RQ: How does variable A affect variable B in [location of study]?

RQ: What is the cause-effect relationship between independent variable A and

dependent variable B in [location of study]?

RQ: How does treatment variable A lead to outcome [response] variable B in

[location of study]?

Hypotheses:

H0: Variable A does not affect variable B in [location of study].

H1: Variable A weakly [or moderately, or strongly] affects variable B by [brief

description of the mechanism] in [location of study].

… and similarly

Example of Causal-Comparative

Research (1)

Article title:

“Academic Procrastination and Performance in Distance Education: A Causal-

comparative Study in an Online Learning Environment”

Authors:

Hasan Ucar, Anadolu University, Turkey

Aras Bozkurt, Anadolu University, Turkey

Olaf Zawacki-Richter, University of Oldenburg, Germany

Journal:

Turkish Online Journal of Distance Education, October 2021, Volume 22,

Number 4, Article 2, 12 pages

Example of Causal-Comparative

Research (1)

(cont.)

Problem:

• To know the procrastination behaviors of the learners in online distance

learning in order to take the necessary precautions in addressing academic

procrastination tendency, as it is significantly related to course performance

and accomplishment.

Purposes:

• To explore whether male and female learners in online distance learning would

have different academic procrastination tendencies.

• To examine whether different procrastination scores produced differences in

academic performance.

• To explore whether academic procrastination tendency was able to predict the

academic performance of online learners.

Example of Causal-Comparative

Research (1)

(cont.)

Research Questions and Hypotheses:

RQ 1. Do male and female learners significantly differ in their level of

procrastination tendencies?

Research hypothesis 1: Online male and female learners will score the same

on academic procrastination.

RQ 2. Do procrastination tendency differences produce differences in the academic

performance of online learners?

Research hypothesis 2: Online learners who scored lower on academic

procrastination tendency will have better academic performance than that of

those who scored higher on academic procrastination tendency.

RQ 3. Does academic procrastination tendency predict the academic performance

of online learners?

Research hypothesis 3: The academic procrastination tendency of online

learners will predict their academic performance.

Example of Causal-Comparative

Research (1)

(cont.)

Methodology

• Research design: quantitative causal-comparative

• Participants:

• 1,200 online undergraduate learners taking an online English course at a state

university in Turkey were invited to take an online survey

• 333 survey responses (after adjustments) [27.75%]

• Instrument:

• Turkish version of the Tuckman Procrastination Scale (1991)

• Widely used, 14-item scale to measure the academic procrastination tendency

of learners

• Data analysis:

• Independent samples t-tests to compare means of:

• Male and female learners’ level of procrastination tendencies (RQ 1)

• Low and high procrastinators, to determine whether there was a statistical

difference in their academic performance scores (RQ 2)

• Simple linear regression (SLR) model to predict the relationship between

academic procrastination tendency and academic performance (RQ 3)

Example of Causal-Comparative

Research (1)

(cont.)

Results

• RQ 1:

• Although the female learners’ average mean score (M = 34.8, SD = 4.1)

was slightly higher than the male learners’ average mean score (M = 34.1,

SD = 5.2), the difference was not statistically significant (t(331) = 1.18,

p = 0.23) in terms of academic procrastination.

• Research hypothesis 1 was confirmed.

• RQ 2:

• On average, low procrastinators had better academic performance (M =

54.76, SD = 10.6, SE = 0.83) than that of high procrastinators (M = 49.65,

SD = 10.5, SE = 0.81).

• This difference was statistically significant (t(331) = 4.38, p < 0.01) with

close to a medium-sized effect (d = 0.48).

• Research hypothesis 2 was confirmed.

Example of Causal-Comparative

Research (1)

(cont.)

Results (cont.)

• RQ 3:

• SLR suggested that learners’ academic procrastination tendencies

explained 2.3% of the variance (R

= 0.023; F(1,131) = 7.808, p < 0.05).

• In other words, the learners’ academic procrastination tendencies

significantly predicted academic performance (B = -0.35, t = -2.79, p < 0.05).

• Research hypothesis 3 was confirmed.

Findings

• Based on age span and socio-economic status of the undergraduate learners,

future research should consider different demographics to better examine and

gain more insight into the academic procrastination phenomenon.

• Low procrastinators had better academic performance [consistent with literature].

• Online learners’ academic procrastination tendencies significantly predicted

academic performance [consistent with literature].

Example of Causal-Comparative

Research (2)

Article title:

“STEM Certification in Georgia’s Schools: A Causal Comparative Study Using

the Georgia Student Growth Model”

Authors:

David E. Proudfoot, University of Phoenix

Michael Green, University of Phoenix

Jan W. Otter, University of Phoenix

David L. Cook, University of Phoenix

Journal:

Georgia Educational Researcher, 2018, Volume 15, Issue 1, Article 2

Example of Causal-Comparative

Research (2)

(cont.)

Problem:

• As Georgia schools become STEM certified, to understand how certification

has influenced achievement in math and science as well as important

non-STEM disciplines such as English language arts and social studies.

Purpose:

• To better understand the early results of current Georgia schools who

receive either the GaDOE STEM Certification or the AdvancED STEM

Certification to guide administrators and teachers in school reform and to

improve efforts to prepare students to be college and career ready in a

globally competitive society.

Example of Causal-Comparative

Research (2)

(cont.)

Research Questions:

RQ 1: Is there a difference in the median growth percentiles in fourth grade

English language arts for STEM certified schools when compared to non-STEM

schools?

RQ 2: Is there a difference in the median growth percentiles in fourth grade

math for STEM certified schools when compared to non-STEM schools?

RQ 3: Is there a difference in the median growth percentiles in fourth grade

science for STEM certified schools when compared to non-STEM schools?

RQ 4: Is there a difference in the median growth percentiles in fourth grade

social studies for STEM certified schools when compared to non-STEM schools?

Example of Causal-Comparative

Research (2)

(cont.)

Research Questions (cont.):

RQ 5: Is there a difference in the median growth percentiles in fifth grade

English language arts for STEM certified schools when compared to non-STEM

schools?

RQ 6: Is there a difference in the median growth percentiles in fifth grade

math for STEM certified schools when compared to non-STEM schools?

RQ 7: Is there a difference in the median growth percentiles in fifth grade

science for STEM certified schools when compared to non-STEM schools?

RQ 8: Is there a difference in the median growth percentiles in fifth grade

social studies for STEM certified schools when compared to non-STEM schools?

Example of Causal-Comparative

Research (2)

(cont.)

Methodology

• Research design: quantitative causal-comparative

• Participants:

• Grades 4 and 5 English language arts (ELA), mathematics, science, and social

studies classes in STEM and non-STEM schools in Georgia

• Measurement:

• Median growth percentiles (MGP) between STEM certified and non-STEM

schools as developed through the Georgia School Growth Model (GSGM)

• Data analysis:

• A purposive sampling technique was used within the Metropolitan Regional

Educational Service Agency (Metro RESA).

• Since the data were not normal for many of the subgroups, the non-parametric

statistical analysis technique, Mann-Whitney U test, which evaluates whether the

mean ranks for two groups differ significantly from each other, was used.

• Homogeneity of variances permitted the use of a pooled variance between

the non-STEM and STEM certified data.

• When the homogeneity assumption did not hold for a set of comparative

groups (Grade 5 ELA), a random sample was taken from the non-STEM

group that was equivalent to number of STEM schools identified for study:

homogeneity failed, so the median test was used (not highly conclusive).

Example of Causal-Comparative

Research (2)

(cont.)

Results

• Fourth grade ELA was the only group to show a significant difference in MGPs

between the STEM certified schools (Mdn = 62.50) and the non-STEM schools

(Mdn = 52), U = 2122.00, p = .004, 2-tailed, η

= .02 [estimate of effect size, meaning

about 2% of total variance of fourth grade ELA MGPs was explained by STEM

certified schools compared to non-STEM schools, i.e., low or small effect size].

• Interestingly, both the fourth and fifth grade social studies groups had a higher mean

rank for the non-STEM groups than the STEM groups.

• All other content areas showed a higher mean rank for the STEM groups.

• The greatest differences between the STEM and non-STEM groups were in the area

of ELA for both and fourth and fifth grades.

Findings

• Based on the statistical differences noted between STEM certified and non-

STEM schools, support was provided for completing a STEM school certification

process in order to increase MGPs for fourth grade ELA.

• No support was provided for completing a STEM school certification process in

order to increase MGPs for other content areas in other grade levels. [Surprising,

based on anticipated growth, given the available literature at the time.]

References

Brewer, E.W., & Kubn, J. (2012). Sage research methods. In Encyclopedia of

research design. SAGE Publications, Inc. https://doi.org/10.4135/9781412961288

Johnson, B. (2000, April 24-28). It's (beyond) time to drop the terms causal-

comparative and correlational research in educational research methods textbooks

[Conference paper]. Annual Meeting of the American Educational Research

Association, New Orleans, LA, United States. https://eric.ed.gov/?id=ED445010

Proudfoot, D.E., Green. M., Otter, J.W, & Cook, D.L. (2018). STEM certification in

Georgia’s schools: A causal comparative study using the Georgia Student Growth

Model, Georgia Educational Researcher, 15(1).

https://digitalcommons.georgiasouthern.edu/gerjournal/vol15/iss1/2

Ucar, H., Bozkurt, A., & Zawacki-Richter, O. (2021, October). Academic

procrastination and performance in distance education: A causal-comparative study

in an online learning environment. Turkish Online Journal of Distance Education,

22(4). https://dergipark.org.tr/en/pub/tojde/issue/65206/1002726

For Further Help: Quantitative Office Hours (go to

Home page of Research Methodology Group in MS Teams)

Discussion and Q&A