CLDR logo
The Center for Large Data Research and Data Sharing in Rehabilitation involves a consortium of investigators from the University of Texas Medical Branch, Colorado State University, Cornell University, and the University of Michigan. The CLDR is funded by NIH - National Institute of Child Health and Human Development, through the National Center for Medical Rehabilitation Research, the National Institute for Neurological Disorders and Stroke, and the National Institute of Biomedical Imaging and Bioengineering. (P2CHD065702).

Education & Training: Analyzing Data

Design: Quasi-Experimental Research Overview

Research involving administrative datasets or large national surveys typically lacks one or more of the three design criteria that define rigorous "experimental research" designs: manipulation, randomization, and control. While randomized controlled trials (RCTs) are the epitome of experimental research and remain the gold standard for inferring causation, methodology advances over the past 20 years have greatly increased our interest in and understanding of quasi-experimental or "observational "research. A major advantage of existing claims or survey data is that they reflect routine practice for large and representative populations, in contrast to the much smaller and often healthier patient populations recruited in clinical trials. In other words, these datasets capture the characteristics and experiences of everyday patients in everyday clinical settings. Moreover, these resources provide the only way to assess policy- or practice-related changes, the so-called "natural experiments."

The fundamental strength of RCTs is the primary criticism of quasi-experimental research: internal validity - the degree to which the relationship between the treatment and outcome is free from the effects of extraneous factors. However, treatment decisions in practice are not randomly assigned. Rather, factors such as prognosis, patient - and provider-preferences, insurance coverage, and out-of-pocket costs influence who gets what treatment. Thus, socio-demographic and clinical characteristics are not balanced between treated and untreated cohorts. External validity - degree to which the results can be generalized to persons or settings outside the experimental situation - is generally less of a concern in observational studies since the experimental situation is routine patients receiving routine care.

When independent variable manipulation and random assignment are beyond the control of the investigator, there are four other design parameters that can strengthen a study's internal validity:

  1. Cohort identification (incident vs. prevalent users)
  2. Control or "counterfactual" group
  3. Pre-period measurement
  4. Post-period measurement

Citations

NF Marko & RJ Well (2010) The role of observational investigations in comparative effectiveness research. Value in Health. 13(8): 989-997.

S Schneeweiss & J Avorn (2005) A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 58: 323-337.

E von Elm et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 147(8): 573-577.

Other Links

STROBE Statement Website STROBE stands for an international, collaborative initiative of epidemiologists, methodologists, statisticians, researchers and journal editors involved in the conduct and dissemination of observational studies, with the common aim of STrengthening the Reporting of OBservational studies in Epidemiology.

Video

Rigorous Quasi-Experimental Comparative Effectiveness Research Study Design by Professor Matthew Maciejewski from Duke University and the Center for Health Services Research at Durham VA Medical Center. This 60 min video recorded during the Comparative Effectiveness Research with Population-Based Data conference in the Baker Institute at Rice University on July 13, 2012.


The CLDR is funded by the National Institutes of Health (grant# P2CHD065702). See About Us for more details.

Give Website Feedback