The Lens

Why Seeing Causality Requires More Than Just Looking

Health Economics and Policy

Spring 2026

The Lens

The Most Important Evidence Is Always Missing

The Central Problem

Every day, health policy decisions affect millions of lives. But how do we know which policies actually work?

The answer isn’t obvious — because the most important evidence is always missing.

This lecture builds the lens you’ll use for the rest of the semester to see through the noise and identify genuine causal effects.

A lens that reveals what the naked eye cannot: causation hiding behind correlation.

The Missing Counterfactual (Visual)

Show R Code 
ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.8, ymin = 2, ymax = 6.8,
           fill = scales::alpha(secondary_teal, 0.18), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 2.7, y = 6.2, label = "Observed Outcome",
           color = secondary_teal, fontface = "bold", size = 5) +
  annotate("text", x = 2.7, y = 4.2, label = "Y | D = 1", color = secondary_teal, size = 4.8) +
  annotate("rect", xmin = 5.2, xmax = 9.4, ymin = 2, ymax = 6.8,
           fill = scales::alpha(slate_gray, 0.08), color = slate_gray,
           linewidth = 1, linetype = "dashed") +
  annotate("text", x = 7.3, y = 6.2, label = "Missing Counterfactual",
           color = slate_gray, fontface = "bold", size = 5) +
  annotate("text", x = 7.3, y = 4.2, label = "Y^0 ?", color = slate_gray, size = 5) +
  annotate("segment", x = 4.9, xend = 5.1, y = 4.2, yend = 4.2,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1) +
  xlim(0, 10) + ylim(1, 7.6) +
  theme_diagram()

Figure 1: We observe one world and infer the other

Today’s Agenda — Building the Lens

  1. The Credibility Crisis — Why most published findings are wrong, and what changed
  2. The Naked Eye — What correlation shows, and what it hides
  3. Building the Lens — The potential outcomes framework
  4. Distortion — How selection bias warps what we see
  5. Correction — How randomization makes the lens clear
  6. Protection — SUTVA and the conditions for valid inference

Most Published Research Findings Are False

Ioannidis (2005) — “Why Most Published Research Findings Are False”

The Broken Lens

Scientific research was producing unreliable results because the lens was distorted by small samples, flexible designs, and researcher incentives.

  • Small study sizes amplify noise
  • Flexible study designs allow specification searching
  • Financial and personal biases shape what gets published
  • Transparency and replication are the corrective

Discussion Questions:

  1. Why does Ioannidis argue that smaller studies are more prone to false findings?

  2. How can pre-registration of studies help address these issues?

  3. What are the trade-offs between replicating studies and conducting new ones?

The “Con” in Econometrics Is Specification Searching

Leamer (1983) — “Let’s Take the Con Out of Econometrics”

Choosing the Lens Changes What You See

Results depended on which model you chose — different specifications, different conclusions. The “lens” was whatever the researcher wanted it to be.

  • Overreliance on econometric models without sensitivity analysis
  • Researchers shape results to fit expectations
  • Robustness checks are essential — show that conclusions hold across specifications

Discussion Questions:

  1. What does Leamer mean by “taking the con out of econometrics”?

  2. How does sensitivity analysis improve reliability?

  3. What role does transparency in data and methods play?

The Credibility Revolution Changed How We Do Science

Angrist & Pischke (2010) — “The Credibility Revolution in Empirical Economics”

Better Lenses, Better Science

The fix wasn’t better data or fancier models — it was better research designs that made the lens trustworthy: RCTs, natural experiments, IV, regression discontinuity.

  • Design-based methods establish causality
  • These designs improved empirical credibility
  • Trade-off: credible methods sometimes limit the scope of questions

Discussion Questions:

  1. What is the “credibility revolution” and why does it matter?

  2. How do research designs like RCTs and IVs strengthen causal inference?

  3. Should researchers prioritize credible methods even at the cost of narrower questions?

Non-Experimental Methods Failed the LaLonde Test

LaLonde (1986) — “Evaluating the Econometric Evaluations of Training Programs”

The Lens Was Tested — and It Failed

LaLonde had a benchmark: experimental results from a job training RCT. When he tried to replicate them with non-experimental methods, the results were wildly different. The observational “lens” couldn’t see straight.

  • Experimental vs. non-experimental methods compared head-to-head
  • Non-experimental methods often failed to replicate experimental results
  • A landmark demonstration that method matters

Discussion Questions:

  1. What lessons do the discrepancies teach us?

  2. How do LaLonde’s findings influence policy evaluation today?

  3. What are the strengths and weaknesses of relying on experimental methods?

What These Four Papers Tell Us Together

Table 1
Paper The Problem The Lens Insight
Ioannidis (2005) Most findings are false Small samples + flexibility = unreliable lens
Leamer (1983) Specification searching Results depend on which lens you choose
Angrist & Pischke (2010) Need for better methods Design-based methods = better lenses
LaLonde (1986) Observational methods fail Non-experimental lenses failed the test

The Bottom Line

Science was broken. Design-based methods fixed it. The rest of this lecture builds the foundational lens that makes those methods work.

The Credibility Revolution: A Timeline

Show R Code 
timeline_data <- tibble(
  year = c(1923, 1925, 1974, 1979, 1983, 1986, 2010),
  event = c("Neyman", "Fisher", "Rubin", "Heckman",
            "Leamer", "LaLonde", "Angrist & Pischke"),
  description = c("Potential outcomes\nframework",
                   "Randomization\nfor experiments",
                   "Rubin Causal\nModel formalized",
                   "Selection\nmodels",
                   "\"Take the Con\nOut\"",
                   "The experimental\nbenchmark test",
                   "Credibility\nRevolution"),
  era = c("Foundations", "Foundations", "Foundations", "Foundations",
          "Crisis", "Crisis", "Revolution"),
  y_pos = c(1, -1, 1, -1, 1, -1, 1)
)

ggplot(timeline_data, aes(x = year, y = 0)) +
  # Timeline axis
  geom_hline(yintercept = 0, linewidth = 1, color = slate_gray) +
  # Era shading
  annotate("rect", xmin = 1920, xmax = 1977, ymin = -1.8, ymax = 1.8,
           alpha = 0.06, fill = primary_blue) +
  annotate("rect", xmin = 1977, xmax = 1995, ymin = -1.8, ymax = 1.8,
           alpha = 0.06, fill = accent_coral) +
  annotate("rect", xmin = 1995, xmax = 2015, ymin = -1.8, ymax = 1.8,
           alpha = 0.06, fill = secondary_teal) +
  # Era labels
  annotate("text", x = 1948.5, y = 1.7, label = "FOUNDATIONS",
           color = primary_blue, fontface = "bold", size = 4) +
  annotate("text", x = 1986, y = 1.7, label = "CRISIS",
           color = accent_coral, fontface = "bold", size = 4) +
  annotate("text", x = 2005, y = 1.7, label = "REVOLUTION",
           color = secondary_teal, fontface = "bold", size = 4) +
  # Points
  geom_point(aes(color = era), size = 5) +
  # Segments connecting to labels
  geom_segment(aes(x = year, xend = year, y = 0, yend = y_pos * 0.3),
               linewidth = 0.5, color = slate_gray) +
  # Labels
  geom_label(aes(y = y_pos * 0.7, label = paste0(event, "\n(", year, ")")),
             size = 3.2, fill = "white", label.size = 0.3,
             fontface = "bold", color = slate_gray) +
  geom_text(aes(y = y_pos * 1.25, label = description),
            size = 2.8, color = slate_gray, lineheight = 0.9) +
  scale_color_manual(values = c("Foundations" = primary_blue,
                                "Crisis" = accent_coral,
                                "Revolution" = secondary_teal)) +
  scale_x_continuous(breaks = seq(1920, 2010, 10), limits = c(1918, 2015)) +
  ylim(-1.8, 1.8) +
  labs(x = NULL, y = NULL,
       title = "The Road to Credible Causal Inference") +
  theme_health_econ(base_size = 14) +
  theme(
    axis.text.y = element_blank(),
    axis.ticks.y = element_blank(),
    panel.grid = element_blank(),
    legend.position = "none"
  )

Figure 2: Key milestones in the development of causal inference

From Crisis to Clarity

Figure 3

Those four papers tell us that science was broken — and that design-based methods fixed it. But to understand why the fix works, we need to first see what goes wrong when we look with the naked eye.

The Naked Eye

What Correlation Shows — and What It Hides

The Hardest Question in Health Policy

The Question:

  • How can we determine if a policy or intervention truly improves outcomes?

The Challenge:

  • Correlation doesn’t imply causation.
  • We often observe outcomes after an intervention but can’t see what would have happened without it.

The Solution:

  • Causal inference provides tools to isolate cause-and-effect relationships
  • But first, we need to understand why the naked eye fails

Policy vs. Counterfactual (Visual)

Show R Code 
p_policy <- ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.7, ymin = 2.4, ymax = 6.8,
           fill = scales::alpha(primary_blue, 0.15), color = primary_blue, linewidth = 1) +
  annotate("text", x = 2.65, y = 6.2, label = "Policy Implemented",
           color = primary_blue, fontface = "bold", size = 4.8) +
  annotate("text", x = 2.65, y = 4.4, label = "Observed Outcome",
           color = primary_blue, size = 4.4) +
  annotate("segment", x = 2.65, xend = 2.65, y = 2.4, yend = 3.6,
           arrow = arrow(length = unit(0.12, "inches")),
           color = primary_blue, linewidth = 1) +
  annotate("text", x = 2.65, y = 2.0, label = "Data", color = primary_blue, size = 4) +
  xlim(0, 5.2) + ylim(1.6, 7.4) +
  theme_diagram()

p_counter <- ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.7, ymin = 2.4, ymax = 6.8,
           fill = scales::alpha(slate_gray, 0.08), color = slate_gray,
           linetype = "dashed", linewidth = 1) +
  annotate("text", x = 2.65, y = 6.2, label = "No Policy",
           color = slate_gray, fontface = "bold", size = 4.8) +
  annotate("text", x = 2.65, y = 4.4, label = "Unobserved Outcome",
           color = slate_gray, size = 4.4) +
  annotate("text", x = 2.65, y = 2.0, label = "Counterfactual", color = slate_gray, size = 4) +
  xlim(0, 5.2) + ylim(1.6, 7.4) +
  theme_diagram()

p_policy + p_counter

Figure 4: We see outcomes under policy, not the world without it

Prediction Tells You What; Causality Tells You Why

Traditional Prediction

  • Detect patterns in data and fit functional relationships with a high degree of accuracy (e.g., “Does this person have heart disease?” “Do I need an umbrella today?”)
  • Not predictions of what effects a decision will have

Causal Inference

  • A type of prediction — but a fundamentally different one
  • Predicts the counterfactual associated with a specific decision
  • We take the predicted/imputed counterfactual and construct a causal effect
  • We hope it’s applicable to similar causes/effects in the future

The Toolkit: From Naked Eye to Precision Lens

Show R Code 
# Panel 1: Prediction
p_predict <- ggplot() +
  # Features
  annotate("rect", xmin = 0.5, xmax = 3.5, ymin = 2, ymax = 8,
           fill = light_gray, color = slate_gray, linewidth = 0.5) +
  annotate("text", x = 2, y = 8.5, label = "Features (X)",
           fontface = "bold", size = 5, color = primary_blue) +
  annotate("text", x = 2, y = 7, label = "Age, BMI,\nBlood Pressure,\nLab Results",
           size = 3.5, color = slate_gray) +
  # Arrow
  annotate("segment", x = 3.5, xend = 5.5, y = 5, yend = 5,
           arrow = arrow(length = unit(0.3, "inches")),
           linewidth = 1.5, color = primary_blue) +
  annotate("text", x = 4.5, y = 5.8, label = "Model\n(pattern matching)",
           size = 3, color = slate_gray, fontface = "italic") +
  # Outcome
  annotate("rect", xmin = 5.5, xmax = 8.5, ymin = 3, ymax = 7,
           fill = scales::alpha(primary_blue, 0.15), color = primary_blue, linewidth = 0.5) +
  annotate("text", x = 7, y = 7.5, label = "Predicted Y",
           fontface = "bold", size = 5, color = primary_blue) +
  annotate("text", x = 7, y = 5, label = "\"This patient\nhas heart\ndisease\"",
           size = 3.5, color = primary_blue) +
  # Title
  annotate("text", x = 4.5, y = 9.5, label = "PREDICTION",
           fontface = "bold", size = 6, color = primary_blue) +
  xlim(0, 9) + ylim(1, 10) +
  theme_void()

# Panel 2: Causation
p_cause <- ggplot() +
  # Treatment
  annotate("rect", xmin = 0.5, xmax = 3, ymin = 4, ymax = 6.5,
           fill = scales::alpha(secondary_teal, 0.15), color = secondary_teal, linewidth = 0.5) +
  annotate("text", x = 1.75, y = 7, label = "Treatment (D)",
           fontface = "bold", size = 5, color = secondary_teal) +
  annotate("text", x = 1.75, y = 5.25, label = "Ventilator\nYes/No",
           size = 3.5, color = secondary_teal) +
  # Confounders
  annotate("rect", xmin = 3, xmax = 6, ymin = 7.5, ymax = 9.5,
           fill = scales::alpha(accent_coral, 0.15), color = accent_coral,
           linewidth = 0.5, linetype = "dashed") +
  annotate("text", x = 4.5, y = 8.5, label = "Confounders (U)\nSeverity, Age, ...",
           size = 3.2, color = accent_coral) +
  # Arrows from confounders
  annotate("segment", x = 3.5, xend = 2.5, y = 7.5, yend = 6.5,
           arrow = arrow(length = unit(0.2, "inches")),
           linewidth = 1, color = accent_coral, linetype = "dashed") +
  annotate("segment", x = 5.5, xend = 6.5, y = 7.5, yend = 6.5,
           arrow = arrow(length = unit(0.2, "inches")),
           linewidth = 1, color = accent_coral, linetype = "dashed") +
  # Arrow from treatment to outcome
  annotate("segment", x = 3, xend = 5.5, y = 5.25, yend = 5.25,
           arrow = arrow(length = unit(0.3, "inches")),
           linewidth = 1.5, color = secondary_teal) +
  annotate("text", x = 4.25, y = 4.5, label = "Causal effect\n(what we want)",
           size = 3, color = secondary_teal, fontface = "italic") +
  # Outcome
  annotate("rect", xmin = 5.5, xmax = 8, ymin = 4, ymax = 6.5,
           fill = scales::alpha(secondary_teal, 0.15), color = secondary_teal, linewidth = 0.5) +
  annotate("text", x = 6.75, y = 7, label = "Outcome (Y)",
           fontface = "bold", size = 5, color = secondary_teal) +
  annotate("text", x = 6.75, y = 5.25, label = "Survival\nYes/No",
           size = 3.5, color = secondary_teal) +
  # Title
  annotate("text", x = 4.5, y = 10.2, label = "CAUSATION",
           fontface = "bold", size = 6, color = secondary_teal) +
  xlim(0, 9) + ylim(3, 10.5) +
  theme_void()

p_predict + p_cause

Figure 5: Prediction matches patterns; causal inference isolates the effect of an intervention

You Can Never See the Road Not Taken

The Counterfactual Problem:

  • We cannot know the counterfactual.
    • We only observe outcomes under one condition (e.g., with or without the intervention)
    • We can never directly observe what would have happened in the alternate scenario

Plato’s Cave:

  • In The Republic, prisoners in a cave see only shadows on a wall, not reality itself
  • In causal inference:
    • Observed outcomes are the shadows
    • The counterfactual is the unseen reality we strive to understand

Plato’s prisoners saw shadows — correlation without causation. Building the lens is how we escape the cave.

The Road Not Taken (Visual)

Show R Code 
ggplot() +
  geom_segment(aes(x = 2, y = 1, xend = 5, yend = 4.5),
               color = primary_blue, linewidth = 4, lineend = "round") +
  geom_segment(aes(x = 5, y = 4.5, xend = 8.5, yend = 7.2),
               color = primary_blue, linewidth = 4, lineend = "round") +
  geom_segment(aes(x = 5, y = 4.5, xend = 8.5, yend = 2),
               color = scales::alpha(slate_gray, 0.4), linewidth = 4, lineend = "round") +
  annotate("text", x = 7.8, y = 7.4, label = "Observed path",
           color = primary_blue, fontface = "bold", size = 5, hjust = 1) +
  annotate("text", x = 7.8, y = 1.8, label = "Counterfactual path",
           color = slate_gray, fontface = "bold", size = 5, hjust = 1) +
  annotate("text", x = 2.2, y = 0.8, label = "Policy choice",
           color = slate_gray, size = 4, hjust = 0) +
  xlim(1, 9.2) + ylim(0.5, 8) +
  theme_diagram()

Figure 6: One path is observed; the other is unseen

The Missing Data Problem Is Unsolvable — Almost

Example: Illness vs. Wellness

  • A person cannot be both ill and not ill at the same time
  • If a patient receives a treatment and recovers, we cannot observe whether they would have recovered without treatment
  • The counterfactual question: What would their outcome have been if the treatment were not given?

Why This Matters:

  • Without the counterfactual, we cannot directly measure the causal effect
  • The Goal: Use statistical methods to approximate the counterfactual as accurately as possible
  • All of causal inference involves imputing missing counterfactuals — and not all imputations are equal

Correlation Asks a Different Question Than Causation

Causal Questions

  • “If a patient is put on a ventilator (D), will her COVID symptoms (Y) improve?”
  • Requires knowing what would happen without the ventilator

Correlational Questions

  • “Does household income fluctuate with Lululemon sales?”
  • “If it rained yesterday, will it rain today?”

The Correlation Formula

\[ \rho = \frac{Cov(D, Y)}{\sqrt{Var_D}\sqrt{Var_Y}} \]

The Key Distinction

Correlation measures co-movement. Causation requires knowing what happens when you intervene.

Correlation vs. Intervention (Visual)

Show R Code 
set.seed(6320)
scatter_df <- tibble(
  x = rnorm(200),
  y = 0.6 * x + rnorm(200, sd = 0.6)
)

p_corr <- ggplot(scatter_df, aes(x = x, y = y)) +
  geom_point(color = scales::alpha(slate_gray, 0.6), size = 2) +
  geom_smooth(method = "lm", se = FALSE, color = primary_blue, linewidth = 1.5) +
  labs(title = "Correlation", subtitle = "Patterns in observed data") +
  theme_diagram() +
  theme(plot.title = element_text(color = primary_blue, face = "bold", size = 14),
        plot.subtitle = element_text(color = slate_gray, size = 11))

p_do <- ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.2, ymin = 3, ymax = 6.6,
           fill = scales::alpha(secondary_teal, 0.18), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 2.4, y = 6.1, label = "Intervention (do D)",
           color = secondary_teal, fontface = "bold", size = 4) +
  annotate("segment", x = 4.2, xend = 6.2, y = 4.8, yend = 4.8,
           arrow = arrow(length = unit(0.15, "inches")),
           color = secondary_teal, linewidth = 1.5) +
  annotate("rect", xmin = 6.2, xmax = 9.4, ymin = 3, ymax = 6.6,
           fill = scales::alpha(secondary_teal, 0.1), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 7.8, y = 6.1, label = "Outcome (Y)",
           color = secondary_teal, fontface = "bold", size = 4) +
  annotate("text", x = 7.8, y = 4.2, label = "Causal effect",
           color = secondary_teal, size = 4) +
  xlim(0, 10) + ylim(2.5, 7.4) +
  labs(title = "Causation", subtitle = "Effect of an intervention") +
  theme_diagram() +
  theme(plot.title = element_text(color = secondary_teal, face = "bold", size = 14),
        plot.subtitle = element_text(color = slate_gray, size = 11))

p_corr + p_do

Figure 7: Correlation describes patterns; interventions identify effects

Sequence Is Not Causation

  • Every morning the rooster crows and then the sun rises
    • Did the rooster cause the sun to rise? Or did the sun cause the rooster to crow?
    • What if a cat killed the rooster?
  • Post hoc ergo propter hoc: “after this, therefore, because of this”

Even True Correlations Can Be Meaningless

Causality May Mask Correlations

A sailor expertly navigating crosswinds constantly adjusts the rudder. To a naive onlooker, the rudder appears to have no effect on direction — because the sailor’s skill hides the causal relationship.

Sequence Is Not Causation (Visual)

Show R Code 
ggplot() +
  geom_segment(aes(x = 1, y = 4, xend = 9, yend = 4),
               color = slate_gray, linewidth = 1.2) +
  annotate("point", x = 3, y = 4, size = 6, color = accent_coral) +
  annotate("point", x = 7, y = 4, size = 6, color = warm_gold) +
  annotate("text", x = 3, y = 5.3, label = "Rooster crows",
           color = accent_coral, size = 4, fontface = "bold") +
  annotate("text", x = 7, y = 5.3, label = "Sun rises",
           color = warm_gold, size = 4, fontface = "bold") +
  annotate("segment", x = 3.3, xend = 6.7, y = 4, yend = 4,
           arrow = arrow(length = unit(0.15, "inches")),
           color = slate_gray, linewidth = 1) +
  annotate("text", x = 5, y = 2.6, label = "Sequence", color = slate_gray, size = 4) +
  annotate("text", x = 5, y = 1.4, label = "No evidence of causation",
           color = slate_gray, size = 4, fontface = "italic") +
  xlim(1, 9) + ylim(1, 6.5) +
  theme_diagram()

Figure 8: Events can follow each other without causation

From Correlation to Causation

Figure 9

The naked eye sees correlation and calls it causation. To see through this illusion, we need a precision instrument — the potential outcomes framework. This is the lens that makes causal inference possible.

Building the Lens

The Potential Outcomes Framework

Two Things Define Every Causal Question

  • Potential vs. Realized Outcomes
    • Potential outcomes: what would happen under different treatments
    • Realized outcomes: what did happen for a given treatment
    • Once the decision to put the patient on the ventilator happens, it eliminates one of them
  • Treatment Assignment Mechanism
    • Why someone receives treatment
    • Why the choice was made, not so much who made it
    • E.g., why did a patient get assigned to a ventilator?
    • It matters if a doctor flipped a coin vs. read the chart and made the “best decision”

Causal Question Anatomy (Visual)

Show R Code 
ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.2, ymin = 4.5, ymax = 7.2,
           fill = scales::alpha(primary_blue, 0.12), color = primary_blue, linewidth = 1) +
  annotate("text", x = 2.4, y = 6.6, label = "Potential Outcomes",
           color = primary_blue, fontface = "bold", size = 4.6) +
  annotate("text", x = 2.4, y = 5.4, label = "Y^1 and Y^0",
           color = primary_blue, size = 4.4) +
  annotate("rect", xmin = 0.6, xmax = 4.2, ymin = 1.8, ymax = 4.1,
           fill = scales::alpha(secondary_teal, 0.12), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 2.4, y = 3.4, label = "Assignment Mechanism",
           color = secondary_teal, fontface = "bold", size = 4.4) +
  annotate("text", x = 2.4, y = 2.3, label = "Why D was chosen",
           color = secondary_teal, size = 4.2) +
  annotate("segment", x = 4.3, xend = 6.1, y = 5.8, yend = 5.8,
           arrow = arrow(length = unit(0.15, "inches")),
           color = slate_gray, linewidth = 1.2) +
  annotate("segment", x = 4.3, xend = 6.1, y = 2.9, yend = 2.9,
           arrow = arrow(length = unit(0.15, "inches")),
           color = slate_gray, linewidth = 1.2) +
  annotate("rect", xmin = 6.2, xmax = 9.4, ymin = 3.4, ymax = 6.6,
           fill = scales::alpha(warm_gold, 0.18), color = warm_gold, linewidth = 1) +
  annotate("text", x = 7.8, y = 5.9, label = "Causal Estimand",
           color = warm_gold, fontface = "bold", size = 4.6) +
  annotate("text", x = 7.8, y = 4.6, label = "ATE / ATT / ATU",
           color = warm_gold, size = 4.2) +
  xlim(0, 10) + ylim(1.5, 7.6) +
  theme_diagram()

Figure 10: Potential outcomes + assignment mechanism define the causal question

The Language of the Lens: Treatment Assignment

There’s a long history of the notation starting in the 1920s with Neyman and Fisher, further complicated from being independently refined by economists and statisticians.

Let’s think about a binary treatment:

\[ D_{i,t} = \begin{cases} 1 & \text{if placed on ventilator at time } t, \\ 0 & \text{if not placed on ventilator at time } t. \end{cases} \]

where \(i\) indexes an individual observation, such as a person

The Language of the Lens: Potential Outcomes

\[ Y^{j}_{i,t} = \begin{cases} Y^1_{i,t} & \text{outcome if placed on ventilator at time } t, \\ Y^0_{i,t} & \text{outcome if not placed on ventilator at time } t. \end{cases} \]

where \(j\) indexes a potential treatment status for the same \(i\) person at the same \(t\) point in time

  • Prior to being placed on the ventilator, there are two possibilities:
    • They could be placed on the ventilator, call it \(Y^1\), and something could happen, or
    • They could not be placed on the ventilator, call it \(Y^0\), and something could happen
  • This can be extended to multiple potential outcomes

Graduate Students: Neyman-Rubin vs. Structural Approaches

The potential outcomes framework (Neyman 1923, Rubin 1974) defines causal effects without specifying a structural model. Pearl’s (2009) structural causal model (SCM) approach uses directed acyclic graphs (DAGs) to represent causal structure. Both frameworks are equivalent for many problems, but differ in how they handle mediation and path-specific effects.

The Causal Effect Is the Difference Between Two Worlds

Definition: Individual Treatment Effect

The individual treatment effect, \(\delta_i\), associated with a ventilator is equal to:

\[ \delta_i = Y^1_i - Y^0_i \]

Where:

  • \(Y^1_i\) is the outcome for individual \(i\) if placed on a ventilator
  • \(Y^0_i\) is the outcome for individual \(i\) if not placed on a ventilator

The causal effect is the difference between two worlds — but we only ever get to live in one.

Reality Chooses One World and Hides the Other

Definition: Switching Equation

An individual’s realized health outcome, \(Y_i\), is determined by treatment assignment, \(D_i\), which selects one of the potential outcomes:

\[ Y_i = D_i Y^1_{i} + (1 - D_i) Y^0_{i} \]

Alternatively:

\[ Y_i = \begin{cases} Y^1_{i} & \text{if } D_i = 1, \\ Y^0_{i} & \text{if } D_i = 0. \end{cases} \]

Note: This is not the same as the treatment assignment mechanism. The treatment assignment mechanism describes why one patient, but not another, was put on the ventilator.

The Switching Equation (Visual)

Show R Code 
ggplot() +
  annotate("rect", xmin = 0.6, xmax = 3.6, ymin = 4.6, ymax = 6.9,
           fill = scales::alpha(secondary_teal, 0.15), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 2.1, y = 5.9, label = "Y^1 (treated)",
           color = secondary_teal, fontface = "bold", size = 4.4) +
  annotate("rect", xmin = 0.6, xmax = 3.6, ymin = 1.6, ymax = 3.9,
           fill = scales::alpha(accent_coral, 0.15), color = accent_coral, linewidth = 1) +
  annotate("text", x = 2.1, y = 2.9, label = "Y^0 (untreated)",
           color = accent_coral, fontface = "bold", size = 4.4) +
  annotate("rect", xmin = 4.4, xmax = 5.8, ymin = 3.2, ymax = 5.3,
           fill = scales::alpha(warm_gold, 0.2), color = warm_gold, linewidth = 1) +
  annotate("text", x = 5.1, y = 4.25, label = "D",
           color = warm_gold, fontface = "bold", size = 5) +
  annotate("segment", x = 3.6, xend = 4.4, y = 5.75, yend = 4.9,
           arrow = arrow(length = unit(0.12, "inches")),
           color = secondary_teal, linewidth = 1.2) +
  annotate("segment", x = 3.6, xend = 4.4, y = 2.85, yend = 3.7,
           arrow = arrow(length = unit(0.12, "inches")),
           color = accent_coral, linewidth = 1.2) +
  annotate("rect", xmin = 6.6, xmax = 9.4, ymin = 3.2, ymax = 5.3,
           fill = scales::alpha(primary_blue, 0.12), color = primary_blue, linewidth = 1) +
  annotate("text", x = 8, y = 4.25, label = "Observed Y",
           color = primary_blue, fontface = "bold", size = 4.6) +
  annotate("segment", x = 5.8, xend = 6.6, y = 4.25, yend = 4.25,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1.2) +
  xlim(0, 10) + ylim(1, 7.4) +
  theme_diagram()

Figure 11: Treatment assignment selects which potential outcome we observe

The Lens Reveals What We’re Missing

Table 2
Patient Treatment \(Y^1\) (with ventilator) \(Y^0\) (without ventilator) \(\delta\) (causal effect)
Patient 1 Ventilator **Survived** ? **???**
Patient 2 Ventilator **Died** ? **???**
Patient 3 Ventilator **Survived** ? **???**
Patient 4 No ventilator ? **Survived** **???**
Patient 5 No ventilator ? **Died** **???**
Patient 6 No ventilator ? **Survived** **???**

The Fundamental Missing Data Problem

  • We only observe one potential outcome per person — the other is forever hidden
  • This isn’t “missing your car keys” — it’s missing unicorns and dragons (fictional data that never existed)
  • Not even more data can solve this — you will always be missing one potential outcome

Since We Can’t See Individuals, We Average

Definition: Average Treatment Effect (ATE)

The average treatment effect (ATE) is the population average of all individual treatment effects:

\[ \begin{align} E[\delta] &= E[Y^1 - Y^0]\\ &= E[Y^1] - E[Y^0] \end{align} \]

  • Aggregate causal parameters are summaries of individual treatment effects

Cannot be calculated because \(Y^1_i\) and \(Y^0_i\) do not exist for the same unit \(i\) due to the switching equation

Conditional Average Treatment Effects

Average Treatment Effect on the Treated (ATT)

\[ \begin{align} E[\delta \mid D = 1] &= E[Y^1 - Y^0 \mid D = 1]\\ &= E[Y^1 \mid D = 1] - E[Y^0 \mid D = 1] \end{align} \]

The ATT asks: what was the average effect among those who actually received treatment?

Cannot be calculated: \(E[Y^0 \mid D = 1]\) is unobserved

Average Treatment Effect on the Untreated (ATU)

\[ \begin{align} E[\delta \mid D = 0] &= E[Y^1 - Y^0 \mid D = 0] \\ &= E[Y^1 \mid D = 0] - E[Y^0 \mid D = 0] \end{align} \]

The ATU asks: what would have been the average effect among those who did not receive treatment?

Cannot be calculated: \(E[Y^1 \mid D = 0]\) is unobserved

ATT vs. ATU (Visual)

Show R Code 
ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.7, ymin = 2.2, ymax = 6.8,
           fill = scales::alpha(secondary_teal, 0.15), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 2.65, y = 6.2, label = "Treated Group",
           color = secondary_teal, fontface = "bold", size = 4.6) +
  annotate("text", x = 2.65, y = 4.6, label = "ATT",
           color = secondary_teal, fontface = "bold", size = 5) +
  annotate("text", x = 2.65, y = 3.4, label = "Effect among treated",
           color = secondary_teal, size = 4) +
  annotate("rect", xmin = 5.3, xmax = 9.4, ymin = 2.2, ymax = 6.8,
           fill = scales::alpha(accent_coral, 0.15), color = accent_coral, linewidth = 1) +
  annotate("text", x = 7.35, y = 6.2, label = "Untreated Group",
           color = accent_coral, fontface = "bold", size = 4.6) +
  annotate("text", x = 7.35, y = 4.6, label = "ATU",
           color = accent_coral, fontface = "bold", size = 5) +
  annotate("text", x = 7.35, y = 3.4, label = "Effect among untreated",
           color = accent_coral, size = 4) +
  xlim(0, 10) + ylim(1.6, 7.4) +
  theme_diagram()

Figure 12: Treatment effects differ by who is treated

The Naive Comparison: What the Naked Eye Sees

Definition: Simple Difference in Mean Outcomes (SDO)

A simple difference in mean outcomes (SDO) can be approximated by comparing the sample average outcome for the treatment group (\(D = 1\)) with a comparison group (\(D = 0\)):

\[ \text{SDO} = E[Y^1 \mid D = 1] - E[Y^0 \mid D = 0] \]

SDO is not a causal parameter because it’s comparing \(Y^1\) and \(Y^0\) for different units, not the same units

The Naive Comparison (Visual)

Show R Code 
naive_df <- tibble(
  group = factor(c("Treated", "Control"), levels = c("Control", "Treated")),
  outcome = c(5, 8)
)

ggplot(naive_df, aes(x = group, y = outcome, fill = group)) +
  geom_col(width = 0.6) +
  geom_segment(aes(x = 1.5, xend = 1.5, y = 5, yend = 8),
               arrow = arrow(ends = "both", length = unit(0.12, "inches")),
               color = primary_blue, linewidth = 1.2, inherit.aes = FALSE) +
  annotate("text", x = 1.5, y = 8.35, label = "SDO", color = primary_blue,
           fontface = "bold", size = 4, vjust = 0) +
  scale_fill_manual(values = c("Control" = slate_gray, "Treated" = secondary_teal)) +
  scale_y_continuous(limits = c(0, 9.2)) +
  labs(x = NULL, y = "Average outcome") +
  theme_diagram() +
  theme(legend.position = "none",
        axis.text.x = element_text(color = slate_gray, face = "bold", size = 11),
        axis.text.y = element_text(color = slate_gray, size = 10),
        axis.title.y = element_text(color = slate_gray, size = 11))

Figure 13: A simple difference mixes causal effect with bias

The Naked Eye Sees Three Things at Once

Decomposition of the SDO

The simple difference in mean outcomes (SDO) is made up of three components:

\[ \underbrace{E[Y^1 \mid D = 1] - E[Y^0 \mid D = 0]}_{\text{SDO (what we observe)}} = \underbrace{ATE}_{\text{What we want}} + \underbrace{E[Y^0 \mid D = 1] - E[Y^0 \mid D = 0]}_{\text{Selection bias}} + \underbrace{(1 - \pi)(ATT - ATU)}_{\text{Heterogeneous treatment effect bias}} \]

where \(\pi\) is the share of units in the treatment group

  • The expected value of the untreated outcome needs to be the same for both treated and control groups
  • Consider a doctor who sees patients, knows each person’s treatment effects, and assigns treatment based on whether gains are positive
  • Eliminating selection bias requires understanding the selection mechanism — why did units end up treated but not others?

The Lens Test

The Golden Rule of Causal Inference

“Before you believe any comparison, ask: What is the lens made of, and is it distorted?”

Three questions to diagnose any empirical claim:

  1. What’s being compared? (Treated vs. untreated? Before vs. after? Both?)
  2. Why did some units get treated? (Random? Self-selected? Assigned by a doctor?)
  3. What’s the distortion? (Selection on \(Y^0\)? On \(Y^1\)? On gains?)

Every method we learn this semester — IV, RDD, DiD — is a different way to correct the distortion.

Graduate Students: The Roy Model and Essential Heterogeneity

Heckman & Vytlacil (2005) show that when individuals select into treatment based on their idiosyncratic gains (the “Roy model”), LATE, ATT, and ATE can all differ substantially. The marginal treatment effect (MTE) framework unifies these parameters and shows they are weighted averages of the same underlying function.

Two Schools of Thought on Correcting the Lens

Historically two ways that selection bias was addressed:

1. Design-Based Methods

  • Experimental design and randomization
  • Rubin and others
  • RCT, IV, RDD

Fix the assignment mechanism so the lens is clear

2. Model-Based Methods

  • Model the selection bias and then remove it mechanically
  • Heckman and others
  • DiD, Event Studies, Synthetic Controls, Matching

Model the distortion and subtract it out

Both have been highly influential, but constitute different approaches to the same problem

From Building to Distortion

Figure 14

We’ve built the lens — potential outcomes, treatment effects, the switching equation. But having a lens doesn’t guarantee clear vision. If patients sort themselves into treatment, the lens distorts. This section is about understanding that distortion.

Distortion in the Lens

Why Simple Comparisons Mislead

Selection Bias Comes from How People Get Treated

  • Selection bias in causal inference is when one or both mean potential outcomes differ by treatment status
  • The source of the bias is why people get treated — the treatment assignment mechanism: Randomization (coin flip assignment) vs. Sorting on potential outcomes (treatment because it’ll help you) — the “Perfect Doctor” or “Roy model”
  • If treatment was assigned randomly (exogeneity), the implications differ from being assigned “optimally” (endogeneity)

Three Ways the Lens Gets Distorted

Selection bias arises when there are different mean potential outcomes by treatment status. Three possibilities:

  1. Selection on \(Y^0\): You chose the treatment because of what will happen if you didn’t take it
  2. Selection on \(Y^1\): You chose the treatment because of what will happen if you do take it
  3. Selection on gains (\(\delta\)): You chose the treatment because the net benefits were positive

Impact on ATE:

  • All three forms of selection bias distort estimates of the Average Treatment Effect
  • The degree of distortion depends on the reasons for sorting into treatment

Selection Bias Types (Visual)

Show R Code 
panel_box <- function(title, subtitle, color) {
  ggplot() +
    annotate("rect", xmin = 0.6, xmax = 4.8, ymin = 1.8, ymax = 6.8,
             fill = scales::alpha(color, 0.12), color = color, linewidth = 1) +
    annotate("text", x = 2.7, y = 6.1, label = title,
             color = color, fontface = "bold", size = 4.2) +
    annotate("text", x = 2.7, y = 4.3, label = subtitle,
             color = color, size = 3.8) +
    xlim(0.5, 5) + ylim(1.5, 7.2) +
    theme_diagram()
}

p_y0 <- panel_box("Selection on Y^0", "Choose treatment because\nY^0 would be bad", accent_coral)
p_y1 <- panel_box("Selection on Y^1", "Choose treatment because\nY^1 would be good", secondary_teal)
p_gain <- panel_box("Selection on gains", "Choose treatment when\nbenefits are positive", warm_gold)

p_y0 + p_y1 + p_gain

Figure 15: Three ways selection can distort causal inference

The Selection Bias Decomposition

Show R Code 
decomp_data <- tibble(
  component = factor(c("SDO\n(What we observe)", "ATE\n(What we want)",
                       "Selection Bias", "Heterogeneity\nBias"),
                     levels = c("SDO\n(What we observe)", "ATE\n(What we want)",
                                "Selection Bias", "Heterogeneity\nBias")),
  value_biased = c(8, 3, 3.5, 1.5),
  value_rct = c(3, 3, 0, 0)
)

decomp_long <- decomp_data %>%
  pivot_longer(cols = c(value_biased, value_rct),
               names_to = "scenario", values_to = "value") %>%
  mutate(scenario = ifelse(scenario == "value_biased",
                           "Observational Study\n(Distorted Lens)",
                           "Randomized Experiment\n(Clear Lens)"))

ggplot(decomp_long, aes(x = component, y = value, fill = component)) +
  geom_col(width = 0.6) +
  geom_text(aes(label = round(value, 1)), vjust = -0.5, fontface = "bold", size = 4) +
  facet_wrap(~scenario) +
  scale_fill_manual(values = c(slate_gray, secondary_teal, accent_coral, warm_gold)) +
  labs(x = NULL, y = "Effect Size",
       title = "Randomization Zeroes Out the Distortion",
       subtitle = "The SDO equals the ATE only when selection bias and heterogeneity bias are zero") +
  theme_health_econ(base_size = 14) +
  theme(legend.position = "none",
        strip.text = element_text(face = "bold", size = 14))

Figure 16: The SDO contains three components – only one of which is the causal effect

The Better the Program, the Worse the Bias

The Paradox:

  • Successful programs are the hardest to study:
    • The better our programs are, the better humans get at assigning treatments to people
    • As interventions correctly target individuals based on treatment gains, selection bias worsens
  • Inefficient programs are easier to study:
    • The less efficient our programs are, the more erratic the assignment mechanism becomes
    • With more mistakes in targeting, selection bias decreases

Natural Experiments Work Because Assignment Is Erratic

  • The drive for natural experiments in causal inference stems from the ability to deduce selection bias from erratic assignment mechanisms
  • This approach works well for less efficient programs but poses challenges for studying highly successful programs

The Key Insight

The entire causal inference toolkit — IV, RDD, DiD — works by finding situations where treatment assignment is as good as random, even though no one flipped a coin.

From Distortion to Correction

Figure 17

Selection bias distorts the lens whenever treatment assignment is correlated with potential outcomes. Is there a way to guarantee the lens is clear? There is — and it’s the single most important idea in causal inference.

The Clearest Lens

How Randomization Corrects the Distortion

Randomization: The Clearest Lens

Summarizing the goals of causal inference

  • Our goal is to estimate aggregate causal parameters with data using treatment assignment mechanisms that plausibly eliminate selection bias
  • Depending on the treatment assignment mechanism, certain procedures are allowed and others are prohibited
  • Let’s look at what happens in an RCT and why this addresses the selection bias terms to see why Fisher (1925) recommended it

Random Assignment (Visual)

Show R Code 
ggplot() +
  annotate("rect", xmin = 0.6, xmax = 4.2, ymin = 3.2, ymax = 6.8,
           fill = scales::alpha(warm_gold, 0.2), color = warm_gold, linewidth = 1) +
  annotate("text", x = 2.4, y = 6.2, label = "Coin Flip",
           color = warm_gold, fontface = "bold", size = 4.6) +
  annotate("text", x = 2.4, y = 4.6, label = "Random D",
           color = warm_gold, size = 4.2) +
  annotate("segment", x = 4.2, xend = 5.6, y = 6.0, yend = 6.0,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1.1) +
  annotate("segment", x = 4.2, xend = 5.6, y = 3.8, yend = 3.8,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1.1) +
  annotate("rect", xmin = 5.8, xmax = 9.4, ymin = 4.6, ymax = 7.2,
           fill = scales::alpha(secondary_teal, 0.15), color = secondary_teal, linewidth = 1) +
  annotate("rect", xmin = 5.8, xmax = 9.4, ymin = 1.8, ymax = 4.4,
           fill = scales::alpha(accent_coral, 0.15), color = accent_coral, linewidth = 1) +
  annotate("text", x = 7.6, y = 6.3, label = "Treated Group",
           color = secondary_teal, fontface = "bold", size = 4.2) +
  annotate("text", x = 7.6, y = 3.5, label = "Control Group",
           color = accent_coral, fontface = "bold", size = 4.2) +
  annotate("text", x = 7.6, y = 5.2, label = "Balanced covariates",
           color = secondary_teal, size = 3.6) +
  annotate("text", x = 7.6, y = 2.4, label = "Balanced covariates",
           color = accent_coral, size = 3.6) +
  xlim(0, 10) + ylim(1.5, 7.6) +
  theme_diagram()

Figure 18: Randomization balances groups on average

Independence Makes the Distortion Disappear

Independence Assumption

Treatment is assigned to a population independent of that population’s potential outcomes

\[ (Y^0, Y^1) \perp\!\!\!\perp D \]

  • This is random or quasi-random assignment and ensures:
    • Mean potential outcomes for the treatment group and control group are the same
    • Other variables are distributed the same for a large sample

\[ E[Y^0|D = 1] = E[Y^0|D = 0] \qquad E[Y^1|D = 1] = E[Y^1|D = 0] \]

Randomization Zeroes Out Selection Bias

\[ \underbrace{E_N [y_i |d_i = 1] - E_N [y_i|d_i = 0]}_{\text{SDO}} = \underbrace{E[Y^1] - E[Y^0]}_{\text{ATE}} + \underbrace{E[Y^0 | D = 1] - E[Y^0 | D = 0]}_{\text{Selection bias}} + \underbrace{(1 - \pi)(ATT - ATU)}_{\text{Heterogeneous treatment effect bias}} \]

Key Insight: If treatment is independent of potential outcomes, we can eliminate selection bias:

\[ E[Y^0 | D = 1] - E[Y^0 | D = 0] = 0 \]

The lens is clear — treated and untreated groups have the same baseline potential outcomes.

Randomization Also Zeroes Out Heterogeneity Bias

How does randomization affect the heterogeneous treatment effects bias? Rewrite ATT and ATU:

\[ \begin{align} ATT &= E[Y^1|D = 1] - E[Y^0|D = 1]\\ ATU &= E[Y^1|D = 0] - E[Y^0|D = 0] \end{align} \]

Under independence:

\[ \begin{align} ATT - ATU =& \; E[Y^1| D=1] - E[Y^0|D = 1] - E[Y^1| D=0] + E[Y^0|D = 0] = 0 \end{align} \]

Therefore:

\[ \begin{align} SDO &= ATE + 0 + 0 = ATE \end{align} \]

Under Randomization, the Simple Comparison Is Causal

Identification with Full Independence

\[ \underbrace{E_N [y_i |d_i = 1] - E_N [y_i|d_i = 0]}_{\text{SDO}} = \underbrace{E[Y^1] - E[Y^0]}_{\text{ATE}} + \underbrace{0}_{\text{Selection bias}} + \underbrace{0}_{\text{Heterogeneous treatment effect bias}} \]

Key Insight:

  • The Simple Difference in Outcomes (SDO) is an unbiased estimate of the ATE with randomized treatment assignment
    • Reason 1: Randomization eliminates selection bias (sets it to zero)
    • Reason 2: ATT equals ATU — everyone has the same expected treatment effect in expectation

Graduate Students: Fisher vs. Neyman Inference

Fisher’s exact test (randomization inference) and Neyman’s repeated sampling approach differ in their targets: Fisher tests the sharp null (no effect for any unit), while Neyman targets the average effect. Modern practice often combines both — using Neyman-style CIs with Fisher-style p-values from randomization inference.

From Correction to Protection

Figure 19

Randomization clears the distortion — but only under certain conditions. If treatment spills across units, or varies in dosage, or doesn’t scale, even a randomized experiment can produce misleading results.

Protecting the Lens

SUTVA and the Conditions for Clear Vision

The Lens Works Only If Conditions Are Stable

  • SUTVA stands for “Stable Unit Treatment Value Assumption” (S = stable; U = units; TV = treatment value; A = assumption)
  • Means that a potential outcome has nothing to do with others’ treatment assignment — only one’s own, and only today

Without SUTVA

If one person’s treatment affects another person’s outcome, or if the treatment varies in dosage, the potential outcomes we defined are not well-defined — and the entire lens breaks down.

Rule 1: Treatment Doesn’t Leak Between Units

  • What if we impose a treatment at one neighborhood but not a contiguous one?
  • Treatment may spill over causing \(Y = Y^1\) even for the control units because of spillovers from treatment group

Mitigation:

  • Careful delineation of treatment and control units so that interference is impossible
  • May even require aggregation — e.g., the classroom becomes the unit, not individual students
  • Think about whether treated and untreated units interact

Rule 2: Everyone Gets the Same Dose

  • SUTVA requires each unit receive the same treatment dosage
    • This is what it means by “stable”
    • Notice that the superscripts contain either 0 or 1, not 0.55, 0.27
  • If we are estimating the effect of aspirin on headaches, we assume treatment is 200mg per person
    • Easy to imagine violations if hospital quality, staffing, or even the ventilators themselves vary across treatment group
  • Be careful what we are and are not defining as the treatment — you may need multiple treatment arms

Rule 3: What Works Small May Not Work Big

  • Scale can affect stability of treatment effects
  • Example: We estimate a causal effect of early childhood intervention in Texas, and the President wants to roll it out for the whole United States — will it have the same effect?
  • Scaling up a policy can be challenging if there are rising costs of production — expansion may require lower quality teachers, creating general equilibrium effects when we only estimated a partial equilibrium effect (external vs. internal validity)

Graduate Students: Partial Identification Without SUTVA

Manski (2013) shows that when SUTVA is violated (e.g., spillovers exist), point identification of treatment effects is generally impossible. However, bounds on treatment effects can still be derived under weaker assumptions. This connects to the broader partial identification literature.

SUTVA Threats (Visual)

Show R Code 
panel_simple <- function(title, subtitle, color) {
  ggplot() +
    annotate("rect", xmin = 0.6, xmax = 4.8, ymin = 1.8, ymax = 6.8,
             fill = scales::alpha(color, 0.12), color = color, linewidth = 1) +
    annotate("text", x = 2.7, y = 6.1, label = title,
             color = color, fontface = "bold", size = 4.2) +
    annotate("text", x = 2.7, y = 4.3, label = subtitle,
             color = color, size = 3.6) +
    xlim(0.5, 5) + ylim(1.5, 7.2) +
    theme_diagram()
}

p_spill <- panel_simple("Spillovers", "Treatment affects\nother units", accent_coral)
p_dose <- panel_simple("Dose variation", "Treatment intensity\nis not constant", secondary_teal)
p_scale <- panel_simple("Scale effects", "Effects change\nwhen scaled up", warm_gold)

p_spill + p_dose + p_scale

Figure 20: Three common violations of stability

From Protection to Synthesis

Figure 21

We’ve built a lens (potential outcomes), identified the distortion (selection bias), found the correction (randomization), and learned the conditions for clear vision (SUTVA). Now let’s take stock of what we’ve built — and preview the specialized lenses coming next.

Key Takeaways

What We’ve Built: The Complete Lens

  • Introduced two defining elements of the design approach to causal inference: Potential Outcomes to define causal parameters and the treatment assignment mechanism (e.g., randomization)
  • Introduced “selection bias” and “heterogeneous treatment effect bias”; under independence and SUTVA, randomization “deleted” the second and third term

\[ \begin{align} SDO =& \; ATE \\ &+ \underbrace{E[Y^0 \mid D = 1] - E[Y^0 \mid D = 0]}_{\text{= 0 under randomization}} \\ &+ \underbrace{(1 - \pi)(ATT - ATU)}_{\text{= 0 under randomization}} \end{align} \]

The Complete Lens (Visual)

Show R Code 
ggplot() +
  annotate("rect", xmin = 0.6, xmax = 3.0, ymin = 3.3, ymax = 5.8,
           fill = scales::alpha(primary_blue, 0.12), color = primary_blue, linewidth = 1) +
  annotate("text", x = 1.8, y = 4.9, label = "Potential\nOutcomes",
           color = primary_blue, fontface = "bold", size = 4) +
  annotate("rect", xmin = 3.4, xmax = 5.8, ymin = 3.3, ymax = 5.8,
           fill = scales::alpha(accent_coral, 0.12), color = accent_coral, linewidth = 1) +
  annotate("text", x = 4.6, y = 4.9, label = "Selection\nBias",
           color = accent_coral, fontface = "bold", size = 4) +
  annotate("rect", xmin = 6.2, xmax = 8.6, ymin = 3.3, ymax = 5.8,
           fill = scales::alpha(secondary_teal, 0.12), color = secondary_teal, linewidth = 1) +
  annotate("text", x = 7.4, y = 4.9, label = "Randomization\nor Design",
           color = secondary_teal, fontface = "bold", size = 4) +
  annotate("rect", xmin = 9.0, xmax = 11.4, ymin = 3.3, ymax = 5.8,
           fill = scales::alpha(warm_gold, 0.16), color = warm_gold, linewidth = 1) +
  annotate("text", x = 10.2, y = 4.9, label = "Causal\nClaim",
           color = warm_gold, fontface = "bold", size = 4) +
  annotate("segment", x = 3.0, xend = 3.4, y = 4.55, yend = 4.55,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1.1) +
  annotate("segment", x = 5.8, xend = 6.2, y = 4.55, yend = 4.55,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1.1) +
  annotate("segment", x = 8.6, xend = 9.0, y = 4.55, yend = 4.55,
           arrow = arrow(length = unit(0.12, "inches")),
           color = slate_gray, linewidth = 1.1) +
  xlim(0.2, 11.8) + ylim(2.8, 6.3) +
  theme_diagram()

Figure 22: A pipeline from assumptions to causal claims

Applying the Lens: How to Read Empirical Papers

For every empirical paper you read this semester, apply these five questions:

  1. What causal question is being asked?
    • Does it align with microeconomic theory? Is it testable?
  2. What lens (method) is being used?
    • RCT, IV, RDD, DiD, matching, synthetic control?
  3. What’s the distortion (source of selection bias)?
    • Why did some units get treated? Is it “as good as random”?
  4. How does the design correct it?
    • What is the identification strategy?
  5. Is SUTVA plausible?
    • Spillovers? Variable dosage? Scalability concerns?

Discussion Questions

  1. The Lens Test: A news headline says “Hospitals that adopted AI diagnostics had 20% fewer misdiagnoses.” Using the Lens Test, identify: What’s being compared? Why did some hospitals adopt AI? What’s the likely distortion?

  2. Selection in action: A pharmaceutical company reports that patients who took their new drug had better outcomes than those who didn’t. A colleague says “selection bias.” Explain exactly which form(s) of selection bias might be at play using the \(Y^0\)/\(Y^1\)/\(\delta\) framework.

  3. The LaLonde challenge: Why is it so devastating for non-experimental methods that they couldn’t replicate the experimental results? What does this tell us about the quality of the “lens” in observational studies?

  4. SUTVA in practice: You’re studying a new hand-hygiene intervention in a hospital. Treatment is randomized at the nurse level — some nurses get the training, others don’t. Why might SUTVA be violated? How would you fix the design?

The Lens Principle

The Core Insight

“Every empirical claim is only as good as the lens used to see it. The potential outcomes framework reveals what’s hidden; selection bias distorts the view; and the entire credibility revolution is the story of building better lenses.”

The rest of this course builds specialized lenses — IV, RDD, DiD — each designed to correct different types of distortion. The fundamentals you learned today are what every lens is made of.

Looking Ahead

We’ve built the foundational lens — potential outcomes, treatment effects, and the conditions for clear vision.

Next: The Detective’s Toolkit (Instrumental Variables)

What happens when you can’t randomize? The first specialized lens uses an instrument — a variable that shifts treatment but doesn’t directly affect the outcome — to see through the distortion of selection bias.

From the foundational lens to the detective’s toolkit — our causal arsenal begins to grow.