Spot the Difference - Difference in Difference

Today, we delve into the world of econometrics, taking a closer look at a popular and widely used research method: Difference in Difference (DiD). The technique has a wealth of applications in policy evaluation, economics, and social sciences. It helps us isolate the effects of a specific intervention or treatment by comparing the pre- and post-treatment changes between a group that received the treatment and another group that did not.

Difference in Difference: An Overview

The Difference in Difference (DiD) method is a quasi-experimental approach used to estimate the causal effect of a particular intervention or treatment on an outcome, by comparing the average change over time in the outcome variable for the treatment group to the average change over time for the control group.

Assumptions

The DiD method rests on some key assumptions:

Parallel Trends Assumption: In the absence of treatment, the average outcomes of the treated and control groups would have followed the same trajectory over time. This is also known as the Common Trends assumption.
No Simultaneous Spillover Effects: The treatment does not affect the control group, and the two groups are not influencing each other's outcomes.
No Composition Changes: The composition of the treatment and control groups does not change over time.

Method

The DiD estimator is calculated as follows:

Example

Let's consider an example. Suppose we want to measure the effect of a job training program on participants' earnings. We have two groups: those who underwent training (treated group) and those who didn't (control group). By comparing the earnings difference before and after the training for both groups, the DiD estimator will provide the causal impact of the training program on earnings.

Pros and Cons

DiD is not without limitations. The validity of the results depends heavily on whether the parallel trends assumption holds. If it doesn't, the estimates may be biased.

However, DiD has its advantages too. It effectively controls for time-invariant unobserved confounding variables, which could otherwise bias the estimated treatment effect.

Difference in Difference in Practice

Now let's visualize the DiD method using a plot. For simplicity, we'll use a hypothetical example. We'll generate some data for a control and treated group over time, apply a treatment effect, and demonstrate how DiD works.

Please note that in a real-world scenario, data would be collected from actual observations rather than being generated.

import numpy as np
import matplotlib.pyplot as plt

# Time periods
pre_treatment = np.arange(1, 6)
post_treatment = np.arange(6, 11)
time_periods = np.concatenate((pre_treatment, post_treatment))

# Generate hypothetical data for control and treatment groups
np.random.seed(0)
control_group = np.concatenate((20 + 2*pre_treatment + np.random.normal(0, 1, 5), 
                                20 + 2*post_treatment + np.random.normal(0, 1, 5)))
treatment_group = np.concatenate((30 + 2*pre_treatment + np.random.normal(0, 1, 5), 
                                  30 + 3*post_treatment + np.random.normal(0, 1, 5)))

# Counterfactual for the treatment group if no treatment was applied
counterfactual = 30 + 2*time_periods

# Plotting
plt.figure(figsize=(10, 6))
plt.plot(time_periods, control_group, label="Control Group", linestyle="--")
plt.plot(time_periods, treatment_group, label="Treatment Group")
plt.plot(time_periods, counterfactual, label="Counterfactual", linestyle=":")
plt.axvline(x=5.5, color='r', linestyle='-', label="Treatment Applied")
plt.xlabel("Time Period")
plt.ylabel("Outcome")
plt.legend()
plt.title("Difference in Difference Plot")
plt.show()

The plot shows parallel trends for both groups before the treatment is applied (dotted vertical line). After the treatment, the outcome for the treatment group increases more than that for the control group. The dotted line represents the counterfactual - what would have happened to the treatment group if the treatment had not been applied.

Conclusion

To sum up, the Difference in Difference method is a robust and widely used technique in policy evaluation and social science research. It provides an intuitive and effective way to estimate causal relationships. However, researchers must be aware of its assumptions and limitations and apply it judiciously.