Home Insights Journey through Panel Data Analysis, Fixed Effects Models, and Difference-in-Difference Methods for...

Journey through Panel Data Analysis, Fixed Effects Models, and Difference-in-Difference Methods for Policy Evaluation – IMPRI Impact and Policy Research Institute

9
0
Unraveling Casual Inference: Journey through Panel Data Analysis, Fixed Effects Models, and Difference-in-Difference Methods for Policy Evaluation

Session Report
Rakesh Pandey

The Generation Alpha Data Centre, at IMPRI Impact and Policy Research Institute, New Delhi conducted a five-Day Immersive Online Introductory Hands-On Certificate Training Course from December 18 to22, 2023.

The course, spread over five-consecutive days, helped to equip policymakers, researchers, and data enthusiasts with the knowledge and tools needed to conduct rigorous impact evaluations with practical datasets and hands-on learning experiences. In this course, we went beyond theory and provided hands-on training in various evaluation designs, Unraveling Casual Inference, statistical techniques, and data collection methods commonly employed in impact evaluation. With these skills, participants will be better equipped to navigate the complex world of data management and make informed decisions that can drive positive change.

On the second day our first speaker, Rakesh is a doctoral candidate in the Research, Analysis, and Design stream at the Pardee RAND Graduate School and an assistant policy researcher at the RAND Corporation, commenced the session with the panel data method and went on to give insightful session on Difference in-Difference.

Overview

Mr. Rakesh Pandey presented a PPT on Difference and In-difference, the session covered important topics ranging from Panel Data Method, Omitted Variable (OVB), Usage of the panel data by researchers, fixed effects allowing for time-invariant unobservable factors, Fixed effects, Fixed effect Model, estimating regression and graph analysis. 

Introduction: Casual Inference

Mr. Rakesh Pandey initiated the session with an explanation of the Panel data method, he gave a glimpse of the last session where he brought up the working of regression and how regression helps to explore the causal effect of treatment on a particular output variable followed by the overview of RCT.

RCT is the most common scenario which is put up as a prospective one i.e. how one plans things in advance and then rolls them out and tries to see the intervention that affects a particular variable. In most cases, people do not have the ideal setting and still want to evaluate some events in history on a certain policy outcome—for example, MGNREGA (National Employment Act where some got the treatment and some didn’t). If the Act gets evaluated to understand its effectiveness, it will never be a prospective but retrospective evaluation.

For the Policy Evaluation, one needs methods and behind the whole evaluation, one needs to look at data on a regular interval to set a scenario where one can generate some sort of experiment. One needs to know if it’s a field experiment, or it’s an experiment where somebody randomly gets something and somebody doesn’t and that’s the ideal setting, where a person can estimate the causal impact of an experiment. Still, because one can do it prospectively it is perspective, one cannot do it for Policies that have already been implemented. 

Then, we ventured into other territories of regression. One of them was the Fixed effects model and Panel data regressions that were discussed further.

He further talked about the sort of correlation setup in a retrospective setting to move towards more casual retrospectives that we establish and how panel data methods will help. He asserted that this would be the basis for the next session on Difference and Indifference 2, and we used a difference in difference setups to establish casualty which is again retrospective.

He illustrated the above with an example- the Rwandan Genocide case that happened in 1994, which was further studied by David Yanagizaw, a professor at the University of Zurich. The case was about the conflict between the Tutsi Community and the Hutu Community, Hutus started killing Tutsi, they had been using radio signals to send messages to people who were in different communes (now called districts). The professor went through 1991 records, figured out the districts in 1991 that had access to radio transmissions and those that didn’t, and observed the number of people who died in a particular district or commune.

The above allowed him to set differences and indifference, where he was able to evaluate the impact of the radio transmission through which Hutus were organizing differences themselves.

It allowed him to set up differences-in-difference. He was able to study how districts with radio and signals through which people were organizing themselves, the Hutus were organizing themselves, the Hutus were organizing themselves led to a 10% increase in the killing of Tutsi.

WHAT IS PANEL DATA?

When we observe the same unit of observation (Person, Geographic, area) at multiple points in time, it is said to be Panel Data.

Selection & Observable: Whatever variable that can affect the treatment or is correlated with the treatment or has included never holds in all cases. Even if one can add 100 variables to your regression equation,  still there will be some variables missing. One cannot get every single data that can be correlated but data points over time can help control for some of the unobserved variables and that is where the power of fixed effects comes in. 

Even if data is not mentioned and they affect variables or it is correlated with the treatment it can be adjusted, that is due to the power of fixed effects and Panel Data Methods.

Mr. Rakesh Pandey went on to explain the difference between time series and panel data. Also, panel data can be balanced and unbalanced.

Omitted Variable Bias

It is one of the sources of endogeneity. The variables can be grouped into two categories based on variation caused by time: Time-Varying and Non- time-varying.

Panel data analysis can help in at least control for variables that are non-time-varying.

Why researchers are interested in using Panel data?

Mr. Rakesh Pandey went on to explain the usage of panel data, on how it helps to analyse the trends and allows to control of unobserved variables which we do not have in the dataset. Usage of fixed effects and many more. 

How do Fixed Effects allow for the time-invariant unobservable factors?

Fixed effects are a within-estimator: that compares outcomes within units instead of across units.

Across units: It compares outcomes for those who received the intervention compared to those who did not.

Compare how outcomes change for the same person from before the intervention to after the intervention

Types of fixed effects

  1. Time fixed effect 
  2. Geographically fixed effect 
  3. School fixed effects
  4. Enumerator fixed effects

Fixed Effects Model

Fixed effects (FE) make inferences based on intra- rather than interpersonal comparisons of satisfaction.  

                                              Yit= βi+β₁ Xit + εit

The fixed effects regression looks like the one above. It has a subscript, indicating that the data varies both between individuals (i) and time (t). 

Estimating a Regression with individual intercepts

Mr. Rakesh also covered the methods to estimate the regression. 

Method 1: 

►Extract the within variation ourselves.

calculate the mean of the dependent variable for each Y.

And subtract that mean out Yit-e(Yi).

►Do the same for X and then run the below regression

                       Yit-Y=βo+β₁(Xit-Xi) +εit

Method2: 

►Add a binary control variable for every individual, in this case, “country”.

►Rather, for every country except one if our model also has an intercept/constant in it, then including every country will make the model impossible to estimate, so we leave one out, or leave the intercept out.

►That one left-out country is still in the analysis, it just doesn’t get its coefficient.

Conclusion:

 Mr. Rakesh Pandey concluded the session by providing more insights on the topic with the usage of graph analysis and real-life examples. He showed the implications of the effects and elaborated on the methodologies followed by clearing the doubts of his fellow attendees.

Posted by Riya Kumari Shah, a research Intern at IMPRI.

Read more event reports of IMPRI here:

Beyond Petri Dishes: Careers in Public Policy For Biotechnology Students in India 

Catalyzing Change: Public Policy Careers for Chemical Engineering Students in India

Previous articleProbability Distributions: Density And Cumulative – IMPRI Impact And Policy Research Institute
Next articleAddressing Gender-based Violence & The Way Forward – IMPRI Impact And Policy Research Institute
IMPRI, a startup research think tank, is a platform for pro-active, independent, non-partisan and policy-based research. It contributes to debates and deliberations for action-based solutions to a host of strategic issues. IMPRI is committed to democracy, mobilization and community building.

LEAVE A REPLY

Please enter your comment!
Please enter your name here