Home Insights Basic Statistics, Econometrics And Impact Evaluation Methods

Basic Statistics, Econometrics And Impact Evaluation Methods

Impact Evaluation Methods, Basic Statistics and Econometrics in Practice

Session Report

Nitya Kuchimanchi

Facilitating a comprehensive understanding of impact evaluation methods through evaluation designs, statistical techniques and data collection methods to enable cost effective and data driven decision making for development outcomes, A Five-Day Immersive Online Introductory Certificate Training Course titled ‘Impact Evaluation in Practice’ was conducted by IMPRI (Impact and Policy Research Institute) from December 18 to December 22, 2023.

Catering to researchers, policy makers or professionals looking to enhance their skills, the course aimed to emphasize theories, concepts and data management tools that can help result in positive change through development and policy interventions.

On the 1st day, the session on Basic Statistics & Econometrics in Practice & Introduction to Impact Evaluation Methods was taken up by Mr. Rakesh Pandey, Assistant Policy Researcher and Doctoral Scholar at Pardee RAND Graduate School, RAND Corporation, USA  wherein he delved into common regression analysis methods and their significance.

Methodological Exploration

The methodological journey began with a thorough examination of the theoretical setup, econometric intricacies, and practical exercises employing sample datasets to bolster comprehension.

Ordinary Least Squares (OLS) and Impact Evaluation Methods

Brief Review (OLS): OLS took center stage, encompassing a comprehensive review. Participants delved into reading regression results, navigated the challenges transitioning from regression to causation, and engaged in hands-on exercises with sample datasets while challenges faced in moving from regression to causation, testing the significance of coefficients and practicing on sample data sets to understand impact evaluation methods.

Causal Linkage in Impact Evaluation: Unveiling the essence of impact evaluation methods, the session underscored its primary objective: establishing a causal linkage between an outcome variable and treatment, which could manifest as a policy, intervention, or external shock. OLS emerged as the go-to method for estimating relationships between variables. Essentially,’ y’ the outcome variable is the linear addition of a coefficient between an intercept alpha and the treatment variable plus an error term, the error term being the unexplained term. The error is generated through an unknown quarterly data generating process.

An attempt is made to link multiple variables to find an association. The treatment is a dummy variable. The direction of the beta slope is an indicator of the nature of the association. Beta is given by the covariance of y and x divided by the square of variance of X. Beta is thus calculated in a simple univariate model. The calculation can be performed in Excel or using R, Python, Stata, etc. OLS is used when the outcome variable is a continuous variable not a discrete variable.

OLS Basics: Unraveling the essentials, OLS was elucidated as the technique applicable when the outcome variable is continuous. The model’s components, including the intercept alpha, the treatment variable’s coefficient, and the error term, were scrutinized. Beta, the coefficient, stood as a pivotal indicator of the nature of the association.

OLS Assumptions: Participants delved into the assumptions integral to OLS, where observations needed to be independently and identically distributed. The conditional distribution of the error term given T was explored, along with the caution against large outliers. The discussion underscored the essential traits of unbiased, consistent, and efficient OLS estimators. Also, the conditional distribution of the error term given T is Zero. Finally, large outliers are unlikely.

Every estimator has to display some properties, first being unbiased, i.e if the coefficient is equal to the true value of the parameter. It also needs to be consistent- when the sample size is large, the uncertainty about the value of Y arising from random variations in the sample is very small and finally efficient – when the coefficient has the smallest variance.

Standard Error and Variability: The nuanced relationship between standard error and sample size was unveiled, emphasizing the advantageous variability in regression. It was noted that standard deviation is the variance of the population and standard error is the variance of the sample. As the independent variable (X) increases, the slope estimate’s precision augmented, that is regression standard error decreases with the sample size and variability in regression is considered good, as X increases, the slope estimate becomes more precise.

Exogeneity of Treatment

Caveats and Considerations

  • Causation and OLS: Participants were reminded that running a regression does not automatically confer causation. Causal linkages demand a robust theoretical model and a meticulously crafted identification strategy.
  • Handling Dummy Variables: A cautionary note was sounded when navigating dummy independent variables. Linear probability models estimated through OLS were cautioned against, particularly when dealing with erroneous results.
  • Advanced Analysis: The discourse touched upon the nuanced realm of advanced analysis, advocating for the adoption of generalized linear models over OLS for outcome variables like dummy and categorical variables.

Practical Application

The theoretical exploration seamlessly transitioned into practical application as participants immersed themselves in hands-on examples using STATA. This concluding  practical engagement fortified the understanding gained from the theoretical discourse, solidifying the participants’ grasp on impact evaluation methods.

Nitya Kuchimanchi is a Research intern at IMPRI.

Read more session reports at IMPRI:

Data Analytics, Stationarity, and Cointegration in Policy Research

Research Ethics in Data Collection and Analysis

Previous articleUnlocking Opportunities: Public Policy Careers For Mechanical Engineering Students In India – IMPRI Impact And Policy Research Institute
Next articleNurturing Change: Exploring Careers In Public Policy For Students Of Rural Development In India – IMPRI Impact And Policy Research Institute
IMPRI, a startup research think tank, is a platform for pro-active, independent, non-partisan and policy-based research. It contributes to debates and deliberations for action-based solutions to a host of strategic issues. IMPRI is committed to democracy, mobilization and community building.


Please enter your comment!
Please enter your name here