Design, Evaluation and Sustainability of Private Schools for the Poor: The Pakistan Urban and Rural Fellowship School Experiments

Estimation Methods

In this section we lay out the rationale for random assignment. For the purpose of this study the outcome variable of interest is school enrollment. However, the approach is general; the use of random assignment is not specific to the evaluation of the impact of this educational program, or to educational programs per se.

Because government resources were in limited supply and the need to expand enrollment was so great, the government of Balochistan needed an accurate measure of the program's success and its prognosis for expansion. The evaluation problem is to find an unbiased estimator of the impact of the fellowship program (a ). Denoting school enrollment in the treatment and control neighborhoods as RT and RN respectively, ideally one would like to estimate for any individual at time t. However, one can not estimate this directly since a child cannot be simultaneously in both the treatment state and the control state.

One way to get an unbiased estimator of a is to use changes in the outcome variable over time. This approach, termed a reflexive evaluation, can be written


The reflexive estimator measures the expected program effect as the gap between the enrollment rate after the program, E(RTt), and the enrollment rate before the program was implemented, E(RT0). The underlying assumption of this method is that the period t outcome in the treatment neighborhood without the program would have been identical to the observed pre-program outcome. In effect, the treatment group in the base period (before intervention) serves as control for the treatment group after program implementation.

However, reflexive evaluations are sensitive to trends that may be national but erroneously attributed to the intervention. Thus, an alternative approach is to use a control group to derive estimates of the counterfactual state. The difference in outcomes between the treatment and control groups is then used as an estimate of a . This could either be the mean difference, defined as:


or difference in differences, defined as:


The mean-difference estimator (2), measures the expected program effect by the observed post-program gap in outcomes between the treatment group and the control group. This method assumes that the control group perfectly matches the treatment group. This is often achieved by randomly assigning groups into the treatment and control populations.

The difference in differences estimator (3), measures the expected program effect by the gap between the post-program outcome in the treatment group, E(RTt), and that in the control group, E(RNt), adjusted by the pre-program difference between the two groups. In this method, it is assumed that the difference in outcomes between the two groups before the program intervention would be constant over time if it were not for the program. Thus, the difference in outcome between the two groups after the program intervention reflects the difference due to the program as well as to the initial difference. Differencing the differences yields an estimate of the program effect. If the assignment into treatment and control groups is successfully undertaken by randomization there is no difference between Eq (2) and eq (3). This is because, if the two groups are identical at the outset, the term in the second bracket in eq (3) equals zero.

For reasons discussed below, it is possible that this condition will be violated. In that case, it is necessary to control for differences between the treatment and control groups that could also influence outcomes. To amplify, consider a general model of individual enrollment choice in year t:


In equation (4), Xit is a vector of observed characteristics, Uit is an error term, and b t is a vector of parameters to be estimated. A covariate post-test estimate of a can be derived from a cross section regression in some period t after program implementation, assuming that the b s are invariant across individuals:


where di is a dummy variable indicating residence in a fellowship school neighborhood. With random assignment into the treatment group, we can assume di is independent of the unobserved variables Uit, so that E(Uit|di)=0. Under this assumption one obtains an unbiased estimate of alpha. Conversely, if di is correlated with the unobserved factors– as, for example, when assignment into the treatment groups is based on observed individual or community enrollment - then the estimate of alpha will be biased.

To check for this possibility, we also conducted the analysis using repeated observations of individuals, an alternative way to estimate the program effect using econometric analysis is to estimate equation (5) in terms of differences in the variables between the base period and a period t after the intervention has taken place, enabling us to first difference away unobserved individual fixed effects. None of the qualitative results were affected.


 Theory of Enrollment Response to the Girls' Fellowship Program

Households are assumed to have parents, a daughter and a son. Parents are assumed to derive utility from their own consumption of goods (Zh) and from the human capital of their daughter (Hf) and their son (Hm). The utility function has the form U=U(Zh, Hf, Hm, T), where T is a vector of taste indicators that are not subject to choice.

Let Y be household income, Pz be the price of consumption goods, and Pf and Pm are the prices of schooling for their daughter and son, respectively. The income constraint on parental utility maximization is Pz Zh+ Pf Hf + PmHm = Y.

Assume that social prohibitions against exposing daughters to the outside world cause parents to discount the utility they get from their daughter's education by some factor df<1 so that Hf = dfUm = dfH.

Then the parents utility will have the form U(Zh, dfH, H, T), with UHf = dfUH(Zh, Hf, Hm,T) and UHm = UH(Zh, Hf, Hm, T).

The first order conditions yield the following relation:

where UHf and UHm represent the marginal utility of girls schooling and boys schooling, respectively. To get parents to equalize schooling for their boys and girls so that , the cost of girls schooling must be discounted by Pf = df Pm < Pm.

Reduced-form equations for boy's and girl's schooling have the following functional forms:

The reduced form equations suggest that enrollment choice for girls and for boys will depend on school fees for girls, school fees for boys, the rate at which parents discount girls' education relative to boys, income, the price level, and tastes. Numerous studies suggest that education is a normal good so that

d Hm/d Y > 0 and d Hf/d Y > 0. Those conditions are sufficient to insure that d Hm/d Pm< 0 and d Hf/d Pf <0. The discount factor df acts as an additional price on girls' schooling, so d Hf/d df < 0.

The girls' fellowship program will lower Pf, so girls' schooling will increase. The impact of the fellowship program on boys' enrollment is ambiguous. However, there are two reasons the girls' fellowship program may have a positive impact on boys' schooling.

First, the program creates a new low-priced private school that can accept boys, lowering Pm, although it lowers Pf even more.

Second, if one treats the travel time to school as a component of the price and adds the overall time endowment to the family’s budget, one can use a similar argument to indicate that the location of the school will influence boys' education for a very practical reason - parents may want their boys to escort their sisters to and from school. Thus, shorter distances to girls schools affect both sons and daughters. This implies that boys' education and girls' education may be complementary goods so that d Hm/d Pf < 0.