Linear Regression

Linear Regression 

Background

In recent years, many countries – including England – have moved from more centralised to more decentralised education systems: specifically, they have introduced reforms aimed at devolving more power (giving greater autonomy) to schools. The rationale for doing so is to increase school performance, usually measured in terms of students’ exam results. This may occur directly – by enabling those with the greatest knowledge about their pupils to make decisions to maximise their exam performance – or indirectly, e.g. by creating greater competition between schools.

In the US, such reforms have led to the creation of “charter” schools; in Sweden, to “free” schools. In England, there have been two major reforms, leading to the introduction of “grant maintained” schools in the 1990s and “academies” in the 2000s. There is a growing literature on the effects of greater school autonomy on student and school performance (e.g. see Epple, Romano and Zimmer (2016) for a recent summary of the literature on the effects of charter schools), including some papers studying the effects of these reforms in England.

This assignment considers Clark (2009), who studied the effects of grant maintained schools, and Eyles and Machin (2015), who studied the effects of academies. You will find both papers and two related datasets on the module webpage: clark.dta contains the actual school-level data used by Clark (2009); academies.dta contains a school-level version of the type of data used by Eyles and Machin (2015). As such, it contains panel data for schools that converted to academies between 2002 and 2010, including information up to four years before and three years after the year in which each school converted.1 Each variable represents a school-level average of all pupils who took their GCSEs (end of secondary school exams) in the academy. The variables in each dataset are clearly labelled and should be self-explanatory.

1 We make similar sample restrictions to those imposed by Eyles and Machin (2015), e.g. requiring four years of data prior to conversion (i.e. omitting new academies) and dropping those that were previously independent (private) schools. For academies formed from two or more predecessor schools we follow Eyles and Machin (2015) in creating weighted versions of the characteristics included in the dataset in the years leading up to conversion.

Questions

1) Briefly outline the approach taken by Clark (2009) to estimate the impact of greater school autonomy on exam performance. Why does he adopt this approach?

2) What assumptions are required for this to be a causal (consistent) estimate of the effect of greater school autonomy on exam performance? Do you think they are likely to hold in this case? Explain why or why not.

3) Replicate the results in the third row (Base+3) of Table 1. (Note that Clark includes controls for GCSE performance in the year of the vote, and dummy variables for the term and year in which the vote took place (gm_attempt1_ballot_year_term) and school type (school_type) from the second specification (column) onwards. He also uses robust standard errors throughout.)

4) Briefly outline the approach taken by Eyles and Machin (2015) to estimate the impact of greater autonomy on exam performance.

5) What assumptions are required for this to be a causal (consistent) estimate of the effect of greater school autonomy on exam performance? Do you think they are likely to hold in this case? Explain why or why not.

6) Use academies.dta to produce school-level estimates of the effect of academy status on exam performance using similar specifications to those in Columns 1, 4 and 7 of Table 6 in Eyles and Machin (2015). (You can regard schks2_eng_exp, schks2_eng_abv, schks2_mat_exp, schks2_mat_abv, schks2_sci_exp and schks2_sci_abv as equivalent to their controls for Key Stage 2 standardised score. Note that to be consistent with Eyles and Machin’s estimates, you should not use school fixed effects and will need to restrict attention to years up to and including 2009.) Explain briefly why these restrictions are necessary.

7) Discuss the results produced in Question 6) above, highlighting what we learn from each specification and any concerns you might have about the findings.

8) Both Clark (2009) and Eyles and Machin (2015) discuss the potential endogeneity of their estimates to school enrolment decisions. Explain what they do to try to overcome their concerns. Given your answer, what concerns, if any, does this give you about the estimates produced in Question 6) above?

An alternative identification strategy to that used by Eyles and Machin (2015) would be to exploit variation in the timing of academy conversion for all schools included in academies.dta.

9) Produce a table with estimates of the effect of academy status on exam performance based on this approach, using four model specifications: the first three should be similar to those in Question 6); the fourth should add school fixed effects to the last of these three models. Would you feel more or less confident about using this approach to estimate the impact of academy status on exam performance than using the approach in Question 6)? Explain your answer.

10) All schools in England can now become an academy if they wish. On the basis of the results discussed in these two papers – and those produced in Question 9) – would you be confident that all schools would, on average, see a boost to their exam performance in the years following conversion? Explain why or why not. 

References

Clark, D. (2009), The performance and competitive effects of school autonomy, Journal of Political Economy, Vol. 117, pp. 745-783.

Epple, D, R. Romano and R. Zimmer (2016), Charter Schools: a survey of research on their characteristics and effectiveness, Handbook of the Economics of Education, Vol. 5, pp. 139-208.

Eyles, A. and S. Machin (2015), The introduction of academy schools to England’s education, CEP Discussion Paper No. 1368, Centre for Economic Performance, London School of Economics. 

Solution 

Questions 

1) Briefly outline the approach taken by Clark (2009) to estimate the impact of greater school autonomy on exam performance. Why does he adopt this approach?

Direct estimates of the effect of becoming an academy school are likely to be plagued by endogeneity bias. In particular, rather than a random assignment of schools to academy/non-academy groups that ensures similar distributions of covariates for both academy and non-academy schools, it is expected that parents of students at abnormally high achieving schools will be less likely to vote for change and parents at abnormally low achieving schools will be more likely to vote for change. The estimates in columns (5) and (6) explicitly use an instrumental variable approach by using an indicator for choosing academy school at the first vote as an instrument for an indicator that indicates whether the school is an academy school. Also, the model specifications in columns (5) and (6) include the vote percentage as a control, first linearly in the estimates in column (5) and then with both linear and square terms in column (6). The inclusion of the vote percentage as a control may also help to control for variables that are included in the residual and correlated with the choice to become an academy school.

2) What assumptions are required for this to be a causal (consistent) estimate of the effect of greater school autonomy on exam performance? Do you think they are likely to hold in this case? Explain why or why not.

For RDD, the assumption is that the assignment to groups is “as good as random” in a neighborhood around the 50% vote threshold that is used as the break point for the discontinuity. In other words, we assume the distributions of other covariates that affect outcomes to be similar around the threshold at which the discontinuity occurs. This seems to be a plausible assumption here. Additionally, Clark explicitly uses an IV approach rather than RDD to obtain a second set of estimates which are fairly similar to the RDD estimates. For the IV approach, we can imagine a binary dependent variable model for the chosen instrument (i.e., chose academy at first vote) that would predict its value and the same variables that would appear in that equation are the variables that are likely to cause the endogeneity problem if the treatment group indicator were used in an OLS model. In other words, chose academy at first vote is likely to be an excellent instrument for chose academy. In summary, these assumptions seem plausible and gain additional credibility by the similar results obtained by two different  approaches, RDD and IV.

3) Replicate the results in the third row (Base+3) of Table 1. (Note that Clark includes controls for GCSE performance in the year of the vote, and dummy variables for the term and year in which the vote took place (gm_attempt1_ballot_year_term) and school type (school_type) from the second specification (column) onwards. He also uses robust standard errors throughout.)

The Stata commands and their results are displayed below:

/* Column 1 – Raw ITT.                                 */

Linear regression                                      Number of obs =     722

F(  1,   720) =   12.02

Prob> F      =  0.0006

R-squared     =  0.0134

Root MSE      =  21.294

——————————————————————————

|               Robust

pt_gcse_5_~3 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

————-+—————————————————————-

gm_win |      5.496      1.585    3.467   0.001        2.384       8.608

_cons |     44.566      1.236   36.042   0.000       42.138      46.993

——————————————————————————

/* Column 2 – ADJ-ITT                                  */

. regress pt_gcse_5_a_c_base_3 gm_wini.school_type i.gm_attempt1_ballot_year_term pt_gcse_5_a_c_base, vce(robust) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

 

Linear regression                                      Number of obs =     722

F( 23,   697) =       .

Prob> F      =       .

R-squared     =  0.8888

Root MSE      =  7.2645

————————————————————————————-

|               Robust

pt_gcse_5_a_c_base_3 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

———————+—————————————————————-

gm_win |      2.381      0.647    3.682   0.000        1.111       3.650

|

school_type |

2  |     -4.080      2.042   -1.998   0.046       -8.090      -0.070

3  |     -3.598      1.387   -2.594   0.010       -6.321      -0.875

4  |     -4.080      1.446   -2.822   0.005       -6.918      -1.241

6  |     -3.731      1.819   -2.051   0.041       -7.302      -0.159

|

gm_attempt1_ballot_year_term |

19913  |     -0.295      1.340   -0.220   0.826       -2.926       2.336

19921  |     -0.824      2.071   -0.398   0.691       -4.891       3.243

19922  |     -0.199      1.323   -0.151   0.880       -2.798       2.399

19923  |     -1.190      1.145   -1.040   0.299       -3.437       1.057

19931  |     -1.404      1.306   -1.076   0.282       -3.968       1.159

19932  |     -2.062      1.284   -1.606   0.109       -4.582       0.459

19933  |     -3.431      1.477   -2.324   0.020       -6.331      -0.532

19941  |     -2.553      1.603   -1.593   0.112       -5.699       0.594

19942  |     -1.831      1.773   -1.033   0.302       -5.313       1.651

19943  |      1.149      2.794    0.411   0.681       -4.336       6.635

19951  |     -1.475      3.478   -0.424   0.672       -8.303       5.353

19952  |     -4.560      1.842   -2.476   0.014       -8.176      -0.944

19953  |      1.218      2.679    0.455   0.649       -4.042       6.479

19961  |     -4.834      4.507   -1.072   0.284      -13.683       4.016

19962  |     -0.500      1.766   -0.283   0.777       -3.967       2.966

19963  |     -1.602      2.598   -0.617   0.538       -6.703       3.499

19971  |      1.009      4.755    0.212   0.832       -8.327      10.345

19972  |     -1.591      1.039   -1.532   0.126       -3.630       0.449

|

pt_gcse_5_a_c_base |      0.887      0.021   42.579   0.000        0.846       0.927

_cons |     12.438      2.305    5.397   0.000        7.913      16.963

————————————————————————————-

/* Column 3- RD-ITT Linear                             */

regress pt_gcse_5_a_c_base_3 gm_wini.school_type i.gm_attempt1_ballot_year_term pt_gcse_5_a_c_base vote, vce(robust) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

 

Linear regression                                      Number of obs =     722

F( 24,   696) =       .

Prob> F      =       .

R-squared     =  0.8897

Root MSE      =  7.2423

————————————————————————————–

|               Robust

pt_gcse_5_a_c_base_3 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

———————+—————————————————————-

gm_win |      4.616      1.146    4.027   0.000        2.366       6.867

|

school_type |

2  |     -4.119      2.036   -2.023   0.043       -8.116      -0.122

3  |     -3.793      1.399   -2.712   0.007       -6.539      -1.047

4  |     -4.283      1.461   -2.932   0.003       -7.151      -1.415

6  |     -4.109      1.829   -2.247   0.025       -7.699      -0.519

|

gm_attempt1_ballot_year_term |

19913  |     -0.324      1.342   -0.241   0.809       -2.959       2.311

19921  |     -1.009      2.041   -0.494   0.621       -5.016       2.998

19922  |      0.111      1.346    0.083   0.934       -2.531       2.753

19923  |     -1.207      1.144   -1.055   0.292       -3.454       1.039

19931  |     -1.420      1.302   -1.090   0.276       -3.977       1.137

19932  |     -2.127      1.288   -1.652   0.099       -4.654       0.401

19933  |     -3.424      1.470   -2.329   0.020       -6.310      -0.538

19941  |     -2.605      1.598   -1.631   0.103       -5.742       0.531

19942  |     -2.048      1.759   -1.165   0.245       -5.501       1.405

19943  |      0.614      2.901    0.212   0.832       -5.081       6.309

19951  |     -2.023      3.186   -0.635   0.526       -8.278       4.233

19952  |     -5.405      1.862   -2.903   0.004       -9.061      -1.749

19953  |      0.869      2.660    0.327   0.744       -4.354       6.092

19961  |     -5.314      4.490   -1.183   0.237      -14.130       3.502

19962  |     -0.862      1.831   -0.471   0.638       -4.458       2.734

19963  |     -1.599      2.651   -0.603   0.547       -6.803       3.606

19971  |      1.225      4.809    0.255   0.799       -8.216      10.666

19972  |     -0.687      1.116   -0.616   0.538       -2.878       1.503

|

pt_gcse_5_a_c_base |      0.885      0.021   42.330   0.000        0.844       0.926

vote |     -0.055      0.024   -2.295   0.022       -0.103      -0.008

_cons |     14.678      2.527    5.808   0.000        9.716      19.639

————————————————————————————–

. /* Column 4 – RD-ITT Linear x Win                           */

regress pt_gcse_5_a_c_base_3 gm_wini.school_type i.gm_attempt1_ballot_year_term pt_gcse_5_a_c_base win_votelose_vote, vce(robust) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

Linear regression                                      Number of obs =     722

F( 25,   695) =       .

Prob> F      =       .

R-squared     =  0.8897

Root MSE      =  7.2462

————————————————————————————–

|               Robust

pt_gcse_5_a_c_base_3 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

———————+—————————————————————-

gm_win |      4.391      1.219    3.603   0.000        1.998       6.784

|

school_type |

2  |     -4.186      2.043   -2.049   0.041       -8.197      -0.174

3  |     -3.865      1.407   -2.746   0.006       -6.628      -1.102

4  |     -4.353      1.473   -2.956   0.003       -7.244      -1.461

6  |     -4.201      1.825   -2.302   0.022       -7.784      -0.617

|

gm_attempt1_ballot_year_term |

19913  |     -0.340      1.343   -0.253   0.801       -2.977       2.298

19921  |     -1.040      2.038   -0.510   0.610       -5.042       2.961

19922  |      0.140      1.348    0.104   0.917       -2.506       2.785

19923  |     -1.223      1.146   -1.068   0.286       -3.472       1.026

19931  |     -1.423      1.304   -1.091   0.275       -3.983       1.137

19932  |     -2.109      1.292   -1.633   0.103       -4.646       0.427

19933  |     -3.419      1.469   -2.327   0.020       -6.304      -0.534

19941  |     -2.624      1.598   -1.642   0.101       -5.762       0.514

19942  |     -2.023      1.772   -1.142   0.254       -5.503       1.457

19943  |      0.680      2.869    0.237   0.813       -4.952       6.312

19951  |     -1.908      3.247   -0.588   0.557       -8.283       4.467

19952  |     -5.401      1.844   -2.930   0.004       -9.021      -1.781

19953  |      0.941      2.691    0.350   0.727       -4.342       6.225

19961  |     -5.152      4.540   -1.135   0.257      -14.066       3.763

19962  |     -0.830      1.804   -0.460   0.646       -4.372       2.713

19963  |     -1.709      2.637   -0.648   0.517       -6.887       3.469

19971  |      1.056      4.803    0.220   0.826       -8.375      10.486

19972  |     -0.592      1.137   -0.521   0.603       -2.825       1.640

|

pt_gcse_5_a_c_base |      0.883      0.021   42.072   0.000        0.842       0.925

win_vote |      6.159      2.699    2.282   0.023        0.861      11.458

lose_vote |      3.174      5.055    0.628   0.530       -6.752      13.099

_cons |     12.410      2.501    4.962   0.000        7.499      17.321

————————————————————————————–

/* Column 5 – 2SLS-TE Lin x Win                             */

. ivregress 2sls pt_gcse_5_a_c_base_3 i.school_type i.gm_attempt1_ballot_year_term pt_gcse_5_a_c_base win_votelose_vote (gm_school = gm_win), vce(robust) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

Instrumental variables (2SLS) regression               Number of obs =     722

Wald chi2(26) =15739.46

Prob> chi2   =  0.0000

R-squared     =  0.8877

Root MSE      =  7.1754

————————————————————————————–

|               Robust

pt_gcse_5_a_c_base_3 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]

———————+—————————————————————-

gm_school |      5.446      1.525    3.571   0.000        2.457       8.436

|

school_type |

2  |     -4.742      2.018   -2.349   0.019       -8.698      -0.785

3  |     -4.320      1.387   -3.115   0.002       -7.037      -1.602

4  |     -4.673      1.447   -3.229   0.001       -7.509      -1.837

6  |     -4.367      1.795   -2.433   0.015       -7.885      -0.849

|

gm_attempt1_ballot_year_term |

19913  |     -0.702      1.360   -0.516   0.606       -3.368       1.963

19921  |     -1.647      2.027   -0.813   0.416       -5.621       2.326

19922  |      0.084      1.326    0.063   0.950       -2.515       2.682

19923  |     -1.221      1.133   -1.078   0.281       -3.441       0.999

19931  |     -1.393      1.288   -1.082   0.279       -3.917       1.131

19932  |     -2.053      1.282   -1.602   0.109       -4.566       0.459

19933  |     -3.272      1.451   -2.255   0.024       -6.117      -0.428

19941  |     -2.494      1.607   -1.552   0.121       -5.643       0.655

19942  |     -1.767      1.750   -1.010   0.312       -5.197       1.662

19943  |      0.972      2.873    0.338   0.735       -4.659       6.603

19951  |     -1.816      3.148   -0.577   0.564       -7.986       4.354

19952  |     -5.391      1.822   -2.959   0.003       -8.963      -1.820

19953  |      1.172      2.643    0.443   0.657       -4.008       6.353

19961  |     -4.795      4.528   -1.059   0.290      -13.670       4.079

19962  |     -0.607      1.823   -0.333   0.739       -4.180       2.966

19963  |     -0.050      2.536   -0.020   0.984       -5.020       4.919

19971  |      1.839      4.844    0.380   0.704       -7.656      11.334

19972  |      4.656      1.992    2.338   0.019        0.753       8.560

|

pt_gcse_5_a_c_base |      0.876      0.021   41.599   0.000        0.834       0.917

win_vote |      5.690      2.595    2.193   0.028        0.604      10.777

lose_vote |      6.348      5.759    1.102   0.270       -4.940      17.635

_cons |     12.062      2.503    4.819   0.000        7.156      16.968

————————————————————————————–

Instrumented:  gm_school

Instruments:   2.school_type 3.school_type 4.school_type 6.school_type

19913.gm_attempt1_ballot_year_term 19921.gm_attempt1_ballot_year_term

19922.gm_attempt1_ballot_year_term 19923.gm_attempt1_ballot_year_term

19931.gm_attempt1_ballot_year_term 19932.gm_attempt1_ballot_year_term

19933.gm_attempt1_ballot_year_term 19941.gm_attempt1_ballot_year_term

19942.gm_attempt1_ballot_year_term 19943.gm_attempt1_ballot_year_term

19951.gm_attempt1_ballot_year_term 19952.gm_attempt1_ballot_year_term

19953.gm_attempt1_ballot_year_term 19961.gm_attempt1_ballot_year_term

19962.gm_attempt1_ballot_year_term 19963.gm_attempt1_ballot_year_term

19971.gm_attempt1_ballot_year_term 19972.gm_attempt1_ballot_year_term

pt_gcse_5_a_c_base win_votelose_votegm_win

/* Column 6 – 2SLS-TE Quad x Win                            */

Instrumental variables (2SLS) regression               Number of obs =     722

Wald chi2(28) =15173.74

Prob> chi2   =  0.0000

R-squared     =  0.8856

Root MSE      =  7.2408

————————————————————————————–

|               Robust

pt_gcse_5_a_c_base_3 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]

———————+—————————————————————-

gm_school |      8.228      2.619    3.141   0.002        3.094      13.362

|

school_type |

2  |     -5.111      2.059   -2.483   0.013       -9.147      -1.076

3  |     -4.543      1.413   -3.216   0.001       -7.312      -1.774

4  |     -4.874      1.464   -3.329   0.001       -7.743      -2.004

6  |     -4.577      1.815   -2.522   0.012       -8.133      -1.020

|

gm_attempt1_ballot_year_term |

19913  |     -0.822      1.394   -0.590   0.555       -3.555       1.910

19921  |     -2.008      2.053   -0.978   0.328       -6.031       2.015

19922  |      0.149      1.336    0.112   0.911       -2.469       2.767

19923  |     -1.215      1.146   -1.060   0.289       -3.461       1.032

19931  |     -1.364      1.298   -1.050   0.294       -3.908       1.181

19932  |     -1.967      1.306   -1.507   0.132       -4.527       0.592

19933  |     -3.164      1.458   -2.171   0.030       -6.020      -0.307

19941  |     -2.616      1.635   -1.600   0.110       -5.822       0.589

19942  |     -1.603      1.748   -0.917   0.359       -5.030       1.824

19943  |      0.874      2.874    0.304   0.761       -4.759       6.508

19951  |     -0.765      3.656   -0.209   0.834       -7.931       6.402

19952  |     -5.450      1.837   -2.967   0.003       -9.049      -1.850

19953  |      1.035      2.663    0.389   0.698       -4.186       6.255

19961  |     -4.284      4.325   -0.990   0.322      -12.762       4.194

19962  |      0.068      1.808    0.038   0.970       -3.475       3.611

19963  |      1.074      2.760    0.389   0.697       -4.335       6.484

19971  |      2.108      5.046    0.418   0.676       -7.782      11.999

19972  |      7.162      2.743    2.611   0.009        1.785      12.538

|

pt_gcse_5_a_c_base |      0.871      0.021   40.594   0.000        0.829       0.913

win_vote |      9.637     10.616    0.908   0.364      -11.171      30.445

lose_vote |     38.114     22.107    1.724   0.085       -5.216      81.443

lose_vote_2 |    -82.487     54.940   -1.501   0.133     -190.167      25.192

win_vote_2 |      8.875     22.518    0.394   0.693      -35.259      53.009

_cons |      9.991      2.895    3.451   0.001        4.316      15.665

————————————————————————————–

Instrumented:  gm_school

Instruments:   2.school_type 3.school_type 4.school_type 6.school_type

19913.gm_attempt1_ballot_year_term 19921.gm_attempt1_ballot_year_term

19922.gm_attempt1_ballot_year_term 19923.gm_attempt1_ballot_year_term

19931.gm_attempt1_ballot_year_term 19932.gm_attempt1_ballot_year_term

19933.gm_attempt1_ballot_year_term 19941.gm_attempt1_ballot_year_term

19942.gm_attempt1_ballot_year_term 19943.gm_attempt1_ballot_year_term

19951.gm_attempt1_ballot_year_term 19952.gm_attempt1_ballot_year_term

19953.gm_attempt1_ballot_year_term 19961.gm_attempt1_ballot_year_term

19962.gm_attempt1_ballot_year_term 19963.gm_attempt1_ballot_year_term

19971.gm_attempt1_ballot_year_term 19972.gm_attempt1_ballot_year_term

pt_gcse_5_a_c_base win_votelose_vote lose_vote_2 win_vote_2 gm_win

4) Briefly outline the approach taken by Eyles and Machin (2015) to estimate the impact of greater autonomy on exam performance.

Eyles and Machin use a comparison across different groups to estimate the effect of conversion to an academy school on the outcome of an exam taken in the 11th year of school, the KS4 exam.  To understand their approach, consider their equation (3):

The first two terms and the last three terms are just school-specific effects, year-specific effects, the effects of covariates and an error term. The key term for measuring the effect is , where Aist is an indicator that is equal 1 if student i attends school s and school s is either an academy school in year t or will become an academy school subsequent to year t but during  the sample period that extends to year 2009, and I is an indicator variable that has the value 1 if the school has become an academy school as of year t. All of the schools in the sample have Aist=1 and (ideally at least, but see below) all of the students in the sample were enrolled in the school before it converted to an academy school. Therefore the term  will be equal to 1 for a student i who was enrolled in school s before it converted to an academy school and takes a KS4 exam after the school has converted to an academy school. The term  will be 0 if  student i takes the KS4 exam in an academy school during  the 2001-2009 sample period but before the time period in which school s becomes an academy school. Thus the idea is to see what the effect of changing the school to an academy school is on KS4 exam performance by comparing exam performance of students taking exams before the school converted to an academy school with the performance of students who were in the school before it changed to an academy school but took the KS4 exam after it became an academy school. Eyles and Machin also introduce an IV estimator to account for the fact that in their full data set (362,424 observations versus 1,264 in our sample), some of the students who are at the school that converts to an academy school take their exams at different schools and some of the students who take the exams at academy schools were not at the academy schools prior to their becoming academy schools (see the discussion on the bottom of page 16 and top of page 17).

5) What assumptions are required for this to be a causal (consistent) estimate of the effect of greater school autonomy on exam performance? Do you think they are likely to hold in this case? Explain why or why not.

Eyles and Machin stress that the students’ who take the exam at the academy school were enrolled at the school prior to its conversion, which eliminates the selection bias problem that would be present if, for example, a student selected into a school that had converted to an academy school. However, there is still a selection bias they have not accounted for, which is the school’s/community’s choice to convert the school to an academy school. The decision to convert the school to an academy school presumably implies the school was not performing well prior to the conversion, which very likely implies a different distribution of variables that affect exam performance than schools that were did not choose to become academy schools.

6) Use academies.dta to produce school-level estimates of the effect of academy status on exam performance using similar specifications to those in Columns 1, 4 and 7 of Table 6 in Eyles and Machin (2015). (You can regard schks2_eng_exp, schks2_eng_abv, schks2_mat_exp, schks2_mat_abv, schks2_sci_exp and schks2_sci_abv as equivalent to their controls for Key Stage 2 standardised score. Note that to be consistent with Eyles and Machin’sestimates, you need to restrict attention to years up to and including 2009. Explain briefly why these restrictions are necessary.

Stata commands and their results follow.

/* Eyles and Machin, Table 6, column 1, with no school fixed effects. */

regress schstdks4_cappedpts i.afterconversioni.year if year<=2009, vce(cluster schid) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

Linear regression                                      Number of obs =     816

F(  8,   157) =   13.80

Prob> F      =  0.0000

R-squared     =  0.0564

Root MSE      =  .39469

(Std. Err. adjusted for 158 clusters in schid)

———————————————————————————

|               Robust

schstdks4_cap~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————-+—————————————————————-

1.afterconver~n |      0.151      0.061    2.483   0.014        0.031       0.271

|

year |

2003  |     -0.280      0.150   -1.870   0.063       -0.576       0.016

2004  |     -0.204      0.177   -1.151   0.251       -0.554       0.146

2005  |     -0.201      0.191   -1.050   0.295       -0.578       0.177

2006  |     -0.242      0.197   -1.224   0.223       -0.632       0.148

2007  |     -0.203      0.203   -0.999   0.319       -0.604       0.198

2008  |     -0.147      0.211   -0.694   0.489       -0.564       0.271

2009  |     -0.118      0.225   -0.523   0.602       -0.562       0.327

|

_cons |     -0.192      0.204   -0.942   0.348       -0.595       0.211

———————————————————————————

/* Eyles and Machin, Table 6, column 4, with no school fixed effects. */

regress schstdks4_cappedpts i.afterconversioni.year schks2_eng_exp schks2_eng_abv schks2_mat_exp schks2_mat_abv schks2_sci_exp schks2_sci_abv schfemaleschfsmschsenschwhiteschblackschasian if year<=2009, vce(cluster schid) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

Linear regression                                      Number of obs =     816

F( 20,   157) =   47.03

Prob> F      =  0.0000

R-squared     =  0.6597

Root MSE      =  .23882

(Std. Err. adjusted for 158 clusters in schid)

———————————————————————————

|               Robust

schstdks4_cap~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————-+—————————————————————-

1.afterconver~n |      0.069      0.034    2.045   0.042        0.002       0.137

|

year |

2003  |     -0.104      0.097   -1.074   0.284       -0.294       0.087

2004  |     -0.143      0.098   -1.456   0.147       -0.336       0.051

2005  |     -0.265      0.114   -2.330   0.021       -0.490      -0.040

2006  |     -0.265      0.124   -2.137   0.034       -0.510      -0.020

2007  |     -0.229      0.122   -1.878   0.062       -0.469       0.012

2008  |     -0.157      0.124   -1.265   0.208       -0.401       0.088

2009  |     -0.123      0.128   -0.960   0.339       -0.376       0.130

|

schks2_eng_exp |      0.977      0.264    3.695   0.000        0.455       1.499

schks2_eng_abv |      2.037      0.288    7.072   0.000        1.468       2.606

schks2_mat_exp |      0.513      0.268    1.911   0.058       -0.017       1.042

schks2_mat_abv |      0.518      0.348    1.489   0.139       -0.169       1.206

schks2_sci_exp |     -0.112      0.390   -0.286   0.775       -0.883       0.659

schks2_sci_abv |     -0.134      0.320   -0.420   0.675       -0.766       0.497

schfemale |      0.069      0.104    0.666   0.506       -0.137       0.275

schfsm |     -0.216      0.136   -1.595   0.113       -0.484       0.052

schsen |     -0.255      0.114   -2.236   0.027       -0.481      -0.030

schwhite |      0.007      0.168    0.039   0.969       -0.326       0.339

schblack |      0.290      0.226    1.279   0.203       -0.158       0.737

schasian |      0.240      0.246    0.976   0.331       -0.246       0.727

_cons |     -1.136      0.388   -2.928   0.004       -1.902      -0.370

———————————————————————————

/* Eyles and Machin, Table 6, column 7, with no school fixed effects.  */

/* For column 7 estimates, create the following dummy variables.       */

generate E_c_minus4 = 1 if year == yrbecomeacademy – 4

replace E_c_minus4 = 0 if E_c_minus4 != 1

generate E_c_minus3 = 1 if year == yrbecomeacademy – 3

replace E_c_minus3 = 0 if E_c_minus3 != 1

generate E_c_minus2 = 1 if year == yrbecomeacademy – 2

replace E_c_minus2 = 0 if E_c_minus2 != 1

generate E_c_minus1 = 1 if year == yrbecomeacademy – 1

replace E_c_minus1 = 0 if E_c_minus1 != 1

generateE_c = 1 if year == yrbecomeacademy

replaceE_c = 0 if E_c != 1

generate E_c_plus1 = 1 if year == yrbecomeacademy + 1

replace E_c_plus1 = 0 if E_c_plus1 != 1

generate E_c_plus2 = 1 if year == yrbecomeacademy + 2

replace E_c_plus2 = 0 if E_c_plus2 != 1

generate E_c_plus3 = 1 if year == yrbecomeacademy + 3

replace E_c_plus3 = 0 if E_c_plus3 != 1

/* Eyles and Machin, Table 6, column 7, with no school fixed effects.  */

/* Omit E_c_plus3 to avoid collinearity between a full set of dummies  */

/* and the intercept.                                                  */

regress schstdks4_cappedpts E_c_minus4 E_c_minus3 E_c_minus2 E_c_minus1 E_c E_c_plus1 E_c_plus2 i.year schks2_eng_exp schks2_eng_abv schks2_mat_exp schk

> s2_mat_abv schks2_sci_exp schks2_sci_abv schfemaleschfsmschsenschwhiteschblackschasian  if year<=2009, vce(cluster schid) cformat(%9.3f) pformat(%5

> .3f) sformat(%8.3f)

Linear regression                                      Number of obs =     816

F( 26,   157) =   40.17

Prob> F      =  0.0000

R-squared     =  0.6659

Root MSE      =  .23751

(Std. Err. adjusted for 158 clusters in schid)

——————————————————————————–

|               Robust

schstdks4_ca~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————+—————————————————————-

E_c_minus4 |     -0.193      0.100   -1.926   0.056       -0.390       0.005

E_c_minus3 |     -0.169      0.092   -1.832   0.069       -0.351       0.013

E_c_minus2 |     -0.123      0.085   -1.453   0.148       -0.290       0.044

E_c_minus1 |     -0.110      0.078   -1.415   0.159       -0.264       0.044

E_c |     -0.097      0.074   -1.319   0.189       -0.243       0.048

E_c_plus1 |      0.002      0.072    0.028   0.978       -0.140       0.144

E_c_plus2 |      0.036      0.067    0.541   0.590       -0.096       0.168

|

year |

2003  |     -0.106      0.097   -1.100   0.273       -0.297       0.085

2004  |     -0.156      0.104   -1.496   0.137       -0.361       0.050

2005  |     -0.280      0.121   -2.324   0.021       -0.519      -0.042

2006  |     -0.283      0.133   -2.124   0.035       -0.545      -0.020

2007  |     -0.272      0.140   -1.936   0.055       -0.549       0.005

2008  |     -0.221      0.149   -1.488   0.139       -0.515       0.072

2009  |     -0.195      0.156   -1.244   0.215       -0.504       0.114

|

schks2_eng_exp |      0.949      0.263    3.611   0.000        0.430       1.468

schks2_eng_abv |      1.999      0.284    7.039   0.000        1.438       2.560

schks2_mat_exp |      0.559      0.275    2.033   0.044        0.016       1.102

schks2_mat_abv |      0.625      0.349    1.789   0.076       -0.065       1.314

schks2_sci_exp |     -0.142      0.390   -0.364   0.717       -0.911       0.628

schks2_sci_abv |     -0.248      0.319   -0.778   0.438       -0.878       0.382

schfemale |      0.075      0.105    0.719   0.473       -0.132       0.283

schfsm |     -0.250      0.134   -1.865   0.064       -0.516       0.015

schsen |     -0.268      0.113   -2.368   0.019       -0.492      -0.044

schwhite |      0.025      0.171    0.146   0.884       -0.312       0.362

schblack |      0.282      0.230    1.229   0.221       -0.171       0.736

schasian |      0.268      0.244    1.100   0.273       -0.213       0.749

_cons |     -0.935      0.402   -2.325   0.021       -1.730      -0.141

——————————————————————————–

The reason for limiting the sample to prior to 2009 is that the 2010 Academies Act made broad changes to the law that render the data incomparable across eras, according to Eyles and Machin (pp. 6-7).  The fixed effects estimator is not appropriate here because we do not have the KS2 standardized score. If that variable were present, a fixed effect would include the average score difference between KS2 and KS4 and we would want to examine whether students’ performed above that average as the result of taking the exam after the school converted to an academy school.

Where controls were asked for, in addition to the school performance variables indicated in the question, percentages of cohorts who are female, eligible for free lunch, special needs, and of White Black or Asian ethnic origin were also included. .

7) Discuss the results produced in Question 6) above, highlighting what we learn from each specification and any concerns you might have about the findings.

Observe that results are broadly consistent with Eyles and Machin in the sense that effects of conversion to academy school are positive and statistically significant. However, adding in the controls for the regression of column reduces the magnitude of the effect to about 45% as large (i.e., 0.069/0.151) as when the controls are not included, which suggests that the column 1 estimates suffer from omitted variable bias. Also, if the conversion to academy school has a positive effect on learning, then the length of time a student is exposed to that treatment would be expected increase their score for our column 7 estimates. Although that is the case for the point estimates for E_c, E_plus1 and E_c_plus2, the 95% confidence intervals overlap.

8) Both Clark (2009) and Eyles and Machin (2015) discuss the potential endogeneity of their estimates to school enrolment decisions. Explain what they do to try to overcome their concerns. Given your answer, what concerns, if any, does this give you about the estimates produced in Question 6) above?

Clark overcomes endogeneity with an 2SLS IV estimator and with an RDD design as discussed more fully in answers to questions 1 and 2. Eyles and Machin eliminate endogeneity from student selection into academy schools by restricting the sample to include only students who were in academy schools prior to their conversion. Clearly the Eyles and Machin approach does not account for endogeneity that stems from the  communities’ decision to convert the school to an academy school. This limits the external validity of their approach, as the communities that choose to convert their schools will very likely have schools that have different distributions of the residuals than the schools that do not choose to convert.

An alternative identification strategy to that used by Eyles and Machin (2015) would be to exploit variation in the timing of academy conversion for all schools included in academies.dta.

9) Produce a table with estimates of the effect of academy status on exam performance based on this approach, using four model specifications: the first three should be similar to those in Question 6); the fourth should add school fixed effects to the last of these three models. Would you feel more or less confident about using this approach to estimate the impact of academy status on exam performance than using the approach in Question 6)? Explain your answer.

I would feel not very much more confident in using this approach than in using the approach of question (6) for reasons explained above in question 8. The difference between this sample and that one is only the extension of years to 2013, and the selection bias problem presumably becomes less severe for schools that choose to convert later, but is still present.

Estimates similar to those presented for question 6 but extended to the full data set follow.

/* Eyles and Machin, Table 6, column 1, with no school fixed effects. */

. regress schstdks4_cappedpts i.afterconversioni.year, vce(cluster schid) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

Linear regression                                      Number of obs =    1264

F( 12,   157) =   26.61

Prob> F      =  0.0000

R-squared     =  0.1345

Root MSE      =   .3662

 

(Std. Err. adjusted for 158 clusters in schid)

———————————————————————————

|               Robust

schstdks4_cap~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————-+—————————————————————-

1.afterconver~n |      0.151      0.061    2.484   0.014        0.031       0.271

|

year |

2003  |     -0.280      0.150   -1.871   0.063       -0.576       0.016

2004  |     -0.204      0.177   -1.151   0.251       -0.554       0.146

2005  |     -0.201      0.191   -1.050   0.295       -0.578       0.177

2006  |     -0.242      0.197   -1.224   0.223       -0.632       0.148

2007  |     -0.203      0.203   -0.999   0.319       -0.604       0.198

2008  |     -0.147      0.211   -0.694   0.489       -0.564       0.271

2009  |     -0.118      0.225   -0.523   0.602       -0.562       0.327

2010  |     -0.102      0.250   -0.406   0.685       -0.595       0.392

2011  |     -0.011      0.250   -0.044   0.965       -0.505       0.483

2012  |     -0.021      0.252   -0.082   0.935       -0.517       0.476

2013  |     -0.071      0.253   -0.282   0.779       -0.571       0.428

|

_cons |     -0.192      0.204   -0.942   0.348       -0.595       0.211

———————————————————————————

/* Eyles and Machin, Table 6, column 4, with no school fixed effects. */

/* Include control variables and year dummies.                        */

. regress schstdks4_cappedpts i.afterconversioni.year schks2_eng_exp schks2_eng_abv schks2_mat_exp schks2_mat_abv schks2_sci_exp schks2_sci_abv schfemaleschfsmschsenschwhiteschblackschasian, vce(cluster schid) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

 

Linear regression                                      Number of obs =    1264

F( 24,   157) =   46.74

Prob> F      =  0.0000

R-squared     =  0.6129

Root MSE      =  .24609

 

(Std. Err. adjusted for 158 clusters in schid)

———————————————————————————

|               Robust

schstdks4_cap~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————-+—————————————————————-

1.afterconver~n |      0.070      0.034    2.067   0.040        0.003       0.138

|

year |

2003  |     -0.103      0.096   -1.067   0.288       -0.293       0.087

2004  |     -0.145      0.101   -1.429   0.155       -0.345       0.055

2005  |     -0.238      0.114   -2.096   0.038       -0.462      -0.014

2006  |     -0.229      0.121   -1.892   0.060       -0.468       0.010

2007  |     -0.214      0.121   -1.770   0.079       -0.454       0.025

2008  |     -0.149      0.124   -1.202   0.231       -0.395       0.096

2009  |     -0.122      0.129   -0.947   0.345       -0.377       0.133

2010  |     -0.091      0.139   -0.656   0.513       -0.366       0.184

2011  |     -0.048      0.139   -0.349   0.728       -0.322       0.226

2012  |     -0.073      0.143   -0.513   0.609       -0.355       0.208

2013  |     -0.086      0.145   -0.594   0.553       -0.372       0.200

|

schks2_eng_exp |      0.856      0.227    3.771   0.000        0.408       1.304

schks2_eng_abv |      1.478      0.270    5.483   0.000        0.946       2.011

schks2_mat_exp |      0.674      0.223    3.023   0.003        0.234       1.115

schks2_mat_abv |      0.801      0.278    2.882   0.004        0.252       1.350

schks2_sci_exp |     -0.295      0.293   -1.007   0.315       -0.874       0.284

schks2_sci_abv |     -0.115      0.287   -0.401   0.689       -0.681       0.451

schfemale |      0.163      0.095    1.719   0.088       -0.024       0.350

schfsm |     -0.250      0.118   -2.124   0.035       -0.482      -0.017

schsen |     -0.139      0.098   -1.413   0.160       -0.333       0.055

schwhite |     -0.019      0.157   -0.119   0.905       -0.328       0.291

schblack |      0.350      0.208    1.684   0.094       -0.061       0.762

schasian |      0.170      0.211    0.805   0.422       -0.247       0.587

_cons |     -1.080      0.314   -3.442   0.001       -1.699      -0.460

———————————————————————————

/* Eyles and Machin, Table 6, column 7, with no school fixed effects.  */

/* Omit E_c_plus3 to avoid collinearity.                               */

 

regress schstdks4_cappedpts E_c_minus4 E_c_minus3 E_c_minus2 E_c_minus1 E_c E_c_plus1 E_c_plus2 i.year schks2_eng_exp schks2_eng_abv schks2_mat_exp schk

> s2_mat_abv schks2_sci_exp schks2_sci_abv schfemaleschfsmschsenschwhiteschblackschasian, vce(cluster schid) cformat(%9.3f) pformat(%5.3f) sformat(%8

> .3f)

Linear regression                                      Number of obs =    1264

F( 30,   157) =   45.97

Prob> F      =  0.0000

R-squared     =  0.6193

Root MSE      =  .24463

 

(Std. Err. adjusted for 158 clusters in schid)

——————————————————————————–

|               Robust

schstdks4_ca~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————+—————————————————————-

E_c_minus4 |     -0.238      0.095   -2.516   0.013       -0.425      -0.051

E_c_minus3 |     -0.217      0.080   -2.706   0.008       -0.375      -0.059

E_c_minus2 |     -0.175      0.072   -2.439   0.016       -0.316      -0.033

E_c_minus1 |     -0.160      0.064   -2.524   0.013       -0.286      -0.035

E_c |     -0.139      0.051   -2.694   0.008       -0.240      -0.037

E_c_plus1 |     -0.069      0.036   -1.891   0.060       -0.141       0.003

E_c_plus2 |     -0.015      0.024   -0.602   0.548       -0.062       0.033

|

year |

2003  |     -0.107      0.097   -1.108   0.269       -0.298       0.084

2004  |     -0.158      0.108   -1.457   0.147       -0.372       0.056

2005  |     -0.256      0.122   -2.091   0.038       -0.497      -0.014

2006  |     -0.249      0.132   -1.885   0.061       -0.510       0.012

2007  |     -0.257      0.143   -1.798   0.074       -0.538       0.025

2008  |     -0.212      0.153   -1.384   0.168       -0.513       0.090

2009  |     -0.194      0.161   -1.207   0.229       -0.512       0.123

2010  |     -0.168      0.170   -0.991   0.323       -0.503       0.167

2011  |     -0.171      0.175   -0.975   0.331       -0.517       0.175

2012  |     -0.223      0.184   -1.216   0.226       -0.586       0.139

2013  |     -0.246      0.192   -1.278   0.203       -0.625       0.134

|

schks2_eng_exp |      0.835      0.226    3.698   0.000        0.389       1.281

schks2_eng_abv |      1.463      0.267    5.481   0.000        0.936       1.991

schks2_mat_exp |      0.712      0.226    3.145   0.002        0.265       1.159

schks2_mat_abv |      0.845      0.278    3.045   0.003        0.297       1.393

schks2_sci_exp |     -0.309      0.292   -1.061   0.290       -0.886       0.267

schks2_sci_abv |     -0.187      0.292   -0.639   0.524       -0.764       0.391

schfemale |      0.169      0.091    1.862   0.064       -0.010       0.348

schfsm |     -0.263      0.118   -2.234   0.027       -0.496      -0.031

schsen |     -0.146      0.098   -1.482   0.140       -0.340       0.049

schwhite |     -0.009      0.160   -0.056   0.955       -0.324       0.306

schblack |      0.326      0.212    1.541   0.125       -0.092       0.744

schasian |      0.190      0.211    0.904   0.367       -0.226       0.606

_cons |     -0.839      0.326   -2.571   0.011       -1.483      -0.194

——————————————————————————–

/* Eyles and Machin, Table 6, column 7, add fixed school effect.  */

/* Omit E_c_plus3 to avoid collinearity.                          */

regress schstdks4_cappedpts E_c_minus4 E_c_minus3 E_c_minus2 E_c_minus1 E_c E_c_plus1 E_c_plus2 i.yeari.schid schks2_eng_exp schks2_eng_abv schks2_mat_exp schks2_mat_abv schks2_sci_exp schks2_sci_abv schfemaleschfsmschsenschwhiteschblackschasian, vce(cluster schid) cformat(%9.3f) pformat(%5.3f) sformat(%8.3f)

note: 9386914.schid omitted because of collinearity

Linear regression                                      Number of obs =    1264

F( 28,   157) =       .

Prob> F      =       .

R-squared     =  0.8243

Root MSE      =  .17783

 

(Std. Err. adjusted for 158 clusters in schid)

——————————————————————————–

|               Robust

schstdks4_ca~s |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]

—————+—————————————————————-

E_c_minus4 |     -0.638      0.127   -5.031   0.000       -0.888      -0.387

E_c_minus3 |     -0.567      0.110   -5.165   0.000       -0.784      -0.350

E_c_minus2 |     -0.479      0.099   -4.827   0.000       -0.675      -0.283

E_c_minus1 |     -0.410      0.085   -4.801   0.000       -0.578      -0.241

E_c |     -0.331      0.067   -4.942   0.000       -0.463      -0.198

E_c_plus1 |     -0.202      0.048   -4.179   0.000       -0.298      -0.107

E_c_plus2 |     -0.087      0.028   -3.080   0.002       -0.143      -0.031

|

year |

2003  |     -0.083      0.062   -1.349   0.179       -0.205       0.039

2004  |     -0.110      0.063   -1.742   0.083       -0.236       0.015

2005  |     -0.183      0.076   -2.403   0.017       -0.333      -0.033

2006  |     -0.227      0.092   -2.460   0.015       -0.410      -0.045

2007  |     -0.270      0.106   -2.551   0.012       -0.479      -0.061

2008  |     -0.281      0.124   -2.270   0.025       -0.525      -0.037

2009  |     -0.316      0.137   -2.297   0.023       -0.587      -0.044

2010  |     -0.337      0.153   -2.206   0.029       -0.639      -0.035

2011  |     -0.368      0.162   -2.267   0.025       -0.689      -0.047

2012  |     -0.468      0.179   -2.612   0.010       -0.822      -0.114

2013  |     -0.565      0.195   -2.897   0.004       -0.949      -0.180

|

schid |

2056905  |     -0.327      0.083   -3.943   0.000       -0.491      -0.163

2066906  |     -0.372      0.065   -5.704   0.000       -0.500      -0.243

2096905  |      0.292      0.077    3.789   0.000        0.140       0.444

2096906  |     -0.674      0.073   -9.197   0.000       -0.819      -0.529

2096907  |     -0.336      0.172   -1.951   0.053       -0.676       0.004

2106907  |     -0.180      0.145   -1.244   0.216       -0.466       0.106

2106908  |     -0.354      0.149   -2.372   0.019       -0.648      -0.059

2106909  |     -0.351      0.106   -3.302   0.001       -0.561      -0.141

2106910  |     -0.173      0.193   -0.896   0.371       -0.554       0.208

2106911  |      0.097      0.054    1.783   0.077       -0.010       0.203

2106912  |     -0.384      0.162   -2.374   0.019       -0.704      -0.065

2126905  |      0.129      0.062    2.069   0.040        0.006       0.252

2136905  |     -0.466      0.092   -5.041   0.000       -0.649      -0.283

2136906  |     -0.667      0.100   -6.702   0.000       -0.864      -0.471

2136908  |     -0.183      0.075   -2.440   0.016       -0.332      -0.035

3036906  |     -0.020      0.020   -0.984   0.327       -0.060       0.020

3046907  |     -0.328      0.160   -2.049   0.042       -0.645      -0.012

3046908  |     -0.487      0.183   -2.664   0.009       -0.849      -0.126

3066905  |     -0.245      0.123   -1.995   0.048       -0.487      -0.002

3066906  |      0.355      0.070    5.089   0.000        0.217       0.493

3066907  |     -0.189      0.049   -3.874   0.000       -0.286      -0.093

3066908  |     -0.001      0.133   -0.004   0.996       -0.264       0.263

3066909  |     -0.056      0.089   -0.622   0.535       -0.232       0.121

3086906  |     -0.245      0.089   -2.751   0.007       -0.421      -0.069

3126906  |     -0.577      0.054  -10.756   0.000       -0.683      -0.471

3156905  |     -0.295      0.060   -4.930   0.000       -0.413      -0.177

3156906  |     -0.724      0.080   -9.011   0.000       -0.882      -0.565

3206905  |     -0.508      0.097   -5.219   0.000       -0.700      -0.316

3306905  |     -0.078      0.196   -0.397   0.692       -0.464       0.309

3306907  |     -0.082      0.050   -1.649   0.101       -0.180       0.016

3306908  |     -0.024      0.176   -0.134   0.894       -0.371       0.324

3306909  |     -0.159      0.117   -1.355   0.177       -0.390       0.073

3316905  |     -0.046      0.028   -1.661   0.099       -0.101       0.009

3336906  |     -0.431      0.180   -2.397   0.018       -0.787      -0.076

3336907  |     -0.143      0.104   -1.377   0.171       -0.347       0.062

3336908  |     -0.307      0.070   -4.377   0.000       -0.446      -0.169

3336909  |     -0.072      0.026   -2.759   0.006       -0.124      -0.021

3336910  |     -0.251      0.070   -3.571   0.000       -0.389      -0.112

3346905  |     -0.460      0.051   -9.016   0.000       -0.561      -0.359

3346906  |     -0.027      0.023   -1.171   0.243       -0.074       0.019

3346924  |      0.188      0.069    2.737   0.007        0.052       0.324

3356906  |     -0.026      0.032   -0.819   0.414       -0.090       0.037

3356907  |     -0.497      0.063   -7.845   0.000       -0.622      -0.372

3366905  |     -0.352      0.076   -4.599   0.000       -0.503      -0.201

3416905  |     -0.471      0.085   -5.569   0.000       -0.638      -0.304

3416906  |     -0.184      0.069   -2.667   0.008       -0.321      -0.048

3506905  |     -0.397      0.192   -2.070   0.040       -0.775      -0.018

3506906  |     -0.073      0.044   -1.648   0.101       -0.160       0.014

3526908  |     -0.462      0.058   -7.939   0.000       -0.577      -0.347

3526909  |     -0.394      0.056   -7.043   0.000       -0.505      -0.284

3526910  |     -0.198      0.102   -1.940   0.054       -0.400       0.004

3526911  |     -0.543      0.086   -6.310   0.000       -0.713      -0.373

3546905  |     -0.206      0.059   -3.481   0.001       -0.322      -0.089

3556905  |     -0.411      0.073   -5.627   0.000       -0.556      -0.267

3556906  |     -0.247      0.027   -9.011   0.000       -0.301      -0.193

3566905  |     -0.646      0.066   -9.725   0.000       -0.777      -0.515

3576905  |     -0.259      0.058   -4.473   0.000       -0.373      -0.144

3576906  |     -0.068      0.050   -1.356   0.177       -0.168       0.031

3706905  |     -0.475      0.074   -6.398   0.000       -0.621      -0.328

3716905  |     -0.280      0.077   -3.654   0.000       -0.432      -0.129

3716906  |     -0.111      0.029   -3.789   0.000       -0.169      -0.053

3716907  |      0.066      0.023    2.822   0.005        0.020       0.112

3726905  |     -0.095      0.024   -3.907   0.000       -0.142      -0.047

3736905  |     -0.696      0.082   -8.477   0.000       -0.858      -0.534

3736906  |     -0.620      0.061  -10.145   0.000       -0.741      -0.500

3736907  |     -0.170      0.072   -2.355   0.020       -0.312      -0.027

3806905  |      0.470      0.097    4.865   0.000        0.279       0.660

3806906  |     -0.682      0.068  -10.076   0.000       -0.815      -0.548

3806907  |     -0.352      0.027  -13.069   0.000       -0.406      -0.299

3806908  |     -0.552      0.227   -2.435   0.016       -1.000      -0.104

3836905  |     -0.593      0.049  -11.988   0.000       -0.691      -0.495

3836906  |     -0.134      0.036   -3.721   0.000       -0.205      -0.063

3836907  |     -0.527      0.067   -7.864   0.000       -0.659      -0.394

3846905  |      0.495      0.043   11.595   0.000        0.411       0.580

3916905  |     -0.590      0.075   -7.897   0.000       -0.738      -0.443

3946905  |     -0.477      0.043  -11.187   0.000       -0.561      -0.393

3946906  |      0.120      0.026    4.549   0.000        0.068       0.173

3946907  |     -0.149      0.039   -3.856   0.000       -0.225      -0.072

8016907  |     -0.429      0.028  -15.469   0.000       -0.483      -0.374

8016910  |     -0.307      0.046   -6.631   0.000       -0.398      -0.215

8016911  |     -0.335      0.041   -8.164   0.000       -0.416      -0.254

8016912  |     -0.294      0.035   -8.453   0.000       -0.363      -0.226

8016913  |     -0.252      0.091   -2.785   0.006       -0.432      -0.073

8036906  |      0.251      0.038    6.550   0.000        0.176       0.327

8036907  |     -0.067      0.019   -3.488   0.001       -0.105      -0.029

8036908  |      0.168      0.046    3.643   0.000        0.077       0.259

8066907  |      0.311      0.079    3.939   0.000        0.155       0.467

8106905  |     -0.041      0.046   -0.898   0.371       -0.133       0.050

8106906  |     -0.158      0.035   -4.485   0.000       -0.227      -0.088

8126905  |     -0.355      0.046   -7.726   0.000       -0.445      -0.264

8126906  |     -0.345      0.063   -5.514   0.000       -0.469      -0.221

8126907  |     -0.476      0.050   -9.500   0.000       -0.575      -0.377

8136905  |     -0.150      0.064   -2.352   0.020       -0.275      -0.024

8216905  |     -0.104      0.037   -2.784   0.006       -0.178      -0.030

8216906  |     -0.609      0.066   -9.224   0.000       -0.739      -0.478

8256905  |     -0.331      0.078   -4.248   0.000       -0.485      -0.177

8266905  |     -0.135      0.064   -2.110   0.036       -0.262      -0.009

8316905  |      0.206      0.068    3.003   0.003        0.070       0.341

8416905  |     -0.436      0.044   -9.869   0.000       -0.523      -0.349

8516905  |     -0.545      0.048  -11.340   0.000       -0.640      -0.450

8526905  |     -0.283      0.040   -7.155   0.000       -0.362      -0.205

8526906  |     -0.314      0.042   -7.461   0.000       -0.398      -0.231

8656905  |     -0.089      0.020   -4.391   0.000       -0.129      -0.049

8666905  |     -0.509      0.058   -8.799   0.000       -0.623      -0.395

8706905  |     -0.813      0.075  -10.899   0.000       -0.960      -0.665

8716905  |     -0.363      0.085   -4.268   0.000       -0.531      -0.195

8746905  |      0.019      0.071    0.269   0.788       -0.121       0.159

8746906  |     -0.187      0.024   -7.707   0.000       -0.235      -0.139

8816905  |     -0.197      0.031   -6.360   0.000       -0.258      -0.136

8816906  |      0.206      0.034    6.050   0.000        0.139       0.273

8816907  |     -0.160      0.024   -6.613   0.000       -0.207      -0.112

8816909  |     -0.201      0.026   -7.790   0.000       -0.252      -0.150

8816910  |     -0.330      0.028  -11.822   0.000       -0.385      -0.275

8836906  |      0.091      0.033    2.778   0.006        0.026       0.156

8846905  |     -0.016      0.036   -0.445   0.657       -0.086       0.054

8856905  |     -0.113      0.023   -4.910   0.000       -0.158      -0.067

8866908  |     -0.425      0.062   -6.818   0.000       -0.548      -0.302

8866909  |     -0.352      0.047   -7.526   0.000       -0.444      -0.260

8866910  |      0.212      0.043    4.945   0.000        0.127       0.297

8866913  |      0.029      0.049    0.597   0.552       -0.067       0.125

8866914  |     -0.172      0.024   -7.163   0.000       -0.220      -0.125

8866915  |     -0.274      0.028   -9.634   0.000       -0.330      -0.217

8866916  |     -0.070      0.037   -1.869   0.064       -0.144       0.004

8876905  |     -0.214      0.040   -5.405   0.000       -0.292      -0.136

8886905  |     -0.108      0.058   -1.884   0.061       -0.222       0.005

8886906  |     -0.238      0.087   -2.736   0.007       -0.410      -0.066

8896905  |     -0.119      0.029   -4.070   0.000       -0.176      -0.061

8916905  |     -0.321      0.033   -9.865   0.000       -0.385      -0.257

8926906  |     -0.570      0.047  -12.012   0.000       -0.664      -0.476

8926907  |      0.227      0.082    2.784   0.006        0.066       0.388

8926919  |     -0.417      0.042  -10.043   0.000       -0.499      -0.335

8946905  |      0.070      0.048    1.461   0.146       -0.025       0.165

8946906  |     -0.100      0.029   -3.481   0.001       -0.157      -0.043

9096905  |     -0.371      0.024  -15.695   0.000       -0.418      -0.325

9096906  |     -0.387      0.030  -13.039   0.000       -0.445      -0.328

9096907  |     -0.087      0.030   -2.890   0.004       -0.147      -0.028

9096908  |     -0.259      0.029   -8.843   0.000       -0.317      -0.201

9196905  |     -0.288      0.031   -9.324   0.000       -0.349      -0.227

9196906  |     -0.310      0.043   -7.182   0.000       -0.396      -0.225

9256905  |     -0.012      0.038   -0.310   0.757       -0.086       0.063

9256906  |      0.052      0.034    1.541   0.125       -0.015       0.119

9256907  |      0.574      0.084    6.816   0.000        0.408       0.741

9256908  |     -0.279      0.037   -7.514   0.000       -0.353      -0.206

9256909  |      0.123      0.023    5.238   0.000        0.077       0.169

9266905  |     -0.354      0.044   -8.092   0.000       -0.441      -0.268

9266906  |     -0.595      0.045  -13.114   0.000       -0.684      -0.505

9286906  |     -0.595      0.043  -13.982   0.000       -0.680      -0.511

9286907  |      0.611      0.076    8.068   0.000        0.462       0.761

9286908  |     -0.200      0.017  -11.793   0.000       -0.233      -0.166

9286909  |     -0.266      0.017  -15.955   0.000       -0.299      -0.233

9296906  |      0.031      0.039    0.783   0.435       -0.047       0.108

9316905  |     -0.436      0.042  -10.276   0.000       -0.520      -0.352

9316906  |     -0.378      0.037  -10.131   0.000       -0.452      -0.304

9386911  |     -0.286      0.019  -15.105   0.000       -0.323      -0.249

9386912  |     -0.225      0.009  -25.877   0.000       -0.242      -0.208

9386913  |     -0.044      0.028   -1.583   0.115       -0.099       0.011

9386914  |      0.000  (omitted)

|

schks2_eng_exp |      0.600      0.190    3.161   0.002        0.225       0.975

schks2_eng_abv |      0.541      0.216    2.507   0.013        0.115       0.968

schks2_mat_exp |      0.336      0.200    1.684   0.094       -0.058       0.731

schks2_mat_abv |      0.281      0.258    1.088   0.278       -0.229       0.792

schks2_sci_exp |     -0.368      0.225   -1.633   0.105       -0.813       0.077

schks2_sci_abv |     -0.314      0.228   -1.376   0.171       -0.765       0.137

schfemale |      0.007      0.156    0.048   0.962       -0.300       0.315

schfsm |     -0.067      0.144   -0.466   0.642       -0.351       0.217

schsen |     -0.045      0.087   -0.516   0.606       -0.216       0.127

schwhite |     -0.266      0.161   -1.653   0.100       -0.583       0.052

schblack |     -0.274      0.286   -0.960   0.339       -0.839       0.290

schasian |      0.137      0.328    0.418   0.677       -0.510       0.784

_cons |      0.535      0.356    1.504   0.135       -0.168       1.239

——————————————————————————–

10) All schools in England can now become an academy if they wish. On the basis of the results discussed in these two papers – and those produced in Question 9) – would you be confident that all schools would, on average, see a boost to their exam performance in the years following conversion? Explain why or why not. 

Both the Eyles and Machin and question 9 results are based on a selection of schools that chose to convert to academy schools, and I believe the results derived from this sample will not be externally valid because of selection bias. The results of the Clark paper are more encouraging, but I would not be confident that schools who have not chosen to convert to academy would see gains that are on average greater than zero.