Esther Duflo, the co-director of the Abdul Latif Jameel Poverty Action Lab at MIT answers reader questions at Managing Globalization, one of the International Herald Tribunes great blogs.
One question/answer particularly caught my former epidemiologist eye:
Q. How far can conclusions derived from randomized controlled trials be stretched when it comes to policy prescription to tackle poverty? If we find out that a certain intervention is having a positive impact on the fight against poverty, then how appropriate would it be to prioritize the intervention at a macro level and deduce national economic policy based on the result of the intervention? Besides local political and economic institutions, what other factors should we be careful of when scaling up policies which are rigorously and successfully tested at a micro level?
When we run an experiment and we get the results, we know the effect of this program had in this particular place. This is much better than the information we have in general to decide on policy (nothingâ€¦), but is it good enough to act on and to move on to recommend a more general policy? There are several obstacles.
First, the results may not replicate across contexts; I discuss that in the next answer.
Second, a program may be implemented in very different ways in a large scale. For example, it may be done well by a non-governmental organization, but corruption problems may creep in when it is implemented by a government. These implementation issues will have to be ironed out. This is important, and scaling up challenges have to be considered, but it does not take away from the finding that we now know what the potential of the program would be if it were correctly implemented. If we find an effective program, this suggests that it is worth investing some effort in figuring out how to correctly implement it on a large scale. This can also be experimented with, by the way: some of the very exciting work in development economics these days is precisely about how to effectively implement programs (see for example Ben Olkenâ€™s work, which I discussed in response to another question).
Third, there may be market equilibrium effects. For example, if I find that by randomly offering secondary school scholarship to some kids, I increase their wage, compared to those who did not receive the scholarships, this may not tell me what the effect of doing this nationwide would be: if everybody received a secondary education, the returns to secondary school may go up or down, compared to a situation where few people received a secondary school education. There are two ways to deal with these: in some cases, it may be possible to organize experiments at the â€œmarketâ€ level (though I think it would be hard in the example I just described). In others, we have to use a priori economic reasoning to think whether market equilibrium effects are going to be important or not. In many cases, we have no reason to think they would be large enough to undo the effect of the policy.