King's College London

This is the first instalment of a new comment series from the School for Government on the future of evidence-based policy. Look out for more contributions to the series in the coming weeks.

Randomised experiments are commonly described as the “gold standard” in the production of evidence – but when trying to assess the effects of public policies, they are not always possible to carry out. Nevertheless, with more willingness from policymakers, randomised policy experiments could be much more prevalent than they currently are.

Think, for instance, about training programmes for the unemployed, a new policy to fight petty crime, a change in the monitoring intensity to reduce tax avoidance, or anti-discrimination policies in hiring. In all these cases it is technically possible to first implement a new policy at small scale in a randomly selected set of cases or regions. By comparing the relevant outcomes under the new policy with the outcomes in the not-yet-treated control group, high-quality evidence is produced regarding the policy’s effect, which can in turn be used when deciding whether to further implement the policy or reject it. However, despite the simplicity and power of randomised policy experiments, experimentation is typically rare.

Why don’t we see many more policy experiments? When confronted with this question, policymakers often point to voters. Voters would not understand how a randomised policy experiment works, let alone appreciate it. Also, voters may be upset (perhaps even outraged) by the inequality in how citizens or firms are treated by the government during the experimental period. Others argue it would only be ethical to involve citizens in a randomised policy experiment after asking and receiving their consent. Those who do not actively consent should not be involved – a condition which often implies that a randomised policy experiment becomes highly impractical or, at least, much more costly.

But how do voters really think about randomised policy experiments? Do they feel strongly about the often-stated concerns? In the past two years, a series of surveys have been conducted among representative samples in several countries, to address these questions. And the results are, without exception, really encouraging.

In our own study, which took place in the Netherlands, we started by carefully explaining to respondents what a randomised policy experiment is, using a couple of realistic examples such as those mentioned above. We found running randomised experiments on policies like these is supported by a majority of voters. Among the proponents of policy experimentation, voters who do not have a strong opinion about the policy are overrepresented, exactly what one would expect if the purpose of the experiments is to learn whether a policy is actually a good idea or not. And more generally, a majority of voters say they appreciate randomised policy experiments. A sizeable group even says that they would more likely vote for a political party if the party would be in favour of running many more policy experiments. Only a minority of voters seem to care much about inequality of outcomes and informed consent.

Our results have in the meantime been replicated in Australia, Finland, France, Ireland, Italy, Poland, Romania and the US. In all these countries, a vast majority of voters appreciate the use of randomised policy experiments.

So what’s next? We feel it is important to communicate these findings to policymakers as widely as possible. In our study, we followed up on the survey among voters by running a survey among politicians, both from the national and regional parliaments. We showed a random selection of politicians the encouraging results about voters from the first wave of data collection and then asked them about their own opinion. We find that, in response to hearing about voters’ positive assessment, politicians converge to the voters’ opinion about policy experimentation. These results make us optimistic that, when the results of our study and the replications mentioned above become more widely known, we may see many more randomised policy experiments. The future of evidence-based policy thus seems bright.

Robert Dur is Professor of Economics at Erasmus University Rotterdam.