Education research is not in crisis, and that is a problem
Education research appears to be running smoothly, with an ever increasing number of new studies being published. Teachers, policy makers, and academics all rely on this growing literature. Yet underneath lurk systematic problems that are not yet being faced. Scientific progress requires research that is transparent and verifiable. However, data and analyses are rarely made available, thus obfuscating the chain of evidential reasoning which should be fully transparent and available for scrutiny. Similarly, education research relies on a publication process that comes with steep access barriers. A preference for ‘interesting results’ leads to a biased literature which hinders cumulative science. Efforts to replicate and confirm findings are all too rare and not incentivized. Finally, substandard measurement practices lead to ‘new’ phenomena being discovered which turn out to be known findings hiding under an alias. In response to similar problems, psychological science is already going through a crisis and must continue to struggle with this for years to come. For the sake of education research, it too should face its own crisis. In this ‘call for crisis’ talk I will take you through the evidential basis of education research and the problems therewith.
Tim van der Zee is a PhD Student at Leiden University, The Netherlands. He studies how people learn from instructional videos in open online education. He is also strongly interested in meta-scientific efforts to estimate and increase the reliability of (education) science. As such, he advocates for increasing the transparency of scientific research, to systematically engage in replication efforts, and to make substantial changes to the publication process. For his efforts to identify large amounts of errors in the scientific literature he was named ‘most important researcher of the year 2017’ at Leiden University.