LSER 2018 - Workshop on Large Scale Education Replication

The big data revolution in education has led to many shifts and advances in the field, including better prediction of student outcomes and better modeling of student knowledge and engagement. However, many findings appear to contradict each other across datasets, differ from each other in subtle ways, or are hindered by methodological shortcomings (e.g., uncorrected multiple testing for statistical significance). As such, the issues of replication and research synthesis across studies facing the community now loom large, and there is increasing worry about spurious studies being accepted as true, often referred to as the “replication crisis” in other areas of science. This issue is particularly critical in the field of educational data mining, where the sharing of datasets for replication purposes is restricted by educational privacy laws (e.g., FERPA) and many studies evaluate novel educational domains with no clear scientific consensus to guide the interpretation of novel methods and results (e.g., MOOCs). This workshop is intended to build working capital up within the EDM community, focusing on the challenges, opportunities, and best practices for reproducibility, replication, and generalization of EDM findings.
The workshop will be organized as a half day event immediately preceding the main EDM conference. This will include invited discussions on work done in the field to date, as well as a poster session. Participants interested in presenting posters are requested to provide a brief (500-1,000 words) abstract of their interests, work, or positions in the area, and may include topics broadly about education and replication including:
- methods, procedures, and norms (e.g., pre-registration)
- reproducibility best practices
- generalization efforts
- computational infrastructure and architectures for replication
- data set openness, availability, and privacy
- issues of ecological validity relating to replication and generalization
- replication efforts (both failed and successful) in experimental randomized
- controlled trials or predictive models in education
- surveys of successful reproducibility and replication methods and efforts in other related domains (e.g., a survey or case study of how the EDM community can learn from reproducibility efforts in medical research using privacy-restricted data)
Important Dates
June 1, 2018 – Submission of Abstracts (send to replicate.education-AT-umich.edu)
July 15, 2018 – Workshop
Organizers
Christopher Brooks (University of Michigan)
Ryan Baker (University of Pennsylvania)
Neil Heffernan (Worcester Polytechnic Institute)
Joshua Gardner (University of Michigan)
Juan Miguel Andres (University of Pennsylvania)