A design research project on fairness in data-driven algorithmic decision-making
DOI:
https://doi.org/10.52041/iase25.139Abstract
Data-driven algorithmic decision-making systems, including many AI technologies, are ubiquitous in today’s society. Early engagement with these systems and the promotion of critical statistical literacy are therefore essential. This involves not only fostering a basic understanding of how such systems work, but also addressing their broader social impact. As part of a design research project, we developed a learning activity that enables upper secondary students and pre-service teachers to explore issues of fairness in the context of automated credit granting. In this paper, we present the design of the activity, outline the intended learning trajectory, and report on initial implementations with pre-service teachers. We also discuss preliminary findings from a qualitative analysis of students’ fairness-related arguments.References
Barocas, S., Hardt, M., & Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities. MIT Press. https://fairmlbook.org/
Caton, S., & Haas, C. (2024). Fairness in Machine Learning: A Survey. ACM Computing Surveys, 56(7), Article 166. https://doi.org/10.1145/3616865
Gal, I. (2002). Adults’ statistical literacy: Meaning, components, responsibilities. International Statistical Review, 70(1), 1–25. https://doi.org/10.1111/j.1751-5823.2002.tb00336.x
Gravemeijer, K. P. E., & Cobb, P. (2006). Design research from a learning design perspective. In Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.). Educational Design Research (pp. 45–85). Taylor and Francis Ltd.
Gould, R. (2010). Statistics and the modern student. International Statistical Review, 78(2), 297–315. https://doi.org/10.1111/j.1751-5823.2010.00117.x
Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. Proceedings of the 30th International Conference on Neural Information Processing Systems (pp. 3323–3331). Curran Associates Inc. https://dl.acm.org/doi/10.5555/3157382.3157469
Hilger, S., & Büscher, C. (in press). Richness of reflective processes on algorithmic decision-making systems by prospective teachers. Proceedings of CERME 14.
O’Neil, C. (2016). Weapons of Math Destruction. How big data increases inequality and threatens democracy. Penguin Books.
Orwat, C. (2019). Risks of Discrimination through the Use of Algorithms. Federal Anti-Discrimination Agency www.antidiskriminierungsstelle.de/SharedDocs/downloads/EN/publikationen/Studie_en_Diskrimi nierungsrisiken_durch_Verwendung_von_Algorithmen.pdf?__blob=publicationFile&v=2
Pessach, D., & Shmueli, E. (2020). A Review on Fairness in Machine Learning. ACM Computing Surveys, 55(3), Article 51. https://doi.org/10.1145/3494672
Schönbrodt, S., Schneider, S., Podworny, S., & Camminady, T. (2025). A learning activity on fairness in data-driven algorithmic decision-making systems. Teaching Statistics. 48, 33–44. https://doi.org/10.1111/test.70016
Wattenberg, M., Viégas, F., & Hardt, M. (n.d.) Attacking discrimination with smarter machine learning. https://research.google.com/bigpicture/attacking-discrimination-in-ml/
Weiland, T. (2017). Problematizing statistical literacy: An intersection of critical and statistical literacies. Educational Studies in Mathematics, 96, 33–47. https://doi.org/10.1007/s10649-017- 9764-5
Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67(3), 223–248. https://doi.org/10.1111/j.1751-5823.1999.tb00442.x
