Fairness in machine learning: A design research study for secondary education
DOI:
https://doi.org/10.52041/iase25.147Abstract
This conceptual paper presents a design research study that explores fairness in machine learning (ML) as an interdisciplinary learning opportunity for secondary education. Drawing on the societal relevance of algorithmic decision-making systems, the study emphasizes the importance of integrating technical and ethical perspectives within a cohesive teaching-learning arrangement – an approach that is still rarely implemented in practice. The paper provides an overview of fairness definitions and algorithmic intervention strategies, alongside a review of relevant educational research on ML and fairness in both school and higher education contexts. It outlines the methodological foundations of the design research study and introduces preliminary ideas for a prototypical teaching-learning arrangement, accompanied by guiding research questions that frame the study.References
Ajunwa, I., & Greene, D. (2019). Platforms at work: Automated hiring platforms and other new intermediaries in the organization of work. In K. Vallas & A. Kovalainen (Eds.), Work and labor in the digital age (Vol. 33, pp. 61–91). Emerald Publishing. https://doi.org/10.1108/S0277- 283320190000033005
Akgun, S., & Greenhow, C. (2022). Artificial intelligence in education: Addressing ethical challenges in K-12 settings. AI and Ethics, 2(3), 431–440. https://doi.org/10.1007/s43681-021-00096-7
Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. (2019). Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In Proceedings of the International Workshop on Education in Artificial Intelligence K-12 (EDUAI’19), IJCAI 2019 (pp. 1–4). IJCAI https://robots.media.mit.edu/wp- content/uploads/sites/7/2019/08/Constructionism__Ethics__and_Creativity.pdf
Ärlebäck, J. B., & Frejd, P. (2025). Exploring Swedish secondary students’ mathematical models of fairness in ranking real-life, multi-faceted heptathlon outcomes. ZDM – Mathematics Education, 57, 229–244. https://doi.org/10.1007/s11858-025-01678-z
Baumer, B. S., Garcia, R. L., Kim, A. Y., Kinnaird, K. M., & Ott, M. Q. (2020). Integrating data science ethics into an undergraduate major: A case study. Smith College, Statistical and Data Sciences. https://scholarworks.smith.edu/sds_facpubs/20
Bilstrup, K.-E. K., Kaspersen, M. H., & Petersen, M. G. (2020). Staging reflections on ethical dilemmas in machine learning: A card-based design workshop for high school students. In Proceedings of the 2020 ACM Designing Interactive Systems Conference, (pp. 1211–1222). https://doi.org/10.1145/3357236.3395558
Borenstein, J., & Howard, A. (2021). Emerging challenges in AI and the need for AI ethics education. AI and Ethics, 1(1), 61–65. https://doi.org/10.1007/s43681-020-00002-7
Caton, S., & Haas, C. (2024). Fairness in machine learning: A survey. ACM Computing Surveys, 56(7), Article 166. https://doi.org/10.1145/3616865
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 797–806). https://doi.org/10.1145/3097983.3098095
Dabbagh, H., Earp, B. D., Mann, S. P., Plozza, M., Salloch, S., & Savulescu, J. (2025). AI ethics should be mandatory for schoolchildren. AI and Ethics, 5(1), 87–92. https://doi.org/10.1007/s43681-024- 00462-1
Dua, D., & Graff, C. (2017). UCI machine learning repository [Data set]. University of California, Irvine. http://archive.ics.uci.edu/ml
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, (pp. 214–226). https://doi.org/10.1145/2090236.2090255
Frank, M., & Roeckerath, C. (Eds.). (2022). Neue Materialien für einen realitätsbezogenen Mathematikunterricht 9: Mathematische Modellierung in interdisziplinären Projekten für die Sekundarstufe (ab Klasse 9). [New materials for reality-based mathematics education 9: Mathematical modeling in interdisciplinary projects for secondary education (from grade 9 onwards)] ISTRON-Schriftenreihe. Springer Spektrum. https://doi.org/10.1007/978-3-662-63647- 3
Gravemeijer, K. P. E., & Cobb, P. (2006). Design research from a learning design perspective. In J. Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational Design Research (pp. 45– 85). Taylor & Francis.
Grosz, B. J., Grant, D. G., Vredenburgh, K., Behrends, J., Hu, L., Simmons, A., & Waldo, J. (2019). Embedded EthiCS: Integrating ethics across CS education. Communications of the ACM, 62(8), 54– 61. https://doi.org/10.1145/3330794
Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, (Vol. 29). https://doi.org/10.48550/arXiv.1610.02413
Hofmann, S., Bata, K., Schönbrodt, S., Kindler, S., Frank, M., Müller, C., Eckert, J., Hörter, J., Büsing, C., & Stamm, B. (in press). Mathematical modeling in action: Design principles for mathematical modeling days. PAMM.
Kaspersen, M. H., Bilstrup, K.-E. K., Van Mechelen, M., Hjorth, A., Bouvin, N. O., & Petersen, M. G. (2021). VotestratesML: A high school learning tool for exploring machine learning and its societal implications. In FabLearn Europe / MakeEd 2021 - An International Conference on Computing, Design and Making in Education, 1–10. https://doi.org/10.1145/3466725.3466728
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017). Inherent trade-offs in the fair determination of risk scores. In Proceedings of the International Conference on Theory of Computing (ITCS) 2017 (Vol. 67, Article 43, pp. 43:1–43:23). https://doi.org/10.4230/LIPICS.ITCS.2017.43
Larson, S., Mattu, L., Kirchner, L., & Angwin, J. (2016). ProPublica COMPAS analysis data set [Data set]. GitHub. https://github.com/propublica/compas-analysis
Li, J., Wang, W., Yu, J., & Zhu, L. (2016). Young children’s development of fairness preference. Frontiers in Psychology, 7, Article 1274. https://doi.org/10.3389/fpsyg.2016.01274
Liu, S., & Vicente, L. N. (2022). Accuracy and fairness trade-offs in machine learning: A stochastic multi-objective approach. Computational Management Science, 19, 513–537. https://doi.org/10.1007/s10287-022-00425-z
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, (pp. 1–16). https://doi.org/10.1145/3313831.3376727
Marques, L. S., Gresse von Wangenheim, C., & Hauck, J. C. R. (2020). Teaching machine learning in school: A systematic mapping of the state of the art. Informatics in Education, 19(2), 283–321. https://doi.org/10.15388/infedu.2020.14
Martins, R. M., & Gresse von Wangenheim, C. (2023). Findings on teaching machine learning in high school: A ten-year systematic literature review. Informatics in Education, 22(3), 421–440. https://doi.org/10.15388/infedu.2023.18
Mashhadi, A., Zolyomi, A., & Quedado, J. (2022). A case study of integrating fairness visualization tools in machine learning education. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, 1–7. https://doi.org/10.1145/3491101.3503568
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022). A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), Article 115. https://doi.org/10.1145/3457607
Niss, M. (2015). Prescriptive modelling—Challenges and opportunities. In G. A. Stillman, W. Blum, & M. S. Biembengut (Eds.), Mathematical modelling in education research and practice: Cultural, social, and cognitive influences (pp. 67–79). Springer.
Prediger, S., Link, M., Hinz, R., Hußmann, S., Thiele, J., & Ralle, B. (2012). Lehr-Lernprozesse initiieren und erforschen – Fachdidaktische Entwicklungsforschung im Dortmunder Modell. Mathematisch-Naturwissenschaftlicher Unterricht, 65(8), 452–457.
Saltz, J., Skirpan, M., Fiesler, C., Gorelick, M., Yeh, T., Heckman, R., Dewar, N., & Beard, N. (2019). Integrating ethics within machine learning courses. ACM Transactions on Computing Education, 19(4), 32:1–32:26. https://doi.org/10.1145/3341164
Schönbrodt, S., Schneider, S., Podworny, S., & Camminady, T. (in press). A learning activity on fairness in data-driven algorithmic decision-making systems. Teaching Statistics.
Skinner, Z., Brown, S., & Walsh, G. (2020). Children of color’s perceptions of fairness in AI: An exploration of equitable and inclusive co-design. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 1–8. https://doi.org/10.1145/3334480.3382901
Skirpan, M., Beard, N., Bhaduri, S., Fiesler, C., & Yeh, T. (2018). Ethics education in context: A case study of novel ethics activities for the CS classroom. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education, 940–945. https://doi.org/10.1145/3159450.3159573
Skovsmose, O. (2023). Critical Mathematics Education. Springer International Publishing. https://doi.org/10.1007/978-3-031-26242-5
Touretzky, D., Gardner-McCune, C., & Seehorn, D. (2023). Machine learning and the five big ideas in AI. International Journal of Artificial Intelligence in Education, 33(2), 233–266. https://doi.org/10.1007/s40593-022-00314-1
Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. (2019). Envisioning AI for K-12: What should every child know about AI? Proceedings of the AAAI Conference on Artificial Intelligence, 33(1), Article 9795. https://doi.org/10.1609/aaai.v33i01.33019795
UNESCO. (2024). AI competency framework for students. https://doi.org/10.54675/JKJB9835
Verma, S., & Rubin, J. (2018). Fairness definitions explained. In Proceedings of the International Workshop on Software Fairness, 1–7. https://doi.org/10.1145/3194770.3194776
Williams, R., Ali, S., Devasia, N., DiPaola, D., Hong, J., Kaputsos, S. P., Jordan, B., & Breazeal, C. (2023). AI + Ethics curricula for middle school youth: Lessons learned from three project-based curricula. International Journal of Artificial Intelligence in Education, 33(2), 325–383. https://doi.org/10.1007/s40593-022-00298-y
Wittmann, E. Ch. (1995). Mathematics education as a design science. Educational Studies in Mathematics, 29(4), 355–374. https://doi.org/10.1007/BF01273911
Yan, X., Zhou, Y., Mishra, A., Mishra, H., & Wang, B. (2024). Exploring visualization for fairness in AI education. In 2024 IEEE 17th Pacific Visualization Conference (PacificVis), 1–10. https://doi.org/10.1109/PacificVis60374.2024.00010
