Effects of a simulation-based training on students conceptual understanding of the Binomial test
DOI:
https://doi.org/10.52041/iase25.106Abstract
Significance tests are used intensively in quantitative empirical research and are also taught at schools and universities. However, even experts and statistics lecturers are subject to misconceptions when interpreting significance tests. Our study focuses on the binomial test, and examines the extent to which a refresher course that focuses on a simulation-based approach (using CODAP) is more conducive to learning than typical education on the binomial test that focuses on calculations. While the conceptual knowledge in the experimental group with simulations improved slightly more than in the control group, the students in the control group showed more improvement in procedural knowledge. However, the pre-test performance was weak overall and only a slight increase was observed in both groups after the 100 minutes of training. A comprehensive development of understanding hypothesis testing is important in teaching, and the results suggest that this cannot be sufficiently improved by our short training session.References
Binder, K. & Erickson, T. (2026). Traditional tests through a randomization lens: treating p-values as data with CODAP [Manuscript submitted for publication].
Budgett, S., & Wild, C. J. (2014). Students’ visual reasoning and the randomization test. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in statistics education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9). ISI/IASE.
Case, C., Battles, M., & Jacobbe, T. (2019). Toward an understanding of p-values: Simulation-based inference in a traditional statistics course. Investigations in Mathematics Learning, 11(3), 195–206. https://doi.org/10.1080/19477503.2018.1438869
Chance, B., Tintle, N., Reynolds, S., Patel, A., Chan, K., & Leader, S. (2022). Student performance in curricula centered on simulation-based inference. Statistics Education Research Journal, 21(3), Article 4. https://doi.org/10.52041/serj.v21i3.6
Chandrakantha, L. (2020). Visualizing the p-value and understanding hypothesis testing concepts using simulations in R. Electronic Journal of Mathematics & Technology, 14(3).
Cobb, G. W. (2007). The introductory statistics course: A Ptolemaic curriculum? Technology Innovations in Statistics Education, 1(1). https://doi.org/10.5070/T511000028
Cumming, G. (2012). Understanding The New Statistics. Effect Sizes, Confidence Intervals, and Meta- Analysis, Routledge. https://doi.org/10.4324/9780203807002
Estrella, Soledad (2025, July 28 – August, 2). What does it mean to prepare our boys and girls to interact with Artificial Agents? Perspectives from early statistical education in today’s classroom [Plenary session]. 48th Conference of the International Group for the Psychology of Mathematics Education, Santiago, Chile. https://eventos.cmm.uchile.cl/pme48/program/
Gigerenzer, G. (2004). Mindless statistics. The Journal of Socio-Economics, 33(5), 587–606. https://doi.org/10.1016/j.socec.2004.09.033
Gigerenzer, G., Krauss, S., & Vitouch, O. (2004). The null ritual. In D. Kaplan (Ed.), The SAGE Handbook of Quantitative Methodology for the Social Sciences (pp. 391–408). SAGE Publications.
Haller, H., & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers. Methods of Psychological Research, 7(1), 1–20.
Herrera-Bennett, A. C., Heene, M., Lakens, D., & Ufer, S. (2020). Improving statistical inferences: Can a MOOC reduce statistical misconceptions about p-values, confidence intervals, and Bayes factors?. PsyArXiv. https://doi.org/10.31234/osf.io/zt3g9
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 21, 1157–1164. https://doi.org/10.3758/s13423-013-0572-3
Jamie, D. M. (2002). Using computer simulation methods to teach statistics: A review of the literature. Journal of Statistics Education, 10(1). https://doi.org/10.1080/10691898.2002.11910548
Oakes, M. (1986). Statistical significance: a commentary for the social and behavioural sciences. Wiley.
Podworny, S. (2018). Students’ Reflections About a Course for Learning Inferential Reasoning Via Simulations. In C. Batanero, & E. Chernoff (Eds.), Teaching and Learning Stochastics: Advances in Probability Education Research (pp. 333–349). Springer. https://doi.org/10.1007/978-3-319- 72871-1_19
Podworny, S., & Biehler, R. (2022). The process of actively building a model for a randomization test– insights into learners’ modeling activities based on a case study. Mathematical Thinking and Learning, 24(4), 291–311. https://doi.org/10.1080/10986065.2021.1922837
Rößner, M., Binder, K., & Ufer, S. (2025). Simulationsbasiert Signifikanztests verstehen. Mathematica Didactica, 48(2), 1–22. https://doi.org/10.18716/ojs/md/2025.2296
Trafimow, D., & Marks, M. (2015). Editorial. Basic and Applied Social Psychology, 37(1), 1–2. http://dx.doi.org/10.1080/01973533.2015.1012991
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129–133. https://doi.org/10.1080/00031305.2016.1154108
Zavez, K., & Harel, O. (2025). Teaching Statistical Concepts Using Computing Tools: A Review of the Literature. Journal of Statistics and Data Science Education. https://doi.org/10.1080/26939169.2024.2445541
