STUDENT PERFORMANCE IN CURRICULA CENTERED ON SIMULATION-BASED INFERENCE

Authors

  • BETH CHANCE California Polytechnic State University
  • NATHAN TINTLE University of Illinois, Chicago
  • SHEA REYNOLDS California Polytechnic State University
  • AJAY PATEL California Polytechnic State University
  • KATHERINE CHAN California Polytechnic State University
  • SEAN LEADER California Polytechnic State University

DOI:

https://doi.org/10.52041/serj.v21i3.6

Keywords:

Statistics education research, Randomization tests, Multi-level models

Abstract

Using simulation-based inference (SBI), such as randomization tests, as the primary vehicle for introducing students to the logic and scope of statistical inference has been advocated with the potential of improving student understanding of statistical inference and the statistical investigative process as a whole. Moving beyond the individual class activity, entirely revised introductory statistics curricula centering on these ideas have been developed and tested. Preliminary assessment data have been largely positive. In this paper, we discuss three years of cross-institutional tertiary-level data from the United States comparing SBI-focused curricula and non-SBI curricula (86 distinct institutions). We examined several pre/post measures of conceptual understanding in the introductory algebra-based course using multi-level modelling to incorporate student-level, instructor-level, and institutional-level covariates. We found that pre-course student characteristics (e.g., prior knowledge) were the strongest predictors of student learning, but also that textbook choice can still have a meaningful impact on student understanding of key statistical concepts. In particular, textbook choice was the strongest “modifiable” predictor of student outcomes of those examined, with simulation-based inference texts yielding the largest changes in student learning outcomes. Further research is needed to elucidate the particular aspects of SBI curricula that contribute to observed student learning gains.

References

Aliaga, M., Cobb, G., Cuff, C., Garfield, J., Gould, R., Lock, R., Moore, T., Rossman, A., Stephenson, B., Utts, J., Velleman, P., & Witmer, J. (2005). Guidelines for Assessment and Instruction in Statistics Education College Report. American Statistical Association. https://www.amstat.org/docs/default-source/amstat-documents/2005gaisecollege_full.pdf

Beckman, M., delMas, R., & Garfield, J. (2017). Cognitive transfer outcomes for a simulation-based introductory statistics curriculum. Statistics Education Research Journal, 16(2), 419–440. https://doi.org/10.52041/serj.v16i2

Case, C., & Jacobbe, T. (2018). A framework to characterize student difficulties in learning information from a simulation-based approach. Statistics Education Research Journal, 17(2), 9–29. https://doi.org/10.52041/serj.v17i2

Chance, B., & McGaughey, K. (2014). Impact of a simulation/randomization-based curriculum on student understanding of p-values and confidence intervals. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://icots.info/icots/9/proceedings/pdfs/ICOTS9_6B1_CHANCE.pdf

Chance, B., & Rossman, A. (2006). Using simulation to teach and learn statistics. In A. Rossman & B. Chance (Eds.), Working Cooperatively in Statistics Education. Proceedings of the Seventh International Conference on Teaching Statistics (ICOTS-7), Salvador, Brazil. International Statistical Institute. www.ime.usp.br/~abe/ICOTS7/Proceedings/PDFs/InvitedPapers/7E1_CHAN.pdf

Chance, B., Wong, J., & Tintle, N. (2017). Student performance in curricula centered on simulation-based inference: A preliminary report. Journal of Statistics Education, 24(3), 114–126. https://doi.org/10.1080/10691898.2016.1223529

Cobb, G. W. (1992). Teaching statistics. In L. A. Steen (Ed.), Heeding the call for change: Suggestions for curriculum action (pp. 3–43). Mathematical Association of America.

Cobb, G. (2007). The introductory statistics course: A Ptolemaic curriculum? Technology Innovations in Statistics Education, 1(1). https://doi.org/10.5070/T511000028

Colt, G. C., Davoudi, M., Murgu, S., & Zamanian Rohani, N. (2011). Measuring learning gain during a one-day introductory bronchoscopy course. Surgical Endoscopy, 25, 207–216.

delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58. https://doi.org/10.52041/serj.v6i2.483

delMas, R., Garfield, J., Ooms, A., & Chance, B. (2007). Assessing students’ conceptual understanding after a first course in statistics. Statistics Education Research Journal, 6(2), 28–58. https://doi.org/10.52041/serj.v6i2.483

Diez, D., Barr, C., & Cetinkeya-Rundel, M. (2014). Introductory statistics with randomization and simulation. Openintro.org. https://openintro.org/book/isrs/

Doerr, H., & English, L. (2003). A modeling perspective on students’ mathematical reasoning about data. Journal for Research in Mathematics Education, 34(2), 110–136. https://doi.org/10.2307/30034902

Fry, E. B. (2014). Introductory statistics instructors’ practices and beliefs regarding technology and pedagogy. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. https://iase-icots/9/proceedings/pdfs/ICOTS9_C202_FRY.pdf?1405041868

Garfield, J. (1995). How students learn statistics. International Statistical Review, 63(1), 25–34.

Garfield, J. & Ben-Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistical Review, 75(3), 372–396.

Garfield, J., delMas, R., & Zieffler, A. (2012). Developing statistical modelers and thinkers in an introductory, tertiary-level statistics course. ZDM Mathematics Education, 44, 883–898. https://doi.org/10.1007/s11858-012-0447-5

Garfield, J., Hogg, B., Schau, C., & Whittinghill, D. (2002). First courses in statistical science: The status of educational reform efforts. Journal of Statistics Education, 10(2). http://dx.doi.org/10.1080/10691898.2002.11910665

Hake, R. R. (1998). Interactive engagement versus traditional methods: A six-thousand student survey of mechanics test data for introductory physics courses, American Journal of Physics, 66, 64–74.

Hildreth, L., Robison-Cox, J., & Schmidt, J. (2018). Comparing student success and understanding in introductory statistics under consensus and simulation-based curricula. Statistics Education Research Journal, 17(1), 103–120. https://doi.org/10.52041/serj.v17i1.178

Konold, C., Harradine, A., & Kazak, S. (2007). Understanding distributions by modeling them. International Journal of Computers for Mathematical Learning, 12, 217–230. https://link.springer.com/article/10.1007/s10758-007-9123-1

Lane-Getaz, S. J. (2007). Toward the development and validation of the reasoning about p-values and statistical significance scale. In B. Philips & L. Weldon (Eds.), Proceedings of the ISI/IASE Satellite Conference on Assessing Student Learning in Statistics. International Statistical Institute. http://www.stat.auckland.ac.nz/~iase/publications/sat07/Lane-Getaz.pdf

Lane-Getaz, S. J. (2017). Is the p-value really dead? Assessing inference learning outcomes for social science students in an introductory statistics course. Statistics Education Research Journal, 16(1), 357–399. https://doi.org/10.52041/serj.v16i1.235

Lee, H. S., Doerr, H. M., Tran, D., & Lovett, J. N. (2015). The role of probability in developing learners’ models of simulation approaches to inference. Statistics Education Research Journal, 15(2), 216–238. https://doi.org/10.52041/serj.v15i2.249

Lock, R., Frazer Lock, P., Lock Morgan, K., Lock, E., & Lock, D. (2013). Statistics: Unlocking the power of data. Wiley.

Lock, R., Frazer Lock, P. F., Lock Morgan, K., Lock, E., & Lock, D. (2014). Intuitive introduction to the important ideas of inference. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://icots.info/icots/9/proceedings/pdfs/ICOTS9_4A3_LOCK.pdf

Malone, C., & Hooks, T. (2012, July 31). Finding an appropriate balance between simulation-based and traditional methods in the teaching of statistical inference [Paper presentation]. Statistics: Growing to Serve a Data-dependent Society, Joint Statistical Meetings, San Diego.

Maurer, K., & Lock, E. (2015). Bootstrapping in the introductory statistics curriculum. Technology Innovations in Statistics Education, 9(1). https://doi.org/10.5070/T591026161

Moore, D. M. (1997). New pedagogy and new content: The case of statistics. International Statistical Review, 65(2), 123–165.

Parker, N., Fry, E., Garfield, J., & Zieffler, A. (2014). Graduate teaching assistants’ beliefs, practices, and preparation for teaching introductory statistics. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_C200_PARKER.pdf?1405041867

Pfannkuch, M., & Budgett, S. (2014). Constructing inferential concepts through bootstrap and randomization-test simulation: A case study. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_8J1_PFANNKUCH.pdf

Posner, M. (2014). A fallacy in student attitudes research: The impact of the first class. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. http://icots.info/9/proceedings/pdfs/ICOTS9_1F3_POSNER.pdf

Reaburn, R. (2014). Introductory statistics course tertiary students’ understanding of p-values. Statistics Education Research Journal, 13(1), 53–65. https://doi.org/10.52041/serj.v13i1.298

Roberts, R., Scheaffer, R., & Watkins, A. (1999). Advanced Placement Statistics: Past, present, and future. The American Statistician, 53(4), 307–320.

Roy, S., & Mcdonnel, T. (2018). Assessing simulation-based inference in secondary schools. Unpublished manuscript. http://www.isi-stats.com/isi/presentations/ICOTS2018-5.pdf

Roy, S., Rossman, A., Chance, B., Cobb, G., VanderStoep, J., Tintle, T., & Swanson, T. (2014). Using simulation/randomization to introduce p-value in Week 1. In K. Makar, B. de Sousa, & R. Gould (Eds.), Sustainability in Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS-9), Flagstaff, Arizona. International Statistical Institute. https://icots.info/9/proceedings/pdfs/ICOTS9_4A2_ROY.pdf

Saldanha, L. A., & Thompson, P. W. (2002). Conceptions of sample and their relationship to statistical inference. Educational Studies in Mathematics, 51, 257–270.

Schau, C. (2003). Survey of Attitudes Toward Statistics (SATS-36). http://evaluationandstatistics.com/

Tabor, J., & Franklin, C. (2013). Statistical reasoning in sports. W.H. Freeman and Company.

Tintle, N. L., Chance, B., Cobb, G., Rossman, A., Roy, S., Swanson, T., & VanderStoep, J. (2015). Introduction to statistical investigations. Wiley.

Tintle, N., Clark, J., Fischer, K., Chance, B., Cobb, G., Roy, S., Swanson, T., & VanderStoep, J. (2018). Assessing the association between pre-course metrics of student preparation and student performance in introductory statistics: Results from early data on simulation-based inference vs. nonsimulation based inference. Journal of Statistics Education, 26(2), 103–109. https://www.tandfonline.com/doi/full/10.1080/10691898.2018.1473061

Tintle, N., Topliff, K., VanderStoep, J., Homes, V.-L., & Swanson, T. (2012). Retention of statistical concepts in a preliminary randomization-based introductory statistics curriculum, Statistics Education Research Journal, 11(1), 21–40. https://doi.org/10.52041/serj.v11i1.340

Tintle, N., VanderStoep, J., Holmes, V-L., Quisenberry, B., & Swanson, T. (2011). Development and assessment of a preliminary randomization-based introductory statistics curriculum, Journal of Statistics Education, 19(1). https://doi.org/10.1080/10691898.2011.11889599

Tobias-Lara, M. G., & Gomez-Blancarte, A. L. (2019). Assessment of informal and formal inferential reasoning: A critical research review. Statistics Education Research Journal, 18(1), 8–25. https://doi.org/10.52041/serj.v18i1.147

van Es, C., & Weaver, M. (2018). Race, sex, and their influences on introductory statistics education. Journal of Statistics Education, 26(1), 48–54. https://www.tandfonline.com/doi/full/10.1080/10691898.2018.1434426

Wild, C. J., Pfannkuch, M., Regan, M., & Horton, N. J. (2011). Towards more accessible conceptions of statistical inference. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174(2), 247–295. https://doi.org/10.1111/j.1467-985X.2010.00678.x

Zieffler, A., & Catalysts for Change. (2015). Statistical thinking: A simulation approach to modeling uncertainty (3rd ed.). Catalyst Press.

Zieffler, A., Garfield, J., Alt, S., Dupuis, D., Holleque, K., & Chang, B. (2008). What does research suggest about the teaching and learning of introductory statistics at the college level? A review of the literature. Journal of Statistics Education, 16(2). http://dx.doi.org/10.1080/10691898.2008.11889566

Zieffler, A., Park, J., Garfield, J. delMas, R., & Bjornsdottier, A. (2012). The Statistics Teaching Inventory: A survey on statistics teachers’ classroom practices and beliefs. Journal of Statistics Education, 20(1). https://doi.org/10.1080/10691898.2012.11889632

Ziegler, L., & Garfield, J. (2018). Developing a statistical literacy assessment for the modern introductory statistics course. Statistics Education Research Journal, 17(2), 161–178. https://doi.org/10.52041/serj.v17i2.164

Downloads

Published

2022-12-01

Issue

Section

Regular Articles