COMPUTES: DEVELOPMENT OF AN INSTRUMENT TO MEASURE INTRODUCTORY STATISTICS INSTRUCTORS’ EMPHASIS ON COMPUTATIONAL PRACTICES
DOI:
https://doi.org/10.52041/serj.v21i1.63Keywords:
Statistics education research, computational thinking, assessment, instructional practicesAbstract
The influx of data and the advances in computing have led to calls to update the introductory statistics curriculum to better meet the needs of the contemporary workforce. To this end, we developed the COMputational Practices in Undergraduate TEaching of Statistics (COMPUTES) instrument, which can be used to measure the extent to which computation practices—specifically data, simulation, and coding practices—are included in the introductory statistics curriculum. Data from 236 instructors were used in a psychometric analysis to evaluate the latent structure underlying instructors’ response patterns and understand the quality of the items. We also examined whether computational practices are being emphasized differently across institutional settings. Results suggest that the latent structure is best captured using a correlated multidimensional model and that most items were contributing information to the measurement process. Across institutional settings, curricular emphasis related to data and simulation practices seem quite similar, while emphasis on coding practices differs.
References
American Educational Research Association. (2014). Standards for educational and psychological testing. American Educational Research Association American Psychological Association National Council on Measurement in Education.
American Statistical Association. (2017). Curriculum guidelines for undergraduate programs in statistical science. American Statistical Association.
Baker, F. B. (2001). The basics of item response theory (2nd ed., ERIC Document Reproduction Service No. ED 458 219). Eric Clearing House on Assessment and Evaluation.
Barr, B. V., & Stephenson, C. (2011). Bringing computational thinking to K–12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905
Baumer, B. S., Cetinkaya-Rundel, M., Bray, A., Loi, L., & Horton, N. J. (2014). R Markdown: Integrating a reproducible analysis tool into introductory statistics. Technology Innovations in Statistics Education, 8(1). https://escholarship.org/uc/item/90b2f5xh
Beckman, M. D., Çetinkaya-Rundel, M., Horton, N. J., Rundel, C. W., Sullivan, A. J, & Tackett, M. (2021). Implementing version control with Git and GitHub as a learning objective in statistics and data science courses. Journal of Statistics and Data Science Education, 29(sup1), S132-S144. https://doi.org/10.1080/10691898.2020.1848485
Benton, L., Hoyles, C., Kalas, I., & Noss, R. (2017). Bridging primary programming and mathematics: Some findings of design research in England. Digital Experiences in Mathematics Education, 3(2), 115–138. https://doi.org/10.1007/s40751-017-0028-x
Blair, R., Kirkman, E., & Maxwell, J. (2018). Statistical abstract of undergraduate programs in the mathematical sciences in the United States: Fall 2015 CBMS Survey. American Mathematical Society. https://doi.org/10.1090/cbmssurvey/2015
Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model (2nd ed.). Lawrence Erlbaum.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061
Brennan, K. & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In A. F. Ball & C. F. Tyson (Eds.), Non satis scire: To know is not enough. Proceedings of the 2012 Annual Meeting of the American Educational Research Association, April 13–17, Vancouver, Canada (Vol. 1, pp. 1–25). American Educational Research Association.
Broatch, J. E., Dietrich, S., & Goelman, D. (2019). Introducing data science techniques by connecting database concepts and dplyr. Journal of Statistics Education, 27(3), 147–153. https://doi.org/10.1080/10691898.2019.1647768
Business Higher Education Forum and PricewaterhouseCoopers. (2017), Investing in America’s data science and analytics talent: The case for action. http://www.bhef.com/sites/default/files/bhef_2017_investing_in_dsa.pdf
Carver, S. M., & Klahr, D. (1986). Assessing children's LOGO debugging skills with a formal model. Journal of Educational Computing Research, 2(4), 487–525. https://doi.org/10.2190/KRD4-YNHH-X283-3P5V
Çetinkaya-Rundel, M., & Tackett, M. (2020). From drab to fab: Teaching visualization via incremental improvements. CHANCE, 33(2), 31–41. https://doi.org/10.1080/09332480.2020.1754074
Chalmers, R. P. (2016). Generating adaptive and non-adaptive test interfaces for multidimensional item response theory applications. Journal of Statistical Software, 71(5), 1–39. http://dx.doi.org/10.18637/jss.v071.i05
Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06
Clements, D. H. (1991). Enhancement of creativity in computer environments. American Educational Research Journal, 28(1), 173–187. https://doi.org/10.2307/1162883
Cobb, G. (2015). Mere renovation is too little too late: We need to rethink our undergraduate curriculum from the ground up. The American Statistician, 69(4), 266–282. https://doi.org/10.1080/00031305.2015.1093029
College Board. (2020). AP Computer Science Principles: Course and exam description. Author. https://apcentral.collegeboard.org/courses/ap-computer-science-principles/course
de Ayala, R. J. (2009). The theory and practice of item response theory. The Guilford Press.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008). Internet, mail, and mixed-mode surveys: The tailored design method. John Wiley.
Dogucu, M., & Çetinkaya-Rundel, M. (2021). Web scraping in the statistics and data science curriculum: Challenges and opportunities. Journal of Statistics Education, 29(sup1), S112–S122. https://doi.org/10.1080/10691898.2020.1787116
Engel, J. (2017). Statistical literacy for active citizenship: A call for data science education. Statistics Education Research Journal, 16(1), 44–49. https://doi.org/10.52041/serj.v16i1.213
Engel, J., Biehler, R., Frischemeier, D [Daniel], Podworny, S., Schiller, A., & Martignon, L. (2019). Zivilstatistik: Konzept einer neuen Perspektive auf Data Literacy und Statistical Literacy. AStA Wirtschafts- und Sozialstatistisches Archiv, 13(3–4), 213–244. https://doi.org/10.1007/s11943-019-00260-w
Erickson, T., Wilkerson, M., Finzer, W., & Reichsman, F. (2019). Data moves. Technology Innovations in Statistics Education, 12(1). https://escholarship.org/uc/item/0mg8m7g6
Fiksel, J., Jager, L. R., Hardin, J. S., & Taub, M. A. (2019). Using gitHub classroom to teach statistics. Journal of Statistics Education, 27(2), 1–10. https://doi.org/10.1080/10691898.2019.1617089
Forbes, S. (2014). The Coming of Age of Statistics Education in New Zealand, and its Influence Internationally. Journal of Statistics Education, 22(2), 2. https://doi.org/10.1080/10691898.2014.11889699
Forbes, S., Chapman, J., Harraway, J., Stirling, D., & Wild, C. (2014). Use of data visualization in the teaching of statistics: A New Zealand perspecitve. Statistics Education Research Journal, 13(2), 187–201. https://doi.org/10.52041/serj.v13i2.290
Futschek, G., & Moschitz, J. (2011). Learning algorithmic thinking with tangible objects eases transition to computer programming. In I. Kalaš & R.T. Mittermeir (Eds.), Situation, evolution, and perspectives. Proceedings of the 5th International Conference on Informatics in Schools, Bratislava, Slovakia, October 26–29, (pp. 155–164). Springer.
Gal, I., Ridgway, J., & Nicholson, J. (2016, December). Exploration of skills andconceptual knowledge needed for understanding statistics about society: A workshop. In J. Engel (Eds.), Promoting understanding of statistics about society. Proceedings of the Roundtable Conference of the International Association of Statistics Education (IASE), Berlin, Germany. https://iase-web.org/documents/papers/rt2016/Gal_Workshop.pdf?1482483777
Gould, R. (2017). Data literacy is statistical literacy. Statistics Education Research Journal, 16(1), 22–25. https://doi.org/10.52041/serj.v16i1.209
Hardin, J. (2018). Dynamic data in the statistics classroom. Technology Innovations in Statistics Education, 11(1). https://escholarship.org/uc/item/13g5g3dm
Hardy, Q. (2012, January 30). BITS; The popularity of statistics. New York Times [online]. http://query.nytimes.com/gst/fullpage.html?res=9B01EED61538F933A05752C0A9649D8B63
Holcomb, J. P., & Spalsbury, A. (2005). Teaching students to use summary statistics and graphics to clean and analyze data. Journal of Statistics Education, 13(3). https://doi.org/10.1080/10691898.2005.11910567
Holdren, J. P., & Lander, E. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. President’s Council of Advisors on Science and Technology. https://eric.ed.gov/?id=ED541511
Horton, N. J., Baumer, B. S., & Wickham, H. (2014). Teaching precursors to data science in introductory and second courses in statistics. arXiv:1401.3269 [stat.CO]. https://arxiv.org/
Horton, N. J., & Hardin, J. S. (2015). Teaching the next generation of statistics students to “think with data”. The American Statistician, 69(4), 259–265. https://doi.org/10.1080/00031305.2015.1094283
IDSSP Curriculum Team (2019). Curriculum Frameworks for Introductory Data Science. http://idssp.org/files/IDSSP_Frameworks_1.0.pdf
Kahlenberg, R. D. (2015). How higher education funding shortchanges community colleges. The Century Foundation. https://vtechworks.lib.vt.edu/handle/10919/83636
Kaplan, D. (2018). Teaching stats for data science. The American Statistician, 72(1), 89–96. https://doi.org/10.1080/00031305.2017.1398107
Marcelino, M. J., Pessoa, T., Vieira, C., Salvador, T., & Mendes, A. J. (2018). Learning computational thinking and scratch at distance. Computers in Human Behavior, 80, 470–477. https://doi.org/10.1016/j.chb.2017.09.025
McNamara, A., & Horton, N. J. (2018). Wrangling categorical data in R. The American Statistician, 72(1), 97–104. https://doi.org/10.1080/00031305.2017.1356375
Melguizo, T., & Wolniak, G. C. (2012). The earnings benefits of majoring in STEM fields among high achieving minority students. Research in Higher Education, 53(4) 383–405. https://doi-org.ezp1.lib.umn.edu/10.1007/s11162-011-9238-z
National Academies of Sciences, Engineering, and Medicine. (2018). Data science for undergraduates: Opportunities and options. The National Academies Press. https://doi.org/10.17226/25104
Nolan, D., & Temple Lang, D. (2010). Computing in the statistics curricula. The American Statistician, 64(2), 97–107. https://doi.org/10.1198/tast.2010.09132
Rash, S. (Director). (1987). Can’t buy me love [Film]. Touchstone Pictures.
Rea, S (2017, October 16). Carnegie Mellon changes statistics department’s name to reflect its eminent position in data science research and education. https://www.cmu.edu/dietrich/news/news-stories/2017/october/stats-name-change.html#:~:text=Carnegie%20Mellon%20University%20has%20changed,Department%20of%20Statistics%20%26%20Data%20Science
Reckase, M. D. (2009). Multidimensional item response theory. Springer. https://doi.org/10.1007/978-0-387-89976-3
Ridgway, J. (2016). Implications of the data revolution for statistics education: The data revolution and statistics education. International Statistical Review, 84(3), 528–549. https://doi.org/10.1111/insr.12110
Ridgway, J., Ridgway, R., & Nicholson, J. (2018, March). Data science for all: A stroll in the foothills. In M. A. Sorto (Ed.), Looking back, looking forward. Proceedings of the 10th International Conference on the Teaching of Statistics (ICOTS10), Kyoto, Japan, July 8–14. International Statistical Institute.
Sáez-López, J. M., Román-González, M., & Vázquez-Cano, E. (2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “Scratch” in five schools. Computers & Education, 97, 129–141. https://doi.org/10.1016/j.compedu.2016.03.003
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monograph No. 17, Part 2). Psychometric Society. http://dx.doi.org.ezp1.lib.umn.edu/10.1007/BF03372160
Schiller, A., & Engel, J. (2016). Civic statistics and the preparation of future secondary school mathematics teachers. In J. Engel (Eds.), Promoting understanding of statistics about society: Proceedings of the Roundtable Conference of the International Association of Statistics Education, Berlin, Germany. https://iase-web.org/documents/papers/rt2016/Schiller.pdf
Thissen, D., Pommerich, M., Billeaud, K., & Williams, V. S. L. (1995). Item response theory for scores on tests including polytomous items with ordered responses. Applied Psychological Measurement, 19, 39-49. https://doi-org.ezp1.lib.umn.edu/10.1177/014662169501900105
U.S. Department of Commerce. (2017). Women in STEM: 2017 Update. ESA Issue Brief #06-17. https://www.commerce.gov/news/fact-sheets/2017/11/women-stem-2017-update
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://www.jstor.org/stable/43867736
Zieffler, A., Park, J., Garfield, J., Delmas, R., & Bjornsdottir, A. (2012). The statistics teaching inventory: A survey on statistics teachers' classroom practices and beliefs. Journal of Statistics Education, 20(1). https://doi.org/10.1080/10691898.2012.11889632
