RESOURCES AND TENSIONS IN STUDENT THINKING ABOUT STATISTICAL DESIGN
DOI:
https://doi.org/10.52041/serj.v22i3.662Keywords:
Statistics education research, Sampling, Generalizability, Power, CausalityAbstract
Reform efforts in statistics education emphasize the need for students to develop statistical thinking. Critical to this goal is a solid understanding of design in the process of collecting data, evaluating evidence, and drawing conclusions. We collected survey responses from over 700 college students at the start of an introductory statistics course to determine how they evaluated the validity of different designs. Despite preferring different designs, students offered a variety of productive arguments supporting their choices. For example, some students viewed intervention as a weakness that disrupted the ability to generalize results, whereas others viewed intervention as critical for identifying causality. Our results highlight that instruction should frame design as the balancing of different priorities: namely causality, generalizability, and power.
References
American Statistical Association Undergraduate Guidelines Workgroup. (2014). Curriculum guidelines for undergraduate programs in statistical science. American Statistical Association. https://www.amstat.org/docs/default-source/amstat-documents/edu-guidelines2014-11-15.pdf
Bennett, K. A. (2015). Using a discussion about scientific controversy to teach central concepts in experimental design. Teaching Statistics, 37(3), 71–77. https://doi.org/10.1111/test.12071
Berland, L. K., & Reiser, B. J. (2009). Making sense of argumentation and explanation. Science Education, 93(1), 26–55. https://doi.org/10.1002/sce.20286
College Board. (2020). AP Statistics course and exam description. https://apcentral.collegeboard.org/media/pdf/ap-statistics-course-and-exam-description.pdf
Crowley, M., Thomas, K., & Haberman, M. (2020, April 5). Ignoring expert opinion, Trump again promotes hydroxychloroquine. New York Times. https://www.nytimes.com/2020/04/05/us/politics/trump-hydroxychloroquine-coronavirus.html
Cummiskey, K., Adams, B., Pleuss, J., Turner, D., Clark, N., & Watts, K. (2020). Causal inference in introductory statistics courses. Journal of Statistics Education, 28(1), 2–8. https://doi.org/10.1080/10691898.2020.1713936
Dasgupta, A. P., Anderson, T. R., & Pelaez, N. (2014). Development and validation of a rubric for diagnosing students’ experimental design knowledge and difficulties. CBE—Life Sciences Education, 13(2), 265–284. https://doi.org/10.1187/cbe.13-09-0192
Deane, T., Nomme, K., Jeffery, E., Pollock, C., & Birol, G. (2014). Development of the biological experimental design concept inventory (BEDCI). CBE—Life Sciences Education, 13(3), 540–551. https://doi.org/10.1187/cbe.13-11-0218
Derry, S. J., Levin, J. R., Osana, H. P., Jones, M. S., & Peterson, M. (2000). Fostering students’ statistical and scientific thinking: Lessons learned from an innovative college course. American Educational Research Journal, 37(3), 747–773. https://doi.org/10.3102/00028312037003747
Easterling, R. (2004). Teaching experimental design. The American Statistician, 58(3), 244–252. https://doi.org/10.1198/000313004X1477
Fauconnier, G., & Turner, M. (2002). The way we think: Conceptual blending and the mind’s hidden complexities. Basic Books.
Findley, K., & Lyford, A. (2019). Investigating students’ reasoning about sampling distributions through a resource perspective. Statistics Education Research Journal, 18(1), 26–45. https://doi.org/10.52041/serj.v18i1.148
Fry, E. B. (2017). Introductory statistics students’ conceptual understanding of study design and conclusions, [Doctoral dissertation, University of Minnesota]. https://iase-web.org/documents/dissertations/17.ElizabethBrondosFry.Dissertation.pdf
GAISE College Report ASA Revision Committee. (2016). Guidelines for Assessment and Instruction in Statistics Education college report 2016. American Statistical Association. http://www.amstat.org/education/gaise
Garfield, J., Le, L., Zieffler, A., & Ben-Zvi, D. (2015). Developing students’ reasoning about samples and sampling variability as a path to expert statistical thinking. Educational Studies in Mathematics, 88, 327–342. https://doi.org/10.1007/s10649-014-9541-7
Gautret, P., Lagier, J.-C., Parola, P., Hoang, V. T., Meddeb, L., Mailhe, M., Doudier, B., Courjon, J., Giordanengo, V., Vieira, V. E., Dupont, H. T., Honoré, S., Colson, P., Chabrière, E., La Scola, B., Rolain, J.-M., Brouqui, P., & Raoult, D. (2020). Hydroxychloroquine and azithromycin as a treatment of COVID-19: Results of an open-label non-randomized clinical trial. International Journal of Antimicrobial Agents, 56(1), 1–6. https://doi.org/10.1016/j.ijantimicag.2020.105949
Hammer, D. (1996). Misconceptions or p-prims: How may alternative perspectives of cognitive structure influence instructional perceptions and intentions. The Journal of the Learning Sciences, 5(2), 97–127. https://doi.org/10.1207/s15327809jls0502_1
Hiebert, S. M. (2007). Teaching simple experimental design to undergraduates: Do your students understand the basics? Advances in Physiology Education, 31(1), 82–92. https://doi.org/10.1152/advan.00033.2006
Imai, K., Keele, L., Tingley, D., & Yamamoto, T. (2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies. American Political Science Review, 105(4), 765–789. https://doi.org/10.1017/S0003055411000414
Kaplan, J. J. (2009). Effect of belief bias on the development of undergraduate students' reasoning about inference. Journal of Statistics Education, 17(1). https://doi.org/10.1080/10691898.2009.11889501
Kupferschmidt, K. (2020). Big studies dim hopes for hydroxychloroquine. Science, 368(6496), 1166–1167. https://doi.org/10.1126/science.368.6496.1166
Next Generation Science Standards Lead States. (2013). Next generation science standards: For states, by states. The National Academies Press.
Saldaña, J. (2021). The coding manual for qualitative researchers. SAGE Publications.
Schield, M. (2018). Confounding and cornfield: Back to the future. In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018). International Statistical Institute/International Association for Statistical Education. https://icots.info/10/proceedings/pdfs/ICOTS10_1C2.pdf?1531364185
Smith, C. L., Maclin, D., Houghton, C., & Hennessey, M. G. (2000). Sixth-grade students’ epistemologies of science: The impact of school science experiences on epistemological development. Cognition and Instruction, 18(3), 349–422. https://doi.org/10.1207/S1532690XCI1803_3
Sotos, A. E. C., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2(2), 98–113. https://doi.org/10.1016/j.edurev.2007.04.001
Tintle, N., Chance, B., Cobb, G., Rossman, A., Roy, S., Swanson, T., & VanderStoep, J. (2013). Challenging the state of the art in post-introductory statistics. Proceedings of the 59th World Statistics Congress of the International Statistical Institute (pp. 295–300). https://core.ac.uk/download/pdf/214190061.pdf
Tintle, N., Chance, B., Cobb, G., Roy, S., Swanson, T., & VanderStoep, J. (2015). Combating anti-statistical thinking using simulation-based methods throughout the undergraduate curriculum. The American Statistician, 69(4), 362–370. https://doi.org/10.1080/00031305.2015.1081619
Wagler, A., & Wagler, R. (2013). Randomizing roaches: Exploring the ‘bugs’ of randomization in experimental design. Teaching Statistics, 36(1), 13–20. https://doi.org/10.1111/test.12029
Watson, J., Fitzallen, N., & Chick, H. (2020). What is the role of statistics in integrating STEM education? In J. Anderson & Y. Li (Eds.), Integrated Approaches to STEM Education. Advances in STEM Education. Springer, Cham. https://doi.org/10.1007/978-3-030-52229-2_6
Wroughton, J. R., McGowan, H. M., Weiss, L. V., & Cope, T. M. (2013). Exploring the role of context in students’ understanding of sampling. Statistics Education Research Journal, 12(2), 32–58. https://doi.org/10.52041/serj.v12i2.303
Zieffler, A., Garfield, J., delMas, R., & Reading, C. (2008). A framework to support research on informal inferential reasoning. Statistics Education Research Journal, 7(2), 40–58. https://doi.org/10.52041/serj.v7i2.469
