CAN WE DISTINGUISH STATISTICAL LITERACY AND STATISTICAL REASONING?
DOI:
https://doi.org/10.52041/serj.v24i1.587Keywords:
Statistics education research; Assessment; Statistical Literacy; Statistical Reasoning; SubscoresAbstract
One of the most important goals in a statistics class is to develop students who are statistically literate and can reason with statistical concepts. The REALI instrument was designed to concurrently assess statistical literacy and reasoning in introductory statistics students. This paper reports a measurement analysis of the statistical literacy and reasoning subscores from the REALI assessment and the extent to which they are reliable and distinct. Investigation of these subscores is used clarify the relationship between the constructs of statistical literacy and statistical reasoning and to what extent they overlap. The results of this analysis, under a Multidimensional Item Response Theory framework, show that the statistical literacy and reasoning subscores provide no added value over a single general statistical knowledge score. This indicates the two constructs might be indistinguishable from one another.
References
Adams, B., Baller, D., Jonas, B., Joseph, A. C., & Cummiskey, K. (2021). Computational skills for multivariable thinking in introductory statistics. Journal of Statistics and Data Science Education, 29(1), 123–131. https://doi.org/10.1080/10691898.2020.1852139
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. https://www.apa.org/science/programs/testing/standards
Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, Handbook I: Cognitive domain. David McKay.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. Bollen & J. Long (Eds.), Testing structural equation models (pp. 136–162). SAGE Publications. https://doi.org/10.1177%2F0049124192021002005
Budgett, S., & Pfannkuch, M. (2007). Assessing students’ statistical literacy. In P. Bidgood, N. Hunt & F. Jolliffe (Eds.), Assessment methods in statistical education: An international perspective (pp. 103–121). John Wiley & Sons. https://doi.org/10.1002/9780470710470.ch9
Callingham, R., & Watson, J. M. (2017). The development of statistical literacy at school. Statistics Education Research Journal, 16(1), 181–201. https://doi.org/10.52041/serj.v16i1.223
Chalmers, R., P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), Article 6. https://doi.org/10.18637/jss.v048.i06
Chance, B. L. (2002). Components of statistical thinking and implications for instruction and assessment. Journal of Statistics Education, 10(3), Article 3. https://doi.org/10.1080/10691898.2002.11910677
Chervaney, N., Collier, R. Fienberg, S., Johnson, P, & Neter, J. (1977). A framework for the development of measurement instruments for evaluating the introductory statistics course, The American Statistician, 31(1), 17–23. https://doi.org/10.1080/00031305.1977.10479186
Chervaney, N., Benson, P. G., & Iyer, R. (1980), The planning stage in statistical reasoning. The American Statistician, 34(4), 222–226. https://doi.org/10.2307/2684064
Cobb, G. (1992). Teaching statistics. In L. A. Steen (Ed.), Heeding the call for change: Suggestions for curricular action (pp. 3–43). The Mathematical Association of America.
Cummiskey, K., Adams, B., Pleuss, J., Turner, D., Clark, N., & Watts, K. (2020). Causal inference in introductory statistics courses. Journal of Statistics Education, 28(1), 2–8. https://doi.org/10.1080/10691898.2020.1713936
De Ayala, R. J. (2009). The theory and practice of item response theory. Guilford Press.
delMas, R. C. (2002). Statistical literacy, reasoning and learning: A commentary. Journal of Statistics Education, 10(3), Article 1. https://doi.org/10.1080/10691898.2002.11910679
delMas, R. (2004). A comparison of mathematical and statistical reasoning. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning, and thinking (pp. 79–95). Kluwer Academic Publishers. https://doi.org/10.1007/1-4020-2278-6_4
Engel, J. (2017). Statistical literacy for active citizenship: A call for data science education. Statistics Education Research Journal, 16(1), 44-49. https://doi.org/10.52041/serj.v16i1.213
GAISE College Report ASA Revision Committee. (2016). Guidelines for assessment and instruction in statistics education report 2016. https://www.amstat.org/education/guidelines-for-assessment-and-instruction-in-statistics-education-(gaise)-reports
Gal, I. (2002). Adults’ statistical literacy: Meanings, components, responsibilities. International Statistical Review, 70(1), 1–25. https://doi.org/10.2307/1403713
Gal, I. (2004). Statistical literacy. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 47–78). Springer. https://doi.org/10.1007/1-4020-2278-6_3
Garfield, J. (2002). The challenge of developing statistical reasoning. Journal of Statistics Education, 10(3), Article 3. https://doi.org/10.1080/10691898.2002.11910676
Garfield, J., & Ben?Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and learning statistics. International Statistical Review, 75(3), 372–396. http://dx.doi.org/10.1111/j.1751-5823.2007.00029.x
Garfield, J., & Ben-Zvi, D. (2008). Developing students’ statistical reasoning: Connecting research and teaching practice. Springer. https://doi.org/10.1007/978-1-4020-8383-9
Garfield, J., & Chance, B. (2000). Assessment in statistics education: Issues and challenges. Mathematical Thinking and Learning, 2(1–2), 99–125. https://doi.org/10.1207/S15327833MTL0202_5
Garfield, J., delMas, R., & Zieffler, A. (2012). Developing statistical modelers and thinkers in an introductory, tertiary-level statistics course. ZDM Mathematics Education, 44(7), 883–898. https://doi.org/10.1007/s11858-012-0447-5
Gibbons, R. D., & Hedeker, D. R. (1992). Full-information item bi-factor analysis. Psychometrika, 57(3), 423-436. https://doi.org/10.1007/BF02295430
Gould, R. (2017). Data literacy is statistical literacy. Statistics Education Research Journal, 16(1), 22–25. https://doi.org/10.52041/serj.v16i1.209
Gould, R. (2021). Toward data?scientific thinking. Teaching Statistics, 43, S11–S22. https://doi.org/10.1111/test.12267
Hawkins, A., Jolliffe, F., & Glickman, L. (1992), Teaching statistical concepts. Longman Publishers. https://doi.org/10.4324/9781315845517
Haberman, S. J. (2008). When can subscores have value? Journal of Educational and Behavioral Statistics, 33(2), 204–229. https://doi.org/10.3102/1076998607302636
Hu, L.T., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.
Hudiburgh, L. M., & Garbinsky, D. (2020). Data visualization: Bringing data to life in an introductory statistics course. Journal of Statistics Education, 28(3), 262–279. https://doi.org/10.1080/10691898.2020.1796399
Jones, G. A., Langrall, C. W., Mooney, E. S., & Thornton, C. A. (2004). Models of development in statistical reasoning. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 97–117). https://doi.org/10.1007/1-4020-2278-6_5
Jones, G. A., Thornton, C. A., Langrall, C. W., Mooney, E. S., Perry, B., & Putt, I. J. (2000). A framework for characterizing children’s statistical thinking. Mathematics Thinking and Learning, 2(4), 269–307. https://doi.org/10.1207/S15327833MTL0204_3
Kaplan, J. J., & Thorpe, J. (2010). Post secondary and adult statistical literacy: Assessing beyond the classroom. Paper presented at the Data and context in statistics education: Towards an evidence-based society. In C. Reading (Ed.), Proceedings of the Eighth International Conference on Teaching Statistics (ICOTS8), Ljubljana, Slovenia. International Statistical Institute. https://www.stat.auckland.ac.nz/~iase/publications/icots8/ICOTS8_5E3_KAPLAN.pdf
Lovett, M. (2001). A collaborative convergence on studying reasoning processes: A case study in statistics. In S. Carver & D. Klahr (Eds.), Cognition and instruction: Twenty-five years of progress. Lawrence Erlbaum.
MacGillivray, H. (2021). Statistics and data science must speak together. Teaching Statistics, 43, S5–S10. https://doi.org/10.1111/test.12281
Mallows, C. (1998). The zeroth problem. The American Statistician, 52(1), 1–9. https://doi.org/10.2307/2685557
McKinley, R. L., & Reckase, M. D. (1983). An extension of the two-parameter logistic model to the multidimensional latent space. American College Testing Program. https://apps.dtic.mil/sti/pdfs/ADA137769.pdf
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–103). Macmillan Publishing; American Council on Education.
Mooney, E. S. (2002). A framework for characterizing middle school students’ statistical thinking. Mathematical Thinking and Learning, 4(1), 23–63. https://doi.org/10.1207/S15327833MTL0401_2
Moore, D. S. (1990). Uncertainty. In L.A. Steen (Ed.), On the shoulders of giants: New approaches to numeracy (pp. 95–138). National Academy Press.
Nisbett, R. (1993), Rules for Reasoning. Lawrence Erlbaum.
Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64. https://doi.org/10.1177%2F01466216000241003
Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298. https://doi.org/10.1177%2F0146621603027004004
Prodromou, T., & Dunne, T. (2017). Statistical literacy in data revolution era: Building blocks and instructional dilemmas. Statistics Education Research Journal, 16(1), 38–43. https://doi.org/10.52041/serj.v16i1.212
R Development Core Team (2016). R: A language and environment for statistical computing [Computer software.] R Foundation for Statistical Computing. http://www.R-project.org/
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9(4), 401–412. https://doi.org/10.1177%2F014662168500900409
Reckase, M. D. (2009). Multidimensional item response theory (pp. 79–112). Springer. https://doi.org/10.1007/978-0-387-89976-3
Ridgway, J., Nicholson, J., & McCusker, S. (2013). Open data and the semantic web require a rethink on statistics teaching. Technology Innovations in Statistics Education, 7(2). https://doi.org/10.5070/T572013907
Rumsey, D. J. (2002). Statistical literacy as a goal for introductory statistics courses. Journal of Statistics Education, 10(3), 6–13. https://doi.org/10.1080/10691898.2002.11910678
Sabbag, A., Garfield, J., & Zieffler, A. (2018). Assessing statistical literacy and statistical reasoning: The REALI instrument. Statistics Education Research Journal, 17(2), 141–160. https://doi.org/10.52041/serj.v17i2.163
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136
Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52(3), 333–343. https://doi.org/10.1007/BF02294360
Sedlmeier, P. (1999), Improving statistical reasoning: Theoretical models and practical implication. Lawrence Erlbaum. https://doi.org/10.1002/acp.749
Sharma, S., Doyle, P., Shandil, V., & Talakia'atu, S. (2011). Developing statistical literacy with Year 9 students. Set: Research Information for Teachers, 1, 43–50. https://doi.org/10.18296/set.0398
Sinharay, S., Puhan, G., Haberman, S. J., & Hambleton, R. K. (2018). Subscores: When to communicate them, what are their alternatives, and some recommendations. In D. Zapata-Rivera (Ed.), Score reporting: Research and applications (pp. 35–49). https://www.taylorfrancis.com/chapters/oa-edit/10.4324/9781351136501-5/subscores-sandip-sinharay-gautam-puhan-shelby-haberman-ronald-hambleton
Snee, R. D. (1990). Statistical thinking and its contribution to total quality. The American Statistician, 44(2), 116–121. https://doi.org/10.2307/2684144
Sugiura N. (1978). Further analysis of the data by Akaike’s information criterion and the finite corrections. Communications in Statistics: Theory and Methods, 7(1), 13–26. https://doi.org/10.1080/03610927808827599
Sylwester, D. (February,1993). Statistical thinking. AMSTAT News.
Tate, R. L. (2004). Implications of multidimensionality for total score and subscore performance. Applied Measurement in Education, 17, 89–112. https://doi.org/10.1207/s15324818ame1702_1
Tintle, N. L., Topliff, K., VanderStoep, J., Holmes, V. L., & Swanson, T. (2012). Retention of statistical concepts in a preliminary randomization-based introductory statistics curriculum. Statistics Education Research Journal, 11(1), 21–40. https://doi.org/10.52041/serj.v11i1.340
Utts, J. (2003). What educated citizens should know about statistics and probability. The American Statistician, 57(2), 74–79. https://doi.org/10.1198/0003130031630
Watson, J., & Callingham, R. (2003). Statistical literacy: A complex hierarchical construct. Statistics Education Research Journal, 2(2), 3–46. https://doi.org/10.52041/serj.v2i2.553
Watson, J. M., Collis, K. F., Callingham, R. A., & Moritz, J. B. (1995). A model for assessing higher order thinking in statistics. Educational Research and Evaluation, 1(3), 247–275. https://doi.org/10.1080/1380361950010303
Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67(3), 223–248. https://doi.org/10.1111/j.1751-5823.1999.tb00442.x
Zimowski, M., Muraki, E., Mislevy, R. J., & Bock, R. D. (2003). BILOG-MG (Version 3) [Computer Program]. Scientific Software.
