CHALLENGES ASSOCIATED WITH MEASURING ATTITUDES USING THE SATS FAMILY OF INSTRUMENTS

Authors

  • DOUGLAS WHITAKER Mount Saint Vincent University
  • ALANA UNFRIED California State University, Monterey Bay
  • MARJORIE BOND Monmouth College

DOI:

https://doi.org/10.52041/serj.v21i1.88

Keywords:

Statistics education research, Attitudes towards statistics, Assessing attitudes, Validity evidence

Abstract

The Survey of Attitudes Toward Statistics (SATS) is a widely used family of instruments for measuring attitude constructs in statistics education. Since the development of the SATS instruments, there has been an evolution in the understanding of validity in the field of educational measurement emphasizing validation as an on-going process. While a 2012 review of statistics education attitude instruments noted that the SATS family had the most validity evidence, two types of challenges to the use of these instruments have emerged: challenges to the interpretations of scale scores and challenges using the SATS instruments in populations other than undergraduate students enrolled in introductory statistics courses. A synthesis of the literature and empirical results are used to document these challenges.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/open-access-files.html

Bandalos, D. L. (2018). Measurement theory and applications for the social sciences. Guilford Press.

Bandura, A. (1977). Social learning theory. Prentice Hall.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.

Batakci, L., Bolon, W., & Bond, M. E. (2018). A framework and survey for measuring instructors’ motivational attitudes toward statistics. In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan. http://iase-web.org/icots/10/proceedings/pdfs/ICOTS10_4J3.pdf

Batanero, C., Estrada, A., Díaz, C., & Fortuny, J. M. (2005). A structural study of future teachers’ attitudes towards statistics. In M. Bosch (Ed.), Proceedings of the Fourth Congress of the European Society for Research in Mathematics Education (pp. 508–517). http://www.mathematik.uni-dortmund.de/~erme/CERME4/CERME4_WG5.pdf

Bateiha, S., Marchionda, H., & Autin, M. (2020). Teaching style and attitudes: A comparison of two collegiate introductory statistics classes. Journal of Statistics Education, 28(2), 154–164. https://doi.org/10.1080/10691898.2020.1765710

Bechrakis, T., Gialamas, V., & Barkatsas, A. N. (2011). Survey of Attitudes Toward Statistics (SATS): An investigation of its construct validity and its factor structure invariance by gender. International Journal of Theoretical Educational Practice, 1(1), 1–15.

Blair, R. M., Kirkman, E. E., & Maxwell, J. W. (2018). Statistical abstract undergraduate programs in the mathematical sciences in the united states: 2015 CBMS survey. American Mathematical Society. http://www.ams.org/cbms-survey/cbms2015-Report.pdf

Bond, M. E. (2008). Survey of Attitudes Toward Statistics: An exploratory look. 2008 JSM Proceedings, (pp. 3758-3760). http://www.statlit.org/pdf/2008BondASA.pdf

Bond, M. E., Perkins, S. N., & Ramirez, C. (2012). Students’ perceptions of statistics: An exploration of attitudes, conceptualizations, and content knowledge of statistics. Statistics Education Research Journal, 11(2), 6–25. https://iase-web.org/documents/SERJ/SERJ11(2)_Bond.pdf?1402525003

Carillo, K., Galy, N., Guthrie, C., & Vanhems, A. (2016).<< J' aime pas les stats!>> Mesure et analyse de l’attitude à l’égard du cours de statistique dans un école de management. Mesure et analyse de l’attitude à l’égard du cours de statistique dans un école de management [“I hate statistics!” — The measure and analysis of attitudes towards statistics in a management school]. Statistique et Enseignement, 7(1), 3–31. http://statistique-et-enseignement.fr/article/view/537

Carlson, K. A., & Winquist, J. R. (2011). Evaluating an active learning approach to teaching introductory statistics: A classroom workbook approach. Journal of Statistics Education, 19(1), 1–23. https://doi.org/10.1080/10691898.2011.11889596

Carmona Márquez, J. (2004). Una revisión de las evidencias de fiabilidad y validez de los cuestionarios de actitudes y ansiedad hacia la estadística. Statistics Education Research Journal, 3(1), 5–28. http://iase-web.org/documents/SERJ/SERJ3(1)_marquez.pdf

Carnell, L. J. (2008). The effect of a student-designed data collection project on attitudes toward statistics. Journal of Statistics Education, 16(1). http://jse.amstat.org/v16n1/carnell.html

Cashin, S. E., & Elmore, P. B. (2005). The survey of attitudes toward statistics acale: A construct validity study. Educational and Psychological Measurement, 65(3), 509–524. https://doi.org/10.1177/0013164404272488

Chance, B., Wong, J., & Tintle, N. (2016). Student performance in curricula centered on simulation-based inference: A preliminary report. Journal of Statistics Education, 24(3), 114–126. https://doi.org/10.1080/10691898.2016.1223529

Chiesi, F., & Primi, C. (2009). Assessing statistics attitudes among college students: Psychometric properties of the Italian version of the Survey of Attitudes Toward Statistics (SATS). Learning and Individual Differences, 19(2), 309–313. https://doi.org/10.1016/j.lindif.2008.10.008

Chiesi, F., & Primi, C. (2010). Cognitive and non-cognitive factors related to students’ statistics achievement. Statistics Education Research Journal, 9(1), 6–26. https://iase-web.org/documents/SERJ/SERJ9(1)_Chiesi_Primi.pdf

Chiesi, F., & Primi, C. (2018). What happens when attitudes toward statisitcs change (increases vs decrease) during the course? In M. A. Sorto, A. White, & L. Guyot (Eds.), Looking back, looking forward. Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan. https://iase-web.org/icots/10/proceedings/pdfs/ICOTS10_7D1.pdf

Cladera, M., Rejón-Guardia, F., Vich-i-Martorell, G. À., & Juaneda, C. (2019). Tourism students’ attitudes toward statistics. Journal of Hospitality, Leisure, Sport & Tourism Education, 24, 202–210. https://doi.org/10.1016/j.jhlste.2019.03.002

Coetzee, S., & Van der Merwe, P. (2010). Industrial psychology students’ attitudes towards statistics. SA Journal of Industrial Psychology, 36(1), 8 pages. https://doi.org/10.4102/sajip.v36i1.843

Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart, and Winston.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://link.springer.com/article/10.1007/BF02310555

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. https://doi.org/10.1037/h0040957

Dauphinee, T. L., Schau, C., & Stevens, J. J. (1997). Survey of attitudes toward statistics: Factor structure and factorial invariance for women and men. Structural Equation Modeling: A Multidisciplinary Journal, 4(2), 129–141. https://doi.org/10.1080/10705519709540066

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th edition). Wiley.

Eccles, J. (1983). Expectancies, values, and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motives: Psychological and sociological approaches (pp. 75–145). W.H. Freeman.

Eccles, J. S. (2014). Expectancy-value theory. In R. Eklund & G. Tenenbaum (Eds.), Encyclopedia of Sport and Exercise Psychology. https://doi.org/10.4135/9781483332222.n110

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. https://www.annualreviews.org/doi/abs/10.1146/annurev.psych.53.100901.135153

Emmioglu Sarikaya, E., Ok, A., Capa Aydin, Y., & Schau, C. (2018). Turkish version of the Survey of Attitudes Toward Statistics: Factorial structure invariance by gender. International Journal of Higher Education, 7(2), 121–127. https://doi.org/10.5430/ijhe.v7n2p121

Estrada, A. (2002). Análisis de las actitudes y conocimientos estadísticos elementales en la formación del profesorado. [Analysis of attitudes and elementary statistical knowledge in training teachers] (Universidad Autónoma de Barcelona). https://www.tdx.cat/handle/10803/4697

Franklin, C., Kader, G., Mewborn, D., Moreno, J., Peck, R., Perry, M., & Scheaffer, R. (2007). Guidelines for Assessment and Instruction in Statistics Education (GAISE) Report: A Pre-K-12 curriculum framework. www.amstat.org/education/gaise

GAISE College Report ASA Revision Committee. (2016). Guidelines for Assessment and Instruction in Statistics Education College Report 2016. http://www.amstat.org/education/gaise

Gal, I., Ginsburg, L., & Schau, C. (1997). Monitoring attitudes and beliefs in statistics education. In I. Gal & J. B. Garfield (Eds.), The assessment challenge in statistics education (pp. 37–51). IOS Press. http://iase-web.org/Books.php?p=book1

García-Martínez, J. A., Fallas-Vargas, M. A., & Romero-Hernández, A. (2015). Las actitudes hacia la estadística del estudiantado de orientación. Revista Electrónica Educare, 19(1), 25–41. https://doi.org/10.15359/ree.19-1.2

Gundlach, E., Richards, K. A. R., Nelson, D., & Levesque-Bristol, C. (2015). A comparison of student attitudes, statistical reasoning, performance, and perceptions for web-augmented traditional, fully online, and flipped sections of a statistical literacy class. Journal of Statistics Education, 23(1). http://jse.amstat.org/v23n1/gundlach.pdf

Hannigan, A., Gill, O., & Leavy, A. M. (2013). An investigation of prospective secondary mathematics teachers’ conceptual knowledge of and attitudes towards statistics. Journal of Mathematics Teacher Education, 16(6), 427–449. https://doi.org/10.1007/s10857-013-9246-3

Harrell-Williams, L. M., Sorto, M. A., Pierce, R. L., Lesser, L. M., & Murphy, T. J. (2014). Validation of scores from a new measure of preservice teachers’ self-efficacy to teach statistics in the middle grades. Journal of Psychoeducational Assessment, 32(1), 40–50. https://doi.org/10.1177/0734282913486256

Harrell-Williams, L. M., Sorto, M. A., Pierce, R., Lesser, L. M., & Murphy, T. J. (2015). Identifying statistical concepts associated with high and low levels of self-efficacy to teach statistics in middle grades. Journal of Statistics Education, 23(1). http://jse.amstat.org/v23n1/harrell-williams.pdf

Harshe, D., & Abraham, D. (2017). A study of attitudes of teaching faculty and postgraduate residents at a tertiary care teaching hospital toward biostatistics. Muller Journal of Medical Sciences and Research, 8(1), 10–14. https://doi.org/10.4103/0975-9727.199362

Hilton, S. C., Schau, C., & Olsen, J. A. (2004). Survey of Attitudes Toward Statistics: Factor structure invariance by gender and by administration time. Structural Equation Modeling: A Multidisciplinary Journal, 11(1), 92–109. https://doi.org/10.1207/S15328007SEM1101_7

Hommik, C., & Luik, P. (2017). Adapting the Survey of Attitudes Towards Statistics (SATS-36) for Estonian secondary school students. Statistics Education Research Journal, 16(1), 228–239. https://doi.org/10.52041/serj.v16i1.229

Hood, M., Creed, P. A., & Neumann, D. L. (2012). Using the expectancy value model of motivation to understand the relationship between student attitudes and achievement in statistics. Statistics Education Research Journal, 11(2), 72–85. https://iase-web.org/documents/SERJ/SERJ11(2)_Hood.pdf

Kerby, A. T., & Wroughton, J. R. (2017). When do students’ attitudes change? Investigating student attitudes at midterm. Statistics Education Research Journal, 16(2), 476–486. https://doi.org/10.52041/serj.v16i2.202

Khavenson, T., Orel, E., & Tryakshina, M. (2012). Adaptation of Survey of Attitudes Towards Statistics (SATS 36) for Russian Sample. Procedia - Social and Behavioral Sciences, 46, 2126–2129. https://doi.org/10.1016/j.sbspro.2012.05.440

Krupa, E. E., Carney, M., & Bostic, J. (2019). Argument-based validation in practice: Examples from mathematics education. Applied Measurement in Education, 32(1), 1–9. https://doi.org/10.1080/08957347.2018.1544139

Lipka, O., & Hess, I. (2016). Attitudes toward statistics studies among students with learning disabilities. Numeracy, 9(2) Article 7. https://doi.org/10.5038/1936-4660.9.2.7

Luh, W. M., Guo, J. H., & Wisenbaker, J. M. (2004). Difference of attitudes toward statistics between cadets and college students. Paper Presented at the 10th International Congress on Mathematical Education, July 4–11, Denmark. http://iase-web.org/documents/papers/icme10/Luh.pdf

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). American Council on Education.

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741

Millar, A. M., & Schau, C. (2010). Assessing students’ attitudes: The good, the bad, and the ugly. 2010 JSM Proceedings (pp. 1133–1143). http://www.statlit.org/pdf/2010MillarSchauASA.pdf

Millar, A. M., & White, B. J. G. (2014). How do attitudes change from one stats course to the next? In K. Makar & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9), Flagstaff, Arizona, July 13–18. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_1F2_MILLAR.pdf

Millar, A. M., White, B. J., & Romo, R. (2013). Comparing apples with apples: Assessing student attitudes in the presence of regression to the mean. 2013 JSM Proceedings. Presented at the Joint Statistics Meetings, San Francisco, CA. http://ec.msvu.ca:8080/xmlui/handle/10587/1284

Moore, C. M. (1987). Group techniques for idea building. SAGE Publications.

Nasser, F. M. (2004). Structural model of the effects of cognitive and affective factors on the achievement of Arabic-speaking pre-service teachers in introductory statistics. Journal of Statistics Education, 12(1). http://ww2.amstat.org/publications/jse/v12n1/nasser.html

Nolan, M. M., Beran, T., & Hecker, K. G. (2012). Surveys assessing students’ attitudes toward statistics: A systematic review of validity and reliability. Statistics Education Research Journal, 11(2), 103–123. https://doi.org/10.52041/serj.v11i2.333

Patton, M. Q. (2002). Qualitative research and evaluation methods (3 ed). SAGE Publications.

Posner, M. A. (2011). The impact of a proficiency-based assessment and reassessment of learning outcomes system on student achievement and attitudes. Statistics Education Research Journal, 10(1), 3–14. https://doi.org/10.52041/serj.v10i1.352

Posner, M. A. (2014). A fallacy in student attitude research: The impact of the first class. In K. Makar & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9), Flagstaff, Arizona, July 13–18. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_1F3_POSNER.pdf

Ramirez, C., & Bond, M. (2014). Comparing attitudes toward statistics among students enrolled in project-based and hybrid statistics courses. In K. Makar & R. Gould (Eds.), Sustainability in Statistics Education. Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9), Flagstaff, Arizona, July 13–18. https://iase-web.org/icots/9/proceedings/pdfs/ICOTS9_1F4_RAMIREZ.pdf?1405041565

Ratanaolarn, T. (2016). The development of a structural equation model of graduate students’ statistics achievement. The Journal of Behavioral Science, 11(2), 153–168. https://so06.tci-thaijo.org/index.php/IJBS/article/view/63288

Rosli, M. K., & Maat, S. M. (2017). Attitude towards statistics and performance among post-graduate students. AIP Conference Proceedings, 1847(1) 030004. https://doi.org/10.1063/1.4983881

Schau, C. (1992). Survey of Attitudes Toward Statistics (SATS-28). http://evaluationandstatistics.com/

Schau, C. (2003a). Students’ attitudes: The “other” important outcome in statistics education. In proceedings of the 2003 Joint Statisitcal Meetings, 3 (pp. 3673–3683). http://www.statlit.org/pdf/2003SchauASA.pdf

Schau, C. (2003b). Survey of Attitudes Toward Statistics (SATS-36). http://evaluationandstatistics.com/

Schau, C. (2005a). Scoring the SATS-28©. https://irp-cdn.multiscreensite.com/281322c3/files/uploaded/SATS28Scoring.pdf

Schau, C. (2005b). Scoring the SATS-36©. https://irp-cdn.multiscreensite.com/281322c3/files/uploaded/Final36scoring.pdf

Schau, C. (2008). Common issues in SATS© research. 2008 JSM Proceedings (pp. 3761–3763). http://www.statlit.org/pdf/2008SchauASA.pdf

Please see the article for the complete list of references.

Downloads

Published

2022-03-02

Issue

Section

Regular Articles