Reassessing the Accuracy and Use of Readability Formulae

Authors

  • Dahlia Janan Faculty of Languages and Communication Universiti Pendidikan Sultan Idris, Malaysia
  • David Wray Institute of Education University of Warwick, United Kingdom

DOI:

https://doi.org/10.32890/mjli2014.11.7

Keywords:

Readability formula, reading material, measurement

Abstract

Purpose â€“ The purpose of the study is to review readability formulae and offer a critique, based on a comparison of the grading of a variety of texts given by six well-known formulae.
Methodology â€“ A total of 64 texts in English were selected either by or for native English speaking children aged between six and 11 years. Each text was assessed using six commonly used readability formulae via the Words Count website (http://www.wordscount. info/) which provides automated readability indices using FOG, Spache, SMOG, Flesh-Kincaid and Dale-Chall. For the ATOS formula, the Renaissance Learning website was used (http://www. renlearn.com/ar/overview/atos/). Statistical tests were then carried out to check the consistency among the six formulae in terms of their predictions of levels of text diffi culty.
 
Findings â€“ The analysis demonstrated significantly different readability indices for the same text using different formulae. It appeared that some of the formulae (but not all) were consistent in their ranking of texts in order of difficulty but were not consistent in their grading of each text. This finding suggests that readability formulae need to be used carefully to support teachers’ judgements about text difficulty rather than as the sole mechanism for text assessment.
 
Significance â€“ Making decisions about matching texts to learners is something regularly required from teachers at all levels. Making such decisions about text suitability is described as measuring the
 
‘readability’ of texts, and for a long time, this measurement has been treated as unproblematic and achieved using formulae which use such features as vocabulary diffi culty and sentence length. This study suggests that the use of such readability formulae is more problematic than may at first appear. Although the study was carried out with native English speaking children using texts in English, it is argued that the lessons learnt apply equally to Malay speakers reading Malay language texts.

References

Abdullah, I. H., & Hasim, R. S. (2007). Readability analysis of Malaysian short stories in English. E-Bangi Jurnal, 2. Retrieved from http://pkukmweb.ukm.my

Abdullah, K. (2013). A study on the use of electronic stories in Malay language teaching at the lower primary level. Journal of Arts and Education, 1, 12–26. Retrieved from http://www. accessinterjournals.org

Anderson, R. C., Hiebert, E. H., Scott, J. & Wilkinson, L. (1985). Becoming a nation of readers. Washington, DC: National Institute of Education.

Anderson, R., Wilkinson, I., & Mason, M. (1987). Do errors on classroom reading tasks slow growth in reading? Center for the Study of Reading Technical Report No. 404. Champaign: University of Illinois at Urbana-Champaign.

Arifin, Z., Halim, Z., & Bakar, K. (2013). The Readability of Balaghah Textbook of Malaysian Higher Certificate of Religion (Stam): A pilot study. Advances in Natural and Applied Sciences, 7, 192–199. Retrieved from http://www. aensiweb.com

Badarudeen, S., & Sabharwal, S. (2010). Assessing readability of patient education materials: Current role in orthopaedics. Clinical Orthopaedics and Related Research, 468, 2572–80.

Bruce, B., Rubin, A. & Starr, K. (1981). Why Readability Formulas http://mjli.uum.edu.my/ Fail. IEEE Transactions on Professional Communication, 24, 50–52.

Burke, V., & Greenberg, D. (2010) Determining readability: How to select and apply easy-to-use readability formulas to assess the difficulty of adult literacy materials. Adult Basic Education and Literacy Journal, 4, 34–42.

Chall, J. S., & Conard, S. (1991). Should textbooks challenge students? The case for easier or harder books. New York: Teachers College Press.

Chall, J. S., & Dale, E. (1995). Readability revisited: the new Dale-Chall readability formula. Cambridge, Mass: Brookline Books.

Cotton, D., & Gresty, K. (2006). Reflecting on the think-aloud method for evaluating e-learning. British Journal of Educational Technology, 37, 45–54.

Cronin, M., O’Hanlon, S., & M. O’Connor. (2011). Readability level of patient information leaflets for older people. Irish Journal of Medical Science, 180, 139–142.

Dale, E., & Chall, J. S. (1948). A formula for predicting readability. Educational Research Bulletin, 27, 11–28.

Dale, E., & Tyler, R. (1934). A study of the factor influencing the difficulty of reading materials for adults of limited reading ability. In W. H. Dubay (Ed.), Unlocking language: The classic readability studies (pp. 80–107). Costa Mesa, California: Impact Information.

Davison, A., & Kantor, R. N. (1982). On the failure of readability formulas to define readable texts: A case study from adaptations. Reading Research Quarterly, 17, 187–209.

Duffy, T. M. (1985). Readability formulas: What’s the use? In T. M. Duffy & R. M. Waller (Eds.), Designing Usable Texts, 113–143. New York: Academic Press.

Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.

Freda, M. C. (2005). The readability of American academy of pediatrics patient education brochures. Journal of Pediatric Health Care, 19, 151–156.

Fry, E. (1977). Elementary reading instruction. New York: McGraw-Hill.

Gallagher, T. L., Fazio, X., & Gunning, T. (2012). Varying readability of science-based text in elementary readers: Challenges for teachers. Reading Improvement, 49, 93–112.

Gambrell, L. B., Wilson, R. M. & Gantt, W. (1981). Classroom http://mjli.uum.edu.my/ observations of task-attending behaviors of good and poor readers. Journal of Educational Research, 74, 400–404.

Goodman, K. S., & Goodman, Y. (1977). Learning about psycholinguistic processes by analyzing oral reading. Harvard Educational Review, 40, 317–33.

Gunning, R. (1952). The technique of clear writing. New York, NY: McGraw-Hill.

Harris, T. L., & Hodges, R. (1995). The literacy dictionary: The vocabulary of reading and writing. Newark, Del.: International Reading Association.

Harrison, C. (1984). Readability in the United Kingdom. Journal of Reading, 29, 521–529.

Janan, D. (2011). Towards a new model of readability (Unpublished doctoral dissertation). University of Warwick, Coventry.

Kincaid, Fishburne, J., R, Rogers, R., & Chissom, B. (1975). Derivation of new readability formulas automated readability index, fog count and flesch reading ease formula for navy enlisted personnel. Research Branch Report 8–75. Millington, TN: Naval Technical Training, U. S. Naval Air Station, Memphis, TN.

Klare, G. R. (1963). The measurement of readability. Ames: Iowa State University Press.

Kletzien, S. B. (1991). Strategy use by good and poor comprehenders reading expository text of differing levels. Reading Research Quarterly,26, 67–86.

Lee, L. W., & Low, H. M. (2014). The development and application of an online Malay language corpus-based lexical database. Kajian Malaysia, 32 (Supp. 1), 151–166.

Lennon, C., & Burdick, H. (2004). The Lexile Framework as an approach for reading measurement and success. Durham, NC: Metametrics.

McClusky, H. Y. (1934). A quantitative analysis of the difficulty of reading materials. Journal of Educational Research, 28, 276–282.

McKenna, M. C., & M. Picard. (2006). Assessment: Revisiting the role of miscue analysis in effective teaching. The Reading Teacher, 60, 378–380.

McLaughlin, G. H. (1969). SMOG grading - A new readability formula. Journal of Reading, 13, 639–646.

Md. Yunus, K. R. (1982). An assessment of structural variables in Malay: A readability formula. Dissertations http://mjli.uum.edu.my/ from ProQuest. Paper 1263. Retrieved from http://scholarly repository.miami.edu

Michelson, J. (1985). IRA, NCTE take stand on readability formulae. Reading Today, 2(3), 1.

Milone, M. (2008). The development of ATOS: The Renaissance readability formula. Wisconsin Rapids, WI: Renaissance Learning.

Oakland, T., & Lane, H. (2004). Language, reading, and readability formulas: Implications for developing and adapting tests. International Journal of Testing, 4, 239–252.

Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: the nature of constructively responsive reading. Hillsdale, N.J.: Lawrence Erlbaum.

Redish, J., & Selzer, J. (1985). The place of readability formulas in technical communication. Journal of Reading Behavior, 32,46–52.

Renaissance Institute. (2000). The ATOS readability formula for books and how it compares to other formulas. Madison, WI: Renaissance Institute.

Ruddell, R. B., & Unrau, N. (2004). Reading as a meaning-construction process: The reader, the text, and the teacher. In R.B., Ruddell, N. Unrau, & H. Singer (Eds.), Theoretical models and processes of reading (pp. 1462–1521). Newark, Delaware: International Reading Association.

Schriver, K. A. (2000). Readability formulas in the new millennium: What’s the use? ACM Journal of Computer Documentation, 24, 138–140.

Spache, G. (1953). A new readability formula for primary-grade reading materials. Elementary School Journal, 53, 410-413.

Stenner, A. J., Burdick, H., Sanford, E. & Burdick, D. (2006). How accurate are Lexile text measures. Journal of Applied Measurement, 7, 307–322.

Thorndike, E. L. (1921). Word knowledge in the elementary school. In W. H. Dubay, (Ed.), Unlocking language: the classic readability studies (pp. 9–43). Costa Mesa, California: Impact Information.

Vogel, M., & Washburne, C. (1928). An objective method of determining grade placement of children’s reading material. In W. H. Dubay, (Ed.), Unlocking language: the classic readability studies (pp. 58–66). Costa Mesa, California: Impact Information. http://mjli.uum.edu.my/

Yaacob, A., & Pinter, A. (2008). Exploring the effectiveness of using big books in teaching primary English in Malaysian classroom. Malaysian Journal of Learning and Instruction, 5, 1–20.

Downloads

Published

25-12-2014

Issue

Vol. 11 (2014): Malaysian Journal of Learning and Instruction (MJLI) Vol. 11, 2014

Section

Articles

How to Cite

Janan, D., & Wray, D. (2014). Reassessing the Accuracy and Use of Readability Formulae. Malaysian Journal of Learning and Instruction, 11, 127-145. https://doi.org/10.32890/mjli2014.11.7

Research impact

Harvested 2026-09-21
0 citations recorded so far

Counts differ between services because each indexes a different body of literature. None of them is the whole picture.

Identifiers DOI 10.32890/mjli2014.11.7