A Comparative Study of Cognitive Processing in Oral Mathematics Tests Among Malaysian Orang Asli Pupils
DOI:
https://doi.org/10.32890/mjli2025.22.2.6Keywords:
Oral mathematics test, computation, academic language, native language, knowing, applyingAbstract
Purpose – This study investigates the mathematics performance of Orang Asli pupils across two cognitive domains—Knowing and Applying—using three oral mathematics tests developed in different languages: the academic language (Bahasa Melayu) (OBM), the academic native language (Bahasa Semai) (OSL), and the tribal native language (Bahasa Temiar) (OTL).
Methodology – Adopting a quantitative approach, the study involved 225 Year 4 Orang Asli pupils who were assessed using a 20-item mathematics computation testlet. The analysis focused on pupils’ performance across the Knowing and Applying domains in the three oral test formats (OBM, OSL, and OTL). One-way Analysis of Variance (ANOVA) was used to compare performance differences across the cognitive domains and test formats.
Findings – In the Knowing domain, pupils performed best in the OSL test, followed by OBM and OTL. Conversely, in the Applying domain, pupils excelled in the OTL test, followed by OSL and OBM. ANOVA results indicated significant differences between OSL and OTL, as well as OBM and OTL in the Knowing domain. However, no significant difference was found between OSL and OBM. For the Applying domain, a significant difference was observed only between OTL and OBM, with no significant differences between OTL and OSL, or between OSL and OBM.
Significance – The findings highlight the potential of oral mathematics tests in native languages—particularly Bahasa Semai—as a culturally responsive and equitable assessment method. This approach enables Orang Asli pupils to better demonstrate their mathematical proficiency helping to address linguistic barriers in mathematics assessment.
References
Abd Jalil, A. H., Abdullah, A. H., & Hamzah, M. H. (2023). Does language matter in the mathematics classroom for Orang Asli students in Malaysia? Malaysian Journal of Social Sciences and Humanities, 8(2), e002061. https://doi.org/10.47405/mjssh.v8i2.2061
Abdi, H., & Williams, L. J. (2010). Tukey’s honestly significant difference (HSD) test. In N. J. Salkind (Ed.), Encyclopedia of research design, 1, 1-5. Sage Publication.
Abdullah, A. H. (2022). A systematic review of what Malaysia can learn to improve Orang Asli students’ mathematics learning from other countries. Sustainability, 14(20), 13201. https://doi.org/10.3390/su142013201
Abedi, J. (2011). Language issues in item development. In T. M. Haladyna & S. M. Downing (Eds.), Handbook of test development (pp. 391-412). Routledge.
Barcelos, T. S., Muñoz-Soto, R., Villarroel, R., Merino, E., & Silveira, I. F. (2018). Mathematics learning through computational thinking activities: A systematic literature review. Journal of Universal Computer Science, 24(7), 815-845.
Bichi, A. A. (2016). Classical test theory: An introduction to linear modeling approach to test and item analysis. International Journal for Social Studies, 2(9), 27-33.
Blackley, C., Redmond, P., & Peel, K. (2021). Teacher decision-making in the classroom: The influence of cognitive load and teacher affect. Journal of Education for Teaching, 47(4), 548-561. https://doi.org/10.1080/02607476.2021.1902748
Bolt, S. E., & Thurlow, M. L. (2007). Item-level effects of the read-aloud accommodation for students with reading disabilities. Assessment for Effective Intervention, 33(1), 15-28. https://doi.org/10.1177/15345084070330010301
Boonen, A. J., de Koning, B. B., Jolles, J., & Van der Schoot, M. (2016). Word problem solving in contemporary math education: A plea for reading comprehension skills training. Frontiers in Psychology, 7, 155518. https://doi.org/10.3389/fpsyg.2016.00191
Boopathiraj, C., & Chellamani, K. (2013). Analysis of test items on difficulty level and discrimination index in the test for research in education. International Journal of Social Science & Interdisciplinary Research, 2(2), 189-193.
Braeuning, D., Hornung, C., Hoffmann, D., Lambert, K., Ugen, S., Fischbach, A., Schiltz, C., Hübner, N., Nagengast, B., & Moeller, K. (2021). Long-term relevance and interrelation of symbolic and non-symbolic abilities in mathematical-numerical development: Evidence from large-scale assessment data. Cognitive Development, 58, 101008. https://doi.org/10.1016/j.cogdev.2021. 101008
Brezovszky, B., McMullen, J., Veermans, K., Hannula-Sormunen, M. M., Rodríguez-Aflecht, G., Pongsakdi, N., Laakkonen, E., & Lehtinen, E. (2019). Effects of a mathematics game-based learning environment on primary school students' adaptive number knowledge. Computers & Education, 128, 63-74. https://doi.org/10.1016/j.compedu.2018.09.011
Cawthon, S. W., Kaye, A. D., Lockhart, L. L., & Beretvas, S. N. (2012). Effects of linguistic complexity and accommodations on estimates of ability for students with learning disabilities. Journal of School Psychology, 50(3), 293-316. https://doi.org/10.1016/j.jsp.2012.01.002
Cho, E., Fuchs, L. S., Seethaler, P. M., Fuchs, D., & Compton, D. L. (2020). Dynamic assessment for identifying Spanish-speaking English learners’ risk for mathematics disabilities: Does language of administration matter? Journal of Learning Disabilities, 53(5), 380-398. https://doi.org/10. 1177%2F0022219419898887
Chow, J. C., & Ekholm, E. (2019). Language domains differentially predict mathematics performance in young children. Early Childhood Research Quarterly, 46, 179-186. https://doi.org/10. 1016/j.ecresq.2018.02.011
Cohen, D. J., Blanc‐Goldhammer, D., & Quinlan, P. T. (2018). A mathematical model of how people solve most variants of the number‐line task. Cognitive Science, 42(8), 2621-2647. https://doi.org/10.1111/cogs.12698
Daroczy, G., Wolska, M., Meurers, W. D., & Nuerk, H. C. (2015). Word problems: A review of linguistic and numerical factors contributing to their difficulty. Frontiers in Psychology, 6, 348. https://doi.org/10.3389/fpsyg.2015.00348
Ellerton, P. (2022). On critical thinking and content knowledge: A critique of the assumptions of cognitive load theory. Thinking Skills and Creativity, 43, Article 100975. https://doi.org/10. 1016/j.tsc.2021.100975
Ercikan, K., & Por, H. H. (2020). Comparability in multilingual and multicultural assessment contexts. In A. I. Berman, E. H. Haertel, & J. W. Pellegrino (Eds.), Comparability of large-scale educational assessments: Issues and recommendations (pp. 205-225). National Academy of Education.
Francis, D. I. C. (2024). Mathematics teaching and learning. In P. A. Schutz & K. R. Muis (Eds.), Handbook of educational psychology (pp. 480-508). Routledge.
Fuchs, L. S., Gilbert, J. K., Fuchs, D., Seethaler, P. M., & Martin, B. N. (2018). Text comprehension and oral language as predictors of word-problem solving: Insights into word-problem solving as a form of text comprehension. Scientific Studies of Reading, 22(2), 152-166. https://doi.org/10.1080/10888438.2017.1398259
Grønmo, L. S., Lindquist, M., Arora, A., & Mullis, I. V. (2015). TIMSS 2015 mathematics framework. https://timssandpirls.bc.edu/timss2015/downloads/t15_fw_chap1.pdf
Ismail, Z., Ching, T. Y., & Muda, N. A. (2020). Numeracy competency of year 5 aboriginal students using written and oral tests. The Mathematics Enthusiast, 17(1), 32-62. https://doi.org/10.54870/1551-3440.1479
Jansen, T., & Möller, J. (2022). Teacher judgments in school exams: Influences of students' lower-order-thinking skills on the assessment of students’ higher-order-thinking skills. Teaching and Teacher Education, 111, 103616. https://doi.org/10.1016/j.tate.2021.103616
Kline, P. (2015). A handbook of test construction (psychology revivals): Introduction to psychometric design. Routledge.
Kosasih, A., Supriyadi, T., Firmansyah, M. I., & Rahminawati, N. (2022). Higher-order thinking skills in primary school. Journal of Ethnic and Cultural Studies, 9(1), 56-76.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2
Kūkea Shultz, P., & Englert, K. (2021, July). Cultural validity as foundational to assessment development: An indigenous example. Frontiers in Education, 6, 1-11. https://doi.org/10.3389/feduc.2021.701973
Livingston, S. A., & Kim, S. (2010). Random‐groups equating with samples of 50 to 400 test takers. Journal of Educational Measurement, 47(2), 175-185. https://doi.org/10.1111/j.1745-3984.2010.00107.x
Matang, R. A., & Owens, K. (2014). The role of indigenous traditional counting systems in children's development of numerical cognition: Results from a study in Papua New Guinea. Mathematics Education Research Journal, 26, 531-553. https://doi.org/10.1007/s13394-013-0115-2
Misrom, N. B., Muhammad, A., Abdullah, A., Osman, S., Hamzah, M., & Fauzan, A. (2020). Enhancing students’ higher-order thinking skills (HOTS) through an inductive reasoning strategy using Geogebra. International Journal of Emerging Technologies in Learning, 15(3), 156-179.
Mullis, I. V., & Martin, M. O. (2017). TIMSS 2019 assessment frameworks. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College and International Association for the Evaluation of Educational Achievement (IEA).
Mullis, I.V.S., Martin, M.O., Hooper, M. (2017). Measuring changing educational contexts in a changing world: Evolution of the TIMSS and PIRLS questionnaires. In M. Rosén, K. Yang Hansen, & U. Wolff (Eds.), Cognitive abilities and educational outcomes. Springer. https://doi.org/10.1007/978-3-319-43473-5_11
Nordby, S. K., Bjerke, A. H., & Mifsud, L. (2022). Computational thinking in the primary mathematics classroom: A systematic review. Digital Experiences in Mathematics Education, 8(1), 27-49. https://doi.org/10.1007/s40751-022-00102-5
Nugraheni, L. P., & Marsigit, M. (2021). Realistic mathematics education: An approach to improve problem solving ability in primary school. Journal of Education and Learning, 15(4), 511-518. https://doi.org/10.11591/edulearn.v15i4.19354
Peng, P., Lin, X., Ünal, Z. E., Lee, K., Namkung, J., Chow, J., & Sales, A. (2020). Examining the mutual relations between language and mathematics: A meta-analysis. Psychological Bulletin, 146(7), 595-634. https://psycnet.apa.org/doi/10.1037/bul0000231
Penner, K. (2016). Oral administration as an effective accommodation for students with ADHD. Literacy Information and Computer Education Journal, 8(2), 2570-2577.
Philpot, R., Lindquist, M., Mullis, I. V., & Aldrich, C. E. (2021). TIMSS 2023 mathematics framework. In I. V. S. Mullis, M. O. Martin, & M. von Davier (Eds.), TIMSS 2023 assessment frameworks (pp. 5-18). TIMSS & PIRLS International Study Centre, Lynch School of Education and Human Development, Boston College and International Association for the Evaluation of Educational Achievement (IEA).
Pongsakdi, N., Kajamies, A., Veermans, K., Lertola, K., Vauras, M., & Lehtinen, E. (2020). What makes mathematical word problem solving challenging? Exploring the roles of word problem characteristics, text comprehension, and arithmetic skills. ZDM, 52, 33-44. https://doi.org/10.1007/s11858-019-01118-9
Radmehr, F., & Drake, M. (2018). An assessment-based model for exploring the solving of mathematical problems: Utilizing revised Bloom’s taxonomy and facets of metacognition. Studies in Educational Evaluation, 59, 41-51. https://doi.org/10.1016/j.stueduc. 2018.02.004
Robertson, S. A., & Graven, M. (2020). Language as an including or excluding factor in mathematics teaching and learning. Mathematics Education Research Journal, 32(1), 77-101. https://doi.org/10.1007/s13394-019-00302-0
Rogers, C. M., Thurlow, M. L., Lazarus, S. S., & Liu, K. K. (2019). A summary of the research on effects of test accommodations: 2015-2016. (NCEO Report 412). National Center on Educational Outcomes.
Roschmann, S., Witmer, S. E., & Volker, M. A. (2021). Examining provision and sufficiency of testing accommodations for English learners. International Journal of Testing, 21(1), 32-55. https://doi.org/10.1080/15305058.2021.1884872
Sani, N., & Idris, A. R. (2018). Implementation of Linus programme based on the model of Van Meter and Van Horn. The Malaysian Online Journal of Educational Sciences, 1(2), 25-36.
Shanmugam, S. K. S., Veloo, A., & Md-Ali, R. (2021). Culturally responsive assessment: Assessing mathematics performance of indigenous pupils in Malaysia using trilingual test. Diaspora, Indigenous, and Minority Education, 15(2), 113-136. https://doi.org/10.1080/15595692.2020. 1846515
Shanmugam, S. K. S., Veloo, A., & Bin Jusoh, Y. A. (2023). The challenges of virtual learning: An exploratory study among Orang Asli pupils from the mathematics teachers’ perspectives. Diaspora, Indigenous, and Minority Education, 1-19. https://doi.org/10.1080/ 15595692.2023.2212823
Shanmugam, S., Veloo, A., & Yusoff, Y. A. B. J. (2024). Examining utility of oral-administered test accommodation in assessing aboriginal pupils’ mathematics performance using score comparability. International Journal of Science and Mathematics Education, 1-24. https://doi.org/10.1007/s10763-024-10451-9
Spaull, N., & Kotze, J. (2015). Starting behind and staying behind in South Africa: The case of insurmountable learning deficits in mathematics. International Journal of Educational Development, 41, 13-24. https://doi.org/10.1016/j.ijedudev.2015.01.002
Spiel, C., Evans, S. W., & Harrison, J. R. (2019). Does reading standardized tests aloud meet the scientific definition of an accommodation? Journal of Applied School Psychology, 35(4), 380-399. https://doi.org/10.1080/15377903.2019.1601145
Susanto, A., Nengsih, R., Akhirina, T. Y., Nulhakim, A. L., & Awaludin, A. A. R. (2019, March). How to improve students mathematics problem solving by implementing Indonesian realistics mathematics education (IRME) approach. In Journal of Physics: Conference Series (Vol. 1175, No. 1, p. 012046). IOP Publishing.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer.
Sweller, J. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68(1), 1-16. https://doi.org/10.1007/s11423-019-09701-3
Tai, K. W. (2022). Translanguaging as inclusive pedagogical practices in English-medium instruction science and mathematics classrooms for linguistically and culturally diverse students. Research in Science Education, 52(3), 975-1012. https://doi.org/10.1007/s11165-021-10018-6
Tambunan, H. (2019). The effectiveness of the problem-solving strategy and the scientific approach to students' mathematical capabilities in high order thinking skills. International Electronic Journal of Mathematics Education, 14(2), 293-302.
Tan, K. J., Ismail, Z., & Abidin, M. (2018). A comparative analysis on cognitive domain for the Malaysian primary four textbook series. EURASIA Journal of Mathematics, Science and Technology Education, 14(4), 1273-1286. https://doi.org/10.29333/ejmste/82625
Van den Heuvel-Panhuizen, M., & Drijvers, P. (2020). Realistic mathematics education. In S. Lerman (Ed.), Encyclopedia of mathematics education (pp. 713-717). Springer.
Veloo, A., Shanmugam, S. K. S., Md-Ali, R., Yusoff, Y. A. J., & Awang-Hashim, R. (2021). Grade five indigenous (Orang Asli) pupil's achievement in bilingual versions of mathematics test. Journal of Language and Linguistic Studies, 17(4), 1863-1872.
Wijns, N., Verschaffel, L., De Smedt, B., & Torbeyns, J. (2021). Associations between repeating patterning, growing patterning, and numerical ability: A longitudinal panel study in 4‐to 6-year-olds. Child Development, 92(4), 1354-1368. https://doi.org/10.1111/cdev.13490
Yushau, B., & Omar, M. H. (2015). Mathematics performance and its relation to English language proficiency level of bilingual Arab university students. Indian Journal of Science and Technology, 8(13), 1-15.
Zapalska, A. M., McCarty, M. D., Young-McLear, K., & White, J. (2018). Design of assignments using the 21st century Bloom’s revised taxonomy model for development of critical thinking skills. Problems and Perspectives in Management, 16(2), 291-305.
Zhang, D., & Rivera, F. D. (2021). Predetermined accommodations with a standardized testing protocol: Examining two accommodation supports for developing fraction thinking in students with mathematical difficulties. The Journal of Mathematical Behavior, 62, 100861. https://doi.org/10.1016/j.jmathb.2021.100861
Published
Issue
Section
How to Cite
Research impact
Harvested 2026-09-21Counts differ between services because each indexes a different body of literature. None of them is the whole picture.
