Examining Oral Test Accommodation in Assessing Malaysian Orang Asli Pupils’ Mathematics Performance for Computation and Word Problem Testlets

Arsaythamby Veloo; S. Kanageswari Suppiah Shanmugam; Suheysen Revindran

doi:10.32890/mjli2024.21.2.8

Authors

Arsaythamby Veloo School of Education, Universiti Utara Malaysia, Malaysia
S. Kanageswari Suppiah Shanmugam School of Education, Universiti Utara Malaysia, Malaysia
Suheysen Revindran Awang Had Salleh Graduate School, Universiti Utara Malaysia, Malaysia

DOI:

https://doi.org/10.32890/mjli2024.21.2.8

Keywords:

Orang Asli, oral test accommodation, mathematics, computation, word problem, academic language, native language

Abstract

Purpose - The language load within mathematics test items can lead to inaccuracy in assessing Orang Asli pupils’ mathematical ability due to their struggle in comprehending the academic language. This study aims to determine the validity of using test accommodations in the form of oral academic language and oral native language when administering mathematics computation and word problem test items among the Orang Asli pupils.

Methodology - By employing a quantitative approach, this study utilised random equivalent group design to examine the validity of using oral test accommodations among 334 Grade Four Orang Asli pupils. Three testlets were constructed and used in this study: written test in the academic language (WAL), oral test in the academic language (OAL), and oral test in the native language (ONL). The three testlets comprised Paper 1 and Paper 2, each consisting of 20 mathematics computation and 20-word problem test items, respectively. One-way ANOVA was used to analyse and examine if there were differences in the mathematics performance between the three groups of tests administered to the Orang Asli pupils.

Findings - The results indicated that the test scores of the pupils on the oral tests were higher compared to the written test administered to the pupils for both mathematics computation and word problem items. For computation items, the pupils performed better in the OAL testlet, followed by the ONL testlet. On the other hand, the pupils obtained the highest test scores in the ONL testlet compared to the OAL testlet for word problem items. Furthermore, the results illustrate that there is a difference in the mathematics performance of the Orang Asli pupils between the oral tests in both languages and written tests in the academic language.

Significant - This study clearly shows that language plays a prominent role in affecting the Orang Asli pupils’ mathematics performance. By utilising the oral test as a promising test accommodation, this measure could help address the language barrier faced by the Orang Asli pupils in mathematics testing.

Author Biography

Arsaythamby Veloo, School of Education, Universiti Utara Malaysia

School of Education and Modern Languages

References

Abdul Halim Abdullah. (2022). A systematic review of what Malaysia can learn to improve Orang Asli students’ mathematics learning from other countries. Sustainability, 14(20), 13201. https://doi.org/10.3390/su142013201

Abedi, J., Leon, S., & Mirocha, J. (2003). Impact of student language background on content-based performance: Analyses of extant data. Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, Graduate School of Education & Information Studies, University of California, Los Angeles.

Achmetli, K., Schukajlow, S., & Rakoczy, K. (2019). Multiple solutions for real-world problems, experience of competence and students’ procedural and conceptual knowledge. International Journal of Science and Mathematics Education, 17(8), 16051625. https://doi.org/10.1007/s10763-018-9936-5 American Educational Research Association (AERA), American Psychological Association (APA), and National Council on

Measurement in Education (NCME). (2014). Standards for Educational and Psychological Testing. AERA. Arsaythamby Veloo, S. Kanageswari Suppiah Shanmugam, Ruzlan Md-Ali, Yus’aiman Jusoh Yusoff, & Rosna Awang-Hashim. (2021). Grade five Indigenous (Orang Asli) pupil’s achievement in bilingual versions of mathematics test. Journal of Language and Linguistic Studies, 17(4), 1863-1872.

Attar, Z., Blom, E., & Le Pichon, E. (2022). Towards more multilingual practices in the mathematics assessment of young refugee students: Effects of testing language and validity of parental assessment. International Journal of Bilingual Education and Bilingualism, 25(4), 1546-1561. https://doi.org/10.1080/13670 050.2020.1779648 Barcelos, T. S., Muñoz-Soto, R., Villarroel, R., Merino, E., & Silveira,

I. F. (2018). Mathematics learning through computational thinking activities: A systematic literature review. Journal of Universal Computer Science, 24(7), 815-845.

Boonen, A. J., de Koning, B. B., Jolles, J., & Van der Schoot, M. (2016). Word problem solving in contemporary math education: A plea for reading comprehension skills training. Frontiers in Psychology, 7(191), 1-10. https://doi.org/10.3389/ fpsyg.2016.00191 Braeuning, D., Hornung, C., Hoffmann, D., Lambert, K., Ugen, S., Fischbach, A., Schiltz, C., Hubner, N., Nagengast, B., &

Moeller, K. (2021). Long-term relevance and interrelation of symbolic and non-symbolic abilities in mathematicalnumerical development: Evidence from large-scale assessment data. Cognitive Development, 58, 101008. https://doi.org/10.1016/j.cogdev.2021.101008 Brezovszky, B., McMullen, J., Veermans, K., Hannula-Sormunen, M. M., Rodríguez-Aflecht, G., Pongsakdi, N., Laakkonen, E., & Lehtinen, E. (2019). Effects of a mathematics game-based learning environment on primary school students’ adaptive number knowledge. Computers and Education, 128, 63-74.

Chan, W. W. L., & Kwan, J. L. Y. (2021). Pathways to word problem solving: The mediating roles of schema construction and mathematical vocabulary. Contemporary Educational Psychology, 65, 101963. https://doi.org/10.1016/j. cedpsych.2021.101963

Cho, E., Fuchs, L. S., Seethaler, P. M., Fuchs, D., & Compton, D. L. (2020). Dynamic assessment for identifying Spanish-speaking English learners’ risk for mathematics disabilities: Does language of administration matter? Journal of Learning Disabilities, 53(5), 380-398. https://doi.org/10.1177%2F0022219419898887

Clarke, D., & Roche, A. (2018). Using contextualized tasks to engage students in meaningful and worthwhile mathematics learning. The Journal of Mathematical Behavior, 51, 95-108. https://doi.org/10.1016/j.jmathb.2017.11.006

Daroczy, G., Wolska, M., Meurers, W. D., & Nuerk, H. C. (2015). Word problems: A review of linguistic and numerical factors contributing to their difficulty. Frontiers in Psychology, 6, 348. https://doi.org/10.3389/fpsyg.2015.00348

Dunbar-Ortiz, R. (2014). An indigenous peoples’ history of the United States (Vol. 3). Beacon Press.

Frigo, T. (1999). Resources and teaching strategies to support Aboriginal children’s numeracy learning. Australian Council for Educational Research. https://research.acer.edu.au/ indigenous_education/11 Fuchs, L. S., Powell, S. R., Cirino, P. T., Schumacher, R. F., Marrin, S., Hamlet, C. L., Fuchs, D., Compton, D. L., & Changas,

P. S. (2014). Does calculation or word-problem instruction provide a stronger route to prealgebraic knowledge? Journal of Educational Psychology, 106(4), 990-1006. Fuchs, L. S., Gilbert, J. K., Fuchs, D., Seethaler, P. M., & N. Martin,

B. (2018). Text comprehension and oral language as predictors of word-problem solving: Insights into word-problem solving as a form of text comprehension. Scientific Studies of Reading, 22(2), 152-166. https://doi.org/10.1080/10888438.20 17.1398259

Geary, D. S. (2013). Early foundations for mathematics learning and their relations to learning disabilities. Current Direction in Psychological Science, 22(1), 23-27.

Graven, M., & Sibanda, L. (2018). Can mathematics assessments be considered valid if learners fail to access what is asked of them? South African Journal of Childhood Education, 8(1), 1-12. https://hdl.handle.net/10520/EJC-15d538c9bd

Haag, N., Heppt, B., Stanat, P., Kuhl, P., & Pant, H. A. (2013). Second language learners’ performance in mathematics: Disentangling the effects of academic language features. Learning and Instruction, 28, 24-34.

Krause, G. H., Vanderberg, M., E. Hung, E., & Skuratowicz, E. (2023). Computational thinking in a bilingual kindergarten classroom: Emergent ideas for teaching across content areas. Education and Information Technologies, 28(8), 9767-9782. https://doi.org/10.1007/s10639-022-11454-1

Krawec, J. L. (2014). Problem representation and mathematical problem solving of students of varying math ability. Journal of Learning Disabilities, 47(2), 103-115. https://doi.org/10.1177/0022219412436976

Kintsch, W., & Greeno, J. G. (1985). Understanding and solving word arithmetic problems. Psychological Review, 92(1), 109-129. https://psycnet.apa.org/doi/10.1037/0033-295X.92.1.109 Kucuk, U., Eyuboglu, M., Kucuk, H. O., & Degirmencioglu,

G. (2016). Importance of using proper post hoc test with ANOVA. International Journal of Cardiology, 209, 346. https://doi.org/10.1016/j.ijcard.2015.11.061

Ledibane, M., Kaiser, K., & Van der Walt, M. (2018). Acquiring mathematics as a second language: A theoretical model to illustrate similarities in the acquisition of English as a second language and mathematics. Pythagoras, 39(1), 1-12.

Leiss, D., Plath, J., & Schwippert, K. (2019). Language and mathematics-key factors influencing the comprehension process in reality-based tasks. Mathematical Thinking and Learning, 21(2), 131-153. https://doi.org/10.1080/10986065.2 019.1570835

Li, J. T., Arizmendi, G. D., & Swanson, H. L. (2022). The role of language comprehension skills and instructional practices in cross-language influence of Spanish-speaking dual language learners’ calculation skills. Early Childhood Research Quarterly, 61, 90-105. https://doi.org/10.1016/j. ecresq.2022.05.004

Livingston, S. A., & Kim, S. (2010). Random-groups equating with samples of 50 to 400 test takers. Journal of Educational Measurement, 47(2), 175-185. https://doi.org/10.1111/j.17453984.2010.00107.x

Lopez, A. (2023). Examining how Spanish-speaking English language learners use their linguistic resources and language modes in a dual language mathematics assessment task. Journal of Latinos and Education, 22(1), 198-210.

Menaga Suseelan, Chew, C. M., & Chin, H. (2022). Research on mathematics problem solving in elementary education conducted from 1969 to 2021: A bibliometric review. International Journal of Education in Mathematics, Science and Technology, 10(4), 1003-1029.

Messick, S. (1995). Standards of validity and the validity of standards in performance asessment. Educational Measurement: Issues and Practice, 14(4), 5-8.

Mishra, P., Pandey, C. M., Singh, U., Gupta, A., Sahu, C., & Keshri, A. (2019). Descriptive statistics and normality tests for statistical data. Annals of Cardiac Anaesthesia, 22(1), 67-72.

Moon, T. R. (2016). Differentiated instruction and assessment: An approach to classroom assessment in conditions of student diversity. In G. Brown & L. Harris (Eds.), Handbook of Human and Social Conditions in Assessment (pp. 300-317). Routledge.

Moschkovich, J. (2019). Codeswitching and mathematics learners: How hybrid language practices provide resources for student participation in mathematical practices. In J. MacSwan & C. J. Faltis (Eds.), Codeswitching in the Classroom (pp. 88-113). Routledge.

Mushin, I., Gardner, R., & Munro, J. M. (2013). Language matters in demonstrations of understanding in early years mathematics assessment. Mathematics Education Research Journal, 25, 415-433. https://doi.org/10.1007/s13394-013-0077-4

Nesher, P. (2020). Solving multiplication word problems. In G. Leinhardt, R. Putnam, & R. A. Hattrup (Eds.), Analysis of arithmetic for mathematics teaching (pp. 189-219). Routledge. Peng, P., Lin, X., Ünal, Z. E., Lee, K., Namkung, J., Chow, J., &

Sales, A. (2020). Examining the mutual relations between language and mathematics: A meta-analysis. Psychological Bulletin, 146(7), 595-634. https://psycnet.apa.org/doi/10.1037/ bul0000231

Perrenet, J. C., & Zwaneveld, B. (2012). The many faces of the mathematical modeling cycle. Journal of Mathematical Modelling and Application, 1(6), 3-21.

Perso, T. (2012). Cultural responsiveness and school education: With particular focus on Australia’s first peoples: A review & synthesis of the literature. Menzies, Australia. Pongsakdi, N., Kajamies, A., Veermans, K., Lertola, K., Vauras,

M., & Lehtinen, E. (2020). What makes mathematical word problem solving challenging? Exploring the roles of word problem characteristics, text comprehension, and arithmetic skills. ZDM-Mathematics Education, 52(1), 33-44. https://doi.org/10.1007/s11858-019-01118-9

Praet, M., Titeca, D., Ceulemans, A., & Desoete, A. (2013). Language in the prediction of arithmetics in kindergarten and grade 1. Learning and Individual Differences, 27, 90-96. https://doi.org/10.1016/j.lindif.2013.07.003

Prediger, S., Erath, K., & Opitz, E. M. (2019). The language dimension of mathematical difficulties. In A. Fritz, V. G. Haase, & P. Räsänen (Eds.), International handbook of mathematical learning difficulties: From the laboratory to the classroom (pp. 437-455). Springer.

Purpura, D. J., & Reid, E. E. (2016). Mathematics and language: Individual and group differences in mathematical language skills in young children. Early Childhood Research Quarterly, 36, 259-268. https://doi.org/10.1016/j.ecresq.2015.12.020

Riccomini, P. J., Smith, G. W., Hughes, E. M., & Fries, K. M. (2015). The language of mathematics: The importance of teaching and learning mathematical vocabulary. Reading & Writing Quarterly, 31(3), 235-252.

Roohr, K. C., & Sireci, S. G. (2017). Evaluating computer-based test accommodations for English learners. Educational Assessment, 22(1), 35-53. Ruzlan Md-Ali, Arsaythamby Veloo, S. Kanageswari Suppiah Shanmugam, Yus’aiman Jusoh Yusoff, & Rosna Awang Hashim. (2021). The issues and challenges of mathematics teaching and learning in Malaysia “Orang Asli” primary schools from teachers’ perspectives. Malaysian Journal of Learning and Instruction, 18(2), 129-160. S. Kanageswari Suppiah Shanmugam, Arsaythamby Veloo, Ruzlan

Md-Ali. (2021). Culturally responsive assessment: Assessing mathematics performance of Indigenous pupils in Malaysia using trilingual test. Diaspora, Indigenous, and Minority Education, 15(2), 113-136. S. Kanageswari Suppiah Shanmugam, Arsaythamby Veloo, &

Yus’aiman Bin Jusoh. (2023). The challenges of virtual learning: An exploratory study among Orang Asli Pupils from the mathematics teachers’ perspectives. Diaspora, Indigenous, and Minority Education, 18(3), 211-229. https://doi.org/10.108 0/15595692.2023.2212823

Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418-429. https://doi.org/10.1111/j.15404781.2011.01210.x

Skinner, M. G., & Cuevas, J. A. (2023). The effects of schema-based instruction on word-problems in a third-grade mathematics classroom. International Journal of Instruction, 16(1), 855880. https://doi.org/10.29333/iji.2023.16148a

Stansfield, C. W. (2011). Oral translation as a test accommodation for ELLs. Language Testing, 28(3), 401-416. https://doi.org/10.1177/0265532211404191

Swanson, H. L. (2020). The relationship between executive processing and computational growth among monolingual and English learners with and without math difficulties: Does it help to be bilingual? Cognitive Development, 56, 100961. https://doi.org/10.1016/j.cogdev.2020.100961. Swanson, H. L., Kong, J., Petcu, S. D., & Asencio Pimentel, M.

F. (2020). Can difficulties in language acquisition and specific learning disabilities be separated among English learners? Exceptional Children, 86(3), 293-309. https://doi.org/10.1016/j.cogdev.2020.100961

Tambunan, H. (2019). The effectiveness of the problem solving strategy and the scientific approach to students’ mathematical capabilities in high order thinking skills. International Electronic Journal of Mathematics Education, 14(2), 293-302.

Timmermans, R. E., Van Lieshout, E. D. C. M., & Verhoevan, L. (2007). Gender related effects of contemporary math instruction for low performers on problem-solving behaviour. Learning and Instruction, 17(1), 42-54.

Trumbull, E., & Nelson-Barber, S. (2019). The ongoing quest for culturally-responsive assessment for Indigenous students in the US. Frontiers in Education, 4(40), 1-11. https://doi.org/10.3389/feduc.2019.00040

UNESCO (2015). Education for All 2000–2015: Achievements and challenges. EFA Global Monitoring Report. Paris: UNESCO.

van Dijk, T. A. (1993). Principles of critical discourse analysis. Discourse & Society, 4(2), 249-283. https://doi.org/1 0.1177%2F0957926593004002006

Verschaffel, L., Schukajlow, S., Star, J., & Van Dooren, W. (2020). Word problem in mathematics education: A survey. ZDM Mathematics Education, 52, 1-16.

Vidal Rodeiro, C., & Macinska, S. (2022). Equal opportunity or unfair advantage? The impact of test accommodations on performance in high-stakes assessments. Assessment in Education: Principles, Policy & Practice, 29(4), 462-481.

Vukovic, R. K., & Lesaux, N. K. (2013). The language of mathematics: Investigating the ways language counts for children’s mathematical development. Journal of Experimental Child Psychology, 115(2), 227-244. https://doi.org/10.1016/j. jecp.2013.02.002

Wijns, N., De Smedt, B., Verschaffel, L., & Torbeyns, J. (2020). Are preschoolers who spontaneously create patterns better in mathematics? British Journal of Educational Psychology, 90(3), 753-769. https://doi.org/10.1111/bjep.12329

Wilkinson, L. C. (2018). Teaching the language of mathematics: What the research tells us teachers need to know and do. The Journal of Mathematical Behavior, 51, 167-174. https://doi.org/10.1016/j.jmathb.2018.05.001

Zaleha Ismail, Tan, Y. C., & Nur Amira Muda. (2020). Numeracy competency of Year 5 aboriginal students using written and oral tests. The Mathematics Enthusiast, 17(1), 32-62. https://doi.org/10.54870/1551-3440.1479