ASSESSMENT RUBRIC FOR RESEARCH REPORT WRITING: A TOOL FOR SUPERVISION

  • Nurliyana Bukhari School of Education, Universiti Utara Malaysia
  • Jamilah Jamal School of Multimedia, Technology, and Communication, Universiti Utara Malaysia
  • Adibah Ismail School of Multimedia, Technology, and Communication, Universiti Utara Malaysia
  • Jauriyah Shamsuddin School of Business Management, Universiti Utara Malaysia

Abstract

Purpose – Assessment rubric often lacks rigor and is underutilized. This article reports the effectiveness of the use of several assessment rubrics for a research writing course. Specifically, we examined students’ perceived changes and observed changes in their Chapter 1 thesis writing as assessed by supervisors using an existing departmental rubric and a new task-specific rubric.
 
Methodology – Using action research methodology, two of the authors played active roles as the course supervisors, i.e., the practitioners. Two final year undergraduate students from a communication department (one from each supervisor) participated by writing three drafts of the first chapter of their research: (1) without a rubric, (2) with an existing departmental rubric, and (3) with a revised rubric. We collected artefacts of students’ writing drafts; students’ interviews; and supervisors’ reflections over the course of four months. We employed content analysis to evaluate students’ writing, while thematic analysis to analyze the students’ semi-structured interview and supervisors’ reflections.
 
Findings – The findings suggest substantial improvements between the three drafts of students’ writing. Each student-supervisor pair acknowledged the improvements in the student’s writing after the introduction of the departmental rubric. With the newly revised rubric, they noted additional and more specific improvements especially in the scope of literature searches, problem statements, formulation of research questions, and operational definitions of variables; more generally, they also indicated improvements in the clarity of writing by using examples and providing relevant explanations tailored to the research topics.
 
Significance – With effective scaffolding in supervision, students will regulate their learning and assess the quality of their own research report writing. We demonstrated the importance and benefits of a properly designed and validated rubric tailored to the program and course objectives to help students improve their writing drafts. Collective collaboration and input-sharing from faculty and instructors in developing and improving a rubric specific to the course and program objectives will produce quality assignments, provide constructive learning experiences for students, and achieve better grading for the program and department.

References

Alonso-Tapia, J., & Panadero, E. (2010). Effect of self-assessment scripts on self-regulation and learning.
Infancia y Aprendizaje, 33(3), 385-397.
Anderson, C., Day, K., & McLaughlin, P. (2006). Mastering the dissertation. Studies in Higher
Education. 31(2), 149-168.

Anderson, J.S., & Mohrweis, L.C. (2008). Using rubrics to assess accounting students’ writing,
oral presentations, and ethics skills. American Journal of Business Education, 1, 85–93.
Andrade, H. (2000). Using rubrics to promote thinking and learning. Educational Leadership
57(5), 13-18.
Andrade, H. G. (2001). The effects of instructional rubrics on learning to write. Current Issues
in Education, 4. Retrieved from https://cie.asu.edu/ojs/index.php/cieatasu/article/view/1630
Andrade, H., & Du, Y. (2005). Student perspectives on rubric-referenced assessment. Practical
Assessment Research & Evaluation, 10(3), 2.
Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through self-assessment.
Theory into practice, 48(1), 12-19.
Awang Had Salleh Graduate School of Arts and Sciences, Universiti Utara Malaysia. (2015). Thesis
format for postgraduates. Retrieved from https://drive.google.com/file/d/1K1Cxpu-SwBvSq4e0V5OyT19_Wty4pQET/view
Bailey, S. (2003). Academic writing: A handbook for international students (3rd ed.). Oxon: Routledge.
Bharuthram, S., & Patel, M. (2017). Co-constructing a rubric checklist with first year university students:
A self-assessment tool. Journal of Applied Language Studies, 11(4), 35-55.
Blankenship, J. M., & Wilson, E. K. (2009). Teaching writing and research skills in an
undergraduate public administration class. Proceedings of the Southern Political Science
Association Annual Meeting. New Orleans, Los Angeles.
Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in
Psychology, 3(2), 77-101.
Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport,
Exercise and Health, 11(4), 589–597.
Burk, B. N. (2020) Strategies for teaching undergraduate writing intensive courses, SCHOLE: A Journal
of Leisure Studies and Recreation Education, DOI: 10.1080/1937156X.2020.1763874

Burns, A. (2009). Doing action research in English language teaching: A guide for
practitioners. New York: Routledge.
Carriveau, R. (2010). Connecting the Dots. Denton, TX: Fancy Fox Publications, Inc.
Carson, L., & Kavish, D. (2018). Scaffolding rubrics to improve student writing: Preliminary
results of using rubrics in a sociology program to enhance learning and mechanical writing skills. Societies, 8(34), 1-9.
Clabough, E. B. D., & Clabough, S. W. (2016). Using rubrics as a scientific writing instructional
method in early stage undergraduate neuroscience study. The Journal of Undergraduate Neuroscience Education, 15(1), 85-93.
Caughlan, S., & Jiang, H. (2014). Observation and teacher quality: Critical analysis of
observational instruments in preservice teacher performance assessment. Journal of Teacher Education, 65(5), 375-388.
Creswell, J. W. (2012). Educational research: Planning, conducting and evaluating quantitative and
qualitative research (4th ed.). Edinburgh Gate: Pearson Education Limited.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th
ed.). Thousand Oaks, California: SAGE Publications Inc.
Dawson, P. (2017). Assessment rubrics: towards clearer and more replicable design, research and
practice. Assessment & Evaluation in Higher Education, 42(3), 347-360.
Denis, C., Colet, N. R., & Lison, C. (2019). Doctoral Supervision in North America: Perception and
Challenges of Supervisor and Supervisee. Higher Education Studies, 9(1), 30-39.
Dochy, F., Segers, M., & Sluijsmans, D. (2006). The use of self-, peer and co-assessment in higher
education: A review. Studies in Higher Education, 24(3), 331–350.
Fraile, J., Panadero, E., & Pardo, R. (2017). Co-creating rubrics: The effects on self-regulated learning,
self-efficacy and performance of establishing assessment criteria with students. Evaluation, 53, 69-76.

Green, R., & Bowser, M. (2006). Observations from the field: Sharing a literature review
rubric. Journal of Library Administration, 45(1-2), 185–202.
Guba, E. G., & Lincoln, Y. S. (1982). Epistemological and methodological bases of naturalistic inquiry.
Educational Communication and Technology Journal, 30 (4), 233-252.
He, X., & Canty, A. (2012). Empowering student learning through rubric-referenced self-
assessment. Journal of Chiropractic Education, 26(1), 24-31.
Hooper, R.I., & Butler, S. (2008). Student transfer of general education English skills to a social
work diversity Course: Is it happening? Journal of the Idaho Academy of Science, 44(2), 1–10.
Horton, E. G., & Diaz, N. (2011). Learning to write and writing to learn social work concepts:
Application of writing across the curriculum strategies and techniques to a course for undergraduate social work students. Journal of Teaching in Social Work, 31(1), 53-64.
Howell, R. J. (2011). Exploring the impact of grading rubrics on academic performance:
Findings from a quasi-experimental, pre-post evaluation. Journal on Excellence in College Teaching, 22(2), 31-49.
Huskin, P. R. (2016). Engagement strategies for increasing student writing success. Education,
136(3), 283-290.
Ismail, A., Zainal Abiddin, N., Hassan, R., & Ro'is, I. (2014). The Profound of Students' Supervision
Practice in Higher Education to Enhance Student Development. Higher Education Studies, 4(4), 1-6.
Jamieson, S., & Gray, C. (2006). The supervision of undergraduate research students:
expectations of student and supervisor. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education. 1(1), 37-59.
Jönsson, A. (2014). Rubrics as a way of providing transparency in assessment. Assessment &
Evaluation in Higher Education, 39(7), 840-852.


Jönsson, A., & Panadero, E. (2018). The use of rubrics to support AfL in higher education. Presented at
the EARLI SIG 1 Assessment Conference, Munich, Germany. Retrieved from http://urn.kb.se/resolve?urn=urn:nbn:se:hkr:diva-18491
Kaur, A., Awang-Hashim, R., & Kaur, M. (2019) Students’ experiences of co-creating classroom
instruction with faculty- a case study in eastern context, Teaching in Higher Education, 24(4), 461-477.
Kinne, L. J., Hasenbank, J. F., & Coffey, D. (2014). Are we there yet? Using rubrics to support progress
toward proficiency and model formative assessment. AILACTE Journal, 11(1), 109-128.
Kitsantas, A., & Zimmerman, B. J. (2006). Enhancing self-regulation of practice: The influence of
graphing and self-evaluative standards. Metacognition and Learning, 201-212.
Krippendorff, K. (2004). Content Analysis: An Introduction to Its Methodology (2nd ed.). Thousand Oaks, CA: Sage
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intracalass correlation coefficients
for reliability research. Journal of Chiropractic Medicine, 15, 155-163.
Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why
they matter. Washington, DC: Association of American Colleges and Universities.
Lankau, M.J., & T.A. Scandura. (2007). Mentoring as a forum for personal learning in organizations. In
B.R. Ragins and K.E. Kram (Ed.), The handbook of mentoring at work. Theory, research, and practice (pp. 95–122). London: Sage.
Linden, J., Ohlin, M., & Brodin, E. M. (2013). Mentorship, supervision, and learning experience in PhD.
education. Studies in Higher Education, 38(5), 639-662.
Lipnevich, A. A., McCallen, L. N., Miles, K. P., & Smith, J. K. (2014). Mind the gap! Students’
use of exemplars and detailed rubrics as formative assessment. Instructional Science, 42(4), 539-559.

Mansilla, V. B., Duraisingh, E. D., Wolfe, C. R., & Haynes, C. (2009). Targeted assessment
rubric: An empirically grounded rubric for interdisciplinary writing. The Journal of Higher Education, 80(3), 334-353.
Maxwell, T. W., & Smyth, R. (2010). Research supervision: the research management matrix. Higher
Education, 59(4), 407-422.
McAlpine, L., & McKinnon, M. (2013). Supervision–the most variable of variables: Student
perspectives. Studies in Continuing Education, 35(3), 265-280.
McCulloch, A., Kumar, V., van Schalkwyk, S., & Wisker, G. (2016) Excellence in doctoral supervision:
An examination of authoritative sources across four countries in search of performance higher than competence. Quality in Higher Education, 22(1), 64-77.
Mertler, C.A. (2001). Designing scoring rubrics for your classroom. Practical Assessment,
Research & Evaluation, 7(25), 1-8.
Moxham, L., Dwyer, T., & Reid-Searl, K. (2013). Articulating expectations for PhD candidature
upon commencement: Ensuring supervisor/student ‘best fit’. Journal of Higher Education Policy and Management, 35(4), 345-354.
Murad Sani, A. (2016). Academic writing essentials: A guide for postgraduate students. Sintok, Kedah,
Malaysia: UUM Press.
Neuman, L. W. (2014). Social research methods: Qualitative and quantitative approaches (7th ed.).
Essex%
Published
2021-07-30
How to Cite
BUKHARI, Nurliyana et al. ASSESSMENT RUBRIC FOR RESEARCH REPORT WRITING: A TOOL FOR SUPERVISION. Malaysian Journal of Learning and Instruction, [S.l.], v. 18, n. 2, p. 1-43, july 2021. ISSN 2180-2483. Available at: <http://e-journal.uum.edu.my/index.php/mjli/article/view/mjli2021.18.2.1>. Date accessed: 07 dec. 2021. doi: https://doi.org/10.32890/mjli2021.18.2.1.