ASSESSMENT RUBRIC FOR RESEARCH REPORT WRITING: A TOOL FOR SUPERVISION

Authors

  • Nurliyana Bukhari School of Education, Universiti Utara Malaysia
  • Jamilah Jamal School of Multimedia, Technology, and Communication, Universiti Utara Malaysia
  • Adibah Ismail School of Multimedia, Technology, and Communication, Universiti Utara Malaysia
  • Jauriyah Shamsuddin School of Business Management, Universiti Utara Malaysia

DOI:

https://doi.org/10.32890/mjli2021.18.2.1

Keywords:

Assessment rubric, constructive learning, research writing, supervision, feedback, action research, scholarship of teaching and learning, high impact educational practices.

Abstract

Purpose – Assessment rubric often lacks rigor and is underutilized. This article reports the effectiveness of the use of several assessment rubrics for a research writing course. Specifically, we examined students’ perceived changes and observed changes in their Chapter 1 thesis writing as assessed by supervisors using an existing departmental rubric and a new task-specific rubric.

Methodology – Using action research methodology, two of the authors played active roles as the course supervisors, i.e., the practitioners. Two final year undergraduate students from a communication department (one from each supervisor) participated by writing three drafts of the first chapter of their research: (1) without a rubric, (2) with an existing departmental rubric, and (3) with a revised rubric. We collected artefacts of students’ writing drafts; students’ interviews; and supervisors’ reflections over the course of four months. We employed content analysis to evaluate students’ writing, while thematic analysis to analyze the students’ semi-structured interview and supervisors’ reflections.

Findings – The findings suggest substantial improvements between the three drafts of students’ writing. Each student-supervisor pair acknowledged the improvements in the student’s writing after the introduction of the departmental rubric. With the newly revised rubric, they noted additional and more specific improvements especially in the scope of literature searches, problem statements, formulation of research questions, and operational definitions of variables; more generally, they also indicated improvements in the clarity of writing by using examples and providing relevant explanations tailored to the research topics.

Significance – With effective scaffolding in supervision, students will regulate their learning and assess the quality of their own research report writing. We demonstrated the importance and benefits of a properly designed and validated rubric tailored to the program and course objectives to help students improve their writing drafts. Collective collaboration and input-sharing from faculty and instructors in developing and improving a rubric specific to the course and program objectives will produce quality assignments, provide constructive learning experiences for students, and achieve better grading for the program and department.

Metrics

Metrics Loading ...

References

Alonso-Tapia, J., & Panadero, E. (2010). Effect of self-assessment scripts on self-regulation and learning.

Infancia y Aprendizaje, 33(3), 385-397.

Anderson, C., Day, K., & McLaughlin, P. (2006). Mastering the dissertation. Studies in Higher

Education. 31(2), 149-168.

Anderson, J.S., & Mohrweis, L.C. (2008). Using rubrics to assess accounting students’ writing,

oral presentations, and ethics skills. American Journal of Business Education, 1, 85–93.

Andrade, H. (2000). Using rubrics to promote thinking and learning. Educational Leadership

(5), 13-18.

Andrade, H. G. (2001). The effects of instructional rubrics on learning to write. Current Issues

in Education, 4. Retrieved from https://cie.asu.edu/ojs/index.php/cieatasu/article/view/1630

Andrade, H., & Du, Y. (2005). Student perspectives on rubric-referenced assessment. Practical

Assessment Research & Evaluation, 10(3), 2.

Andrade, H., & Valtcheva, A. (2009). Promoting learning and achievement through self-assessment.

Theory into practice, 48(1), 12-19.

Awang Had Salleh Graduate School of Arts and Sciences, Universiti Utara Malaysia. (2015). Thesis

format for postgraduates. Retrieved from https://drive.google.com/file/d/1K1Cxpu-SwBvSq4e0V5OyT19_Wty4pQET/view

Bailey, S. (2003). Academic writing: A handbook for international students (3rd ed.). Oxon: Routledge.

Bharuthram, S., & Patel, M. (2017). Co-constructing a rubric checklist with first year university students:

A self-assessment tool. Journal of Applied Language Studies, 11(4), 35-55.

Blankenship, J. M., & Wilson, E. K. (2009). Teaching writing and research skills in an

undergraduate public administration class. Proceedings of the Southern Political Science

Association Annual Meeting. New Orleans, Los Angeles.

Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in

Psychology, 3(2), 77-101.

Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport,

Exercise and Health, 11(4), 589–597.

Burk, B. N. (2020) Strategies for teaching undergraduate writing intensive courses, SCHOLE: A Journal

of Leisure Studies and Recreation Education, DOI: 10.1080/1937156X.2020.1763874

Burns, A. (2009). Doing action research in English language teaching: A guide for

practitioners. New York: Routledge.

Carriveau, R. (2010). Connecting the Dots. Denton, TX: Fancy Fox Publications, Inc.

Carson, L., & Kavish, D. (2018). Scaffolding rubrics to improve student writing: Preliminary

results of using rubrics in a sociology program to enhance learning and mechanical writing skills. Societies, 8(34), 1-9.

Clabough, E. B. D., & Clabough, S. W. (2016). Using rubrics as a scientific writing instructional

method in early stage undergraduate neuroscience study. The Journal of Undergraduate Neuroscience Education, 15(1), 85-93.

Caughlan, S., & Jiang, H. (2014). Observation and teacher quality: Critical analysis of

observational instruments in preservice teacher performance assessment. Journal of Teacher Education, 65(5), 375-388.

Creswell, J. W. (2012). Educational research: Planning, conducting and evaluating quantitative and

qualitative research (4th ed.). Edinburgh Gate: Pearson Education Limited.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th

ed.). Thousand Oaks, California: SAGE Publications Inc.

Dawson, P. (2017). Assessment rubrics: towards clearer and more replicable design, research and

practice. Assessment & Evaluation in Higher Education, 42(3), 347-360.

Denis, C., Colet, N. R., & Lison, C. (2019). Doctoral Supervision in North America: Perception and

Challenges of Supervisor and Supervisee. Higher Education Studies, 9(1), 30-39.

Dochy, F., Segers, M., & Sluijsmans, D. (2006). The use of self-, peer and co-assessment in higher

education: A review. Studies in Higher Education, 24(3), 331–350.

Fraile, J., Panadero, E., & Pardo, R. (2017). Co-creating rubrics: The effects on self-regulated learning,

self-efficacy and performance of establishing assessment criteria with students. Evaluation, 53, 69-76.

Green, R., & Bowser, M. (2006). Observations from the field: Sharing a literature review

rubric. Journal of Library Administration, 45(1-2), 185–202.

Guba, E. G., & Lincoln, Y. S. (1982). Epistemological and methodological bases of naturalistic inquiry.

Educational Communication and Technology Journal, 30 (4), 233-252.

He, X., & Canty, A. (2012). Empowering student learning through rubric-referenced self-

assessment. Journal of Chiropractic Education, 26(1), 24-31.

Hooper, R.I., & Butler, S. (2008). Student transfer of general education English skills to a social

work diversity Course: Is it happening? Journal of the Idaho Academy of Science, 44(2), 1–10.

Horton, E. G., & Diaz, N. (2011). Learning to write and writing to learn social work concepts:

Application of writing across the curriculum strategies and techniques to a course for undergraduate social work students. Journal of Teaching in Social Work, 31(1), 53-64.

Howell, R. J. (2011). Exploring the impact of grading rubrics on academic performance:

Findings from a quasi-experimental, pre-post evaluation. Journal on Excellence in College Teaching, 22(2), 31-49.

Huskin, P. R. (2016). Engagement strategies for increasing student writing success. Education,

(3), 283-290.

Ismail, A., Zainal Abiddin, N., Hassan, R., & Ro'is, I. (2014). The Profound of Students' Supervision

Practice in Higher Education to Enhance Student Development. Higher Education Studies, 4(4), 1-6.

Jamieson, S., & Gray, C. (2006). The supervision of undergraduate research students:

expectations of student and supervisor. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education. 1(1), 37-59.

Jönsson, A. (2014). Rubrics as a way of providing transparency in assessment. Assessment &

Evaluation in Higher Education, 39(7), 840-852.

Jönsson, A., & Panadero, E. (2018). The use of rubrics to support AfL in higher education. Presented at

the EARLI SIG 1 Assessment Conference, Munich, Germany. Retrieved from http://urn.kb.se/resolve?urn=urn:nbn:se:hkr:diva-18491

Kaur, A., Awang-Hashim, R., & Kaur, M. (2019) Students’ experiences of co-creating classroom

instruction with faculty- a case study in eastern context, Teaching in Higher Education, 24(4), 461-477.

Kinne, L. J., Hasenbank, J. F., & Coffey, D. (2014). Are we there yet? Using rubrics to support progress

toward proficiency and model formative assessment. AILACTE Journal, 11(1), 109-128.

Kitsantas, A., & Zimmerman, B. J. (2006). Enhancing self-regulation of practice: The influence of

graphing and self-evaluative standards. Metacognition and Learning, 201-212.

Krippendorff, K. (2004). Content Analysis: An Introduction to Its Methodology (2nd ed.). Thousand Oaks, CA: Sage

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intracalass correlation coefficients

for reliability research. Journal of Chiropractic Medicine, 15, 155-163.

Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why

they matter. Washington, DC: Association of American Colleges and Universities.

Lankau, M.J., & T.A. Scandura. (2007). Mentoring as a forum for personal learning in organizations. In

B.R. Ragins and K.E. Kram (Ed.), The handbook of mentoring at work. Theory, research, and practice (pp. 95–122). London: Sage.

Linden, J., Ohlin, M., & Brodin, E. M. (2013). Mentorship, supervision, and learning experience in PhD.

education. Studies in Higher Education, 38(5), 639-662.

Lipnevich, A. A., McCallen, L. N., Miles, K. P., & Smith, J. K. (2014). Mind the gap! Students’

use of exemplars and detailed rubrics as formative assessment. Instructional Science, 42(4), 539-559.

Mansilla, V. B., Duraisingh, E. D., Wolfe, C. R., & Haynes, C. (2009). Targeted assessment

rubric: An empirically grounded rubric for interdisciplinary writing. The Journal of Higher Education, 80(3), 334-353.

Maxwell, T. W., & Smyth, R. (2010). Research supervision: the research management matrix. Higher

Education, 59(4), 407-422.

McAlpine, L., & McKinnon, M. (2013). Supervision–the most variable of variables: Student

perspectives. Studies in Continuing Education, 35(3), 265-280.

McCulloch, A., Kumar, V., van Schalkwyk, S., & Wisker, G. (2016) Excellence in doctoral supervision:

An examination of authoritative sources across four countries in search of performance higher than competence. Quality in Higher Education, 22(1), 64-77.

Mertler, C.A. (2001). Designing scoring rubrics for your classroom. Practical Assessment,

Research & Evaluation, 7(25), 1-8.

Moxham, L., Dwyer, T., & Reid-Searl, K. (2013). Articulating expectations for PhD candidature

upon commencement: Ensuring supervisor/student ‘best fit’. Journal of Higher Education Policy and Management, 35(4), 345-354.

Murad Sani, A. (2016). Academic writing essentials: A guide for postgraduate students. Sintok, Kedah,

Malaysia: UUM Press.

Neuman, L. W. (2014). Social research methods: Qualitative and quantitative approaches (7th ed.).

Essex%

Additional Files

Published

30-07-2021

How to Cite

Bukhari, N., Jamal, J., Ismail, A., & Shamsuddin, J. (2021). ASSESSMENT RUBRIC FOR RESEARCH REPORT WRITING: A TOOL FOR SUPERVISION. Malaysian Journal of Learning and Instruction, 18(2), 1–43. https://doi.org/10.32890/mjli2021.18.2.1