EXPLORING CONFIDENCE ACCURACY AND ITEM DIFFICULTY IN CHANGING MULTIPLE-CHOICE ANSWERS OF SCIENTIFIC REASONING TEST

Abstract

Purpose – Researchers discovered that when students were given the opportunity to change their answers, a majority changed their
responses from incorrect to correct, and this change often increased the overall test results. What prompts students to modify their
answers? This study aims to examine the modification of scientific reasoning test, with additional exploration on confidence accuracy
and its relation to item difficulty.

Methodology –A set of pre-test and post-test experiments which included 20 items of scientific reasoning test with confidence
judgement on each item were used. The items of the instruments were assessed for their validity by analysing their psychometric properties using the three-parameter (3PL) Item Response Theory, which was carried out in R studio. The set of items were randomly administered to 205 Indonesian undergraduate students with a background in science education related major. The accuracy of confidence was determined by categorising correct or incorrect answers to scientific reasoning questions based on their level of confidence.

Findings –The results revealed that responses were modified more frequently from incorrect to correct than from correct to incorrect,
resulting in a significant gain in overall scientific reasoning score although these modifications were not shown to be connected to the
item’s difficulty level. Even though confidence level also increased significantly, it was observed that Indonesian students repeatedly
responded with overconfidence even after sitting for the same test after three weeks, which could indicate a lack of metacognitive ability. The findings of this study serve to spur educators to begin actively engaging in metacognitive training in their teaching and learning activities as a result of overconfidence that frequently occurs among Indonesian students in examinations.

Significance –This study provides further substantiation in the field of scientific reasoning and cognitive science; that a trend of confidence accuracy change in scientific reasoning test has been observed. It also contributes to uncovering the true ability of Indonesian students when performing such reasoning tests through their repeated attempts.