A Con fi rmatory Factor Analysis of a Newly Integrated Multidimensional School Engagement Scale

Inconsistencies in measures of school engagement in the literature have called for a re-conceptualization of the school engagement construct. Although many researchers view school engagement as a multifaceted construct, to our knowledge, none of the available instruments have integrated all the important domains that represent its multifaceted nature. This study is our fi rst attempt to examine the psychometric properties of a newly integrated Multidimensional School Engagement Scale (MSES). Data were gathered from 2,381 secondary school students, aged 14 to 16, from 40 day schools in northern Malaysia. Exploratory factor analysis and confi rmatory factor analytic techniques were used to examine the instrument. Based on the available literature, we posited an a priori hypothesis that the scales could be explained by three fi rst-order factors and one second-order factor. We used SPSS v.12 and AMOS 6.0 to analyze the data. Findings supported our hypothesis that the school engagement construct can be explained by three fi rst-order factors and one hierarchical factor comprising cognitive engagement, behavioural engagement, and psychological engagement sub-scales. Findings also showed acceptable internal consistency reliability for the overall scale and the three specifi c sub-scales of adolescent school engagement.


INTRODUCTION
While much has been accomplished in the education of Malaysian youths as the nation progresses towards developed nation status, parents, social scientists, and policymakers alike recognize a general decline in respect for authority and school institutions, especially among adolescents, and have expressed concern that they are becoming more disengaged from school.Reports of truancy among school children are rampant (Ponteng, 2005), and student crime has been on the rise, with an average of seven schoolchildren aged between 13 and 15 being arrested every day, nationwide (Student crime, 2005).Random incidents reported in the media on gangsterism and school bullying also represent symptoms of school disengagement.Modell and Elder (2002) have argued that teachers and school administrators can no longer expect students to automatically respect and comply with norms of school behaviour and academic expectations.In fact, instead of being perceived as an interesting place to go, school may be regarded as not worth the effort (Pope, 2002), leading to a signifi cant decline in motivational levels across grades (Eccles & Midgley, 1989;Eccles et al., 1993;Fredericks & Eccles, 2002).Disengagement from school among its adolescents does not bode well for a rapidly developing economy that needs knowledge workers who can synthesize and evaluate new information critically and solve novel problems continuously.
Substantial evidence gathered in the West suggests that levels of academic achievement during adolescence and educational attainment later in life are strong predictors of well-being in adulthood (McNeal, 1995;Cairns, Cairns, & Neckerman, 1989).Ross and Wu (1995) found that better educated people tend to be healthier and report having higher levels of socio-emotional wellbeing.Thus, earlier school engagement may steer individuals towards future positive well-being, which is an important ingredient for a stable society.Given the importance of educational adjustment in adolescence, there is a need to examine the level of school engagement among Malaysian adolescents, so that areas that may need intervention can be identifi ed.However, studies on school engagement tend to be camoufl aged by topics such as motivation, self-regulation and help-seeking, making it diffi cult for us to capture what school engagement entails.Hence, it is imperative that we synergize research efforts by having an instrument that can measure the different facets of school engagement.This study is our initial attempt to develop a Multidimensional School Engagement Scale (MSES) by integrating three school engagement dimensions: Cognitive, behavioural, and psychological engagement.

SCHOOL ENGAGEMENT
School engagement is one aspect of adolescents' development that deserves further scrutiny as it can be regarded as an antecedent of their educational functioning.Lack of school engagement in many adolescents today may be seen as a sign of school alienation.Although there is no clear consensus on what constitutes school engagement or how it can be identifi ed, it is often viewed as a multifaceted construct which is dynamically interrelated within individuals (Fredricks, Blumenfeld, & Paris, 2004;Guthrie & Anderson, 1999;Guthrie & Wigfi eld, 2000;Jimerson, Campos, & Grief, 2003;Libbey, 2004).Some of its components are not readily observable, and thus, are either inferred or assessed from self-report measures.Generally, school engagement includes both emotional and behavioural factors.Attending school and classes, completing schoolwork and participating in extracurricular activities are all related to positive school performance, an experience which should lead to adolescents' positive identifi cation with school.On the other hand, coming to class unprepared, not completing schoolwork, becoming disinterested in school, getting bad grades or getting suspended are all symptoms of adolescents' disengaging from school.In a broader sense, school engagement often refers to students' level of connectedness to school (Connell, Spencer, & Aber, 1994), which includes their participation in school activities, their identifi cation with school and acceptance of school values (Morse, Christenson, & Lehr, 2003).School engagement literature often covers variables such as relationship building, meaningful participation, caring peer relationships and school bonding, and some researchers use these terms interchangeably to refer to school engagement (see Libbey, 2004).Morse et al. (2003) further posited four dimensions of school engagement: (1) Academic Engagement: The amount of time spent on task and the number of credits earned.
(2) Cognitive Engagement: Students' focus on and thinking about academic tasks, processing information, and selfdirected learning.
(3) Behavioral Engagement: Participation in classroom and extra-curricular activities (4) Psychological Engagement: Students' sense of identifi cation with school, sense of membership at school, and positive relationships with peers.
Due to the proliferation of studies on school engagement, several researchers have called for a reconceptualization of the construct (Fredricks, Blumenfeld, & Paris, 2004;Libbey, 2004).The terms school engagement, school belonging, school bonding, and school connectedness are often used interchangeably in literature on motivation, risk and resilience.Although different terminologies have been used, arguably these represent the same construct because they are measured with either similar instruments or assessed with similar, if not identical items (see Goodenow, 1993aGoodenow, , 1993b)).Libbey (2004) identifi ed nine main themes that have emerged in the literature on school connectedness: academic engagement, belongingness, discipline and fairness, liking for school, student voice, extracurricular activities, peer relations, safety and teacher support.Despite the use of various scales in the literature, Libbey concluded that all these school connectedness variables were related highly with student outcomes.Adolescents who can identify themselves with school or feel connected to school perform better academically.Evidence from health and education literature shows that these non-academic factors contribute signifi cantly to the variance in school success.
In a special issue of the California School Psychologist (Volume 8, 2003) on "school engagement, youth development, and school success, " Jimerson, Campos and Greif (2003) critically reviewed conceptual defi nitions and measurements of school engagement, which were synthesized according to fi ve contexts: 1) academic performance; 2) classroom behaviour; 3) extracurricular involvement; 4) interpersonal relationships; and 5) the school community.Similar to many school engagement researchers, they concluded that school engagement is a multifaceted construct comprising affective, behavioural, and cognitive dimensions.
Like Libbey (2004) and Jimerson, Campos andGreif (2003), Fredericks, Blumenfeld andParis (2004) have highlighted the need to consolidate the various constructs used in the school engagement literature.They propose that engagement is a multifaceted construct comprising three components, i.e., behavioural, emotional or psychological, and cognitive; and is "…malleable, responsive to contextual factors, and amenable to environmental change" (p.59).Thus, school engagement is seen as reactive and may fl uctuate according to contexts, such as peer and teacher support or school climate.The researchers call for improvement in the measurement of school engagement to allow 'fi nely tuned interventions" (p.59).Drawing on the available literature, it is also observed that the emotional component has been referred to interchangeably as the affective (see Jimerson, 2003) and psychological component.

A CONCEPTUAL FRAMEWORK FOR SCHOOL ENGAGEMENT
In keeping with current student engagement literature, this study defi nes engagement as a multifaceted construct comprising three dimensions: behavioural, cognitive, and psychological.Each dimension will be discussed next, in relation to the constructs used in previous research endeavours.

Behavioural engagement
Behavioural engagement has been studied in terms of effort and time investment (Jordon & Nettles, 1999).Research in behavioural engagement often involves the following dimensions: positive conduct, involvement in learning and academic tasks, and participation in extra-curricular activities (Fredricks, Blumenfeld, & Paris, 2004).Among the variables that have been studied are: school and class attendance, tardiness, preparedness, homework completion, attentiveness to lessons, involvement in extra-curricular activities/sports/hobby, on-task behaviour, positive conduct, effort, persistence, question asking, contribution to class discussion, and participation in school-related activities.

Cognitive engagement
Cognitive engagement literature often examines student motivational goals and self-regulated learning (Boekarts, Pintrich, & Zeidner, 2000;Rosna, 2004a;Rosna, 2004b;Rosna, O'Neil & Hocevar, 2003).In this study, cognitive engagement refers to the extent of self-regulated learning as proposed by Snow (1992).Snow (1989Snow ( , 1992) ) broadened the concept of aptitude to include motivational and cognitive factors of individuals, not merely their cognitive abilities.He posited that both cognitive abilities and motivation conjointly contribute to effective functioning through two unique pathways: a performance pathway and a commitment pathway.The performance pathway describes the processes by which cognitive resources are activated, retrieved, assembled, and executed in order to accomplish a particular task.The commitment pathway describes a parallel process by which motivational resources are activated to guide and energize behaviour toward accomplishing a particular goal in a given context.The former is akin to cognitive engagement through selfregulated learning.Corno and Mandinach (1983) have argued that self-regulated learning is the highest form of cognitive engagement whereby individuals plan and manage their own learning and have a high degree of personal control and autonomy.

Psychological engagement
Psychological engagement refers to students' sense of identifi cation with school, sense of membership at school, and positive relationships with peers.Among the variables studied are: interest and values, fl ow, emotional reactions to school/ teachers, enjoyment (while doing a learning activity), affective reactions in the classroom (happiness, sadness, anxiety, and boredom), identifi cation with school, and feelings toward school (liking or disliking) (Goodenow, 1993a(Goodenow, , 1993b)).The extent to which adolescents identify with their school is found to be an important determinant of success in school.
Based on the available literature, we hypothesized an a priori structure that the School Engagement construct could be explained by three fi rst order factors (cognitive, behavioral, and psychological) and also one second-order factor.

OBJECTIVES OF THE STUDY
School engagement research has inspired a host of measures, such as effort, cognitive engagement, and help-seeking behaviours.However, the inconsistencies of these measures, as a result of the different operational defi nitions chosen by the various researchers, may ultimately hinder appropriate intervention to increase school engagement.Thus, there is a need to develop a more comprehensive instrument that can better capture the school engagement construct.The present study attempts to develop such an instrument by integrating behavioural, cognitive and psychological dimensions, as suggested in recent literature.It serves as a preliminary investigation of the psychometric properties of the Bahasa Malaysia version of the Multidimensional School Engagement Scale (MSES).
We examined the construct validity of this newly integrated MSES instrument.Specifi cally, the aims of the study were twofold: fi rstly, to assess the internal consistency reliability of the MSES dimensions and the total score, and secondly, to assess the construct validity of the MSES with secondary school students, utilizing exploratory and confi rmatory factor analytic procedures.The items and dimensions of MSES were developed and also adapted based on existing instruments that tapped the three dimensions of school engagement: behavioural, cognitive, and psychological.Apart from providing evidence of construct validity of the scale, the fi ndings would also inform school engagement researchers if a hierarchical model for this construct could be supported.

Participants
Participants were 2381 adolescents in Form Two and Form Four (8 th and 10 th graders, respectively) from 40 day schools in the state of Kedah.Males made up 47.0% (1119) of the respondents while females made up 53.0% (1262).As national-type secondary schools in Kedah are predominantly Malay-populated, the majority of respondents in the schools selected were Malays, who made up 81.6% (1942) of the total number of respondents, followed by 10.6% (252) Chinese, 6.1% (145) Indians and less than 2% (42) from other ethnic groups.Form Two students comprised 50.1% (1202) of the participants, while 49.9% (1189) were in Form Four.The mean age was 15.1 years (SD=1.01).

School Engagement
The Multidimensional School Engagement Scale was assessed with three subscales: behavioural engagement, psychological engagement, and cognitive engagement.All items were rated on a fi ve-point Likert scale: 1 = never, 2 = seldom, 3 = sometimes, 4 = frequently, 5 = always.Scoring of this scale was determined by summing the ratings within each scale and dividing this by the total number of items in that scale.Negatively-keyed items on the instrument were reverse-scored so that a high score on the instrument indicated a high degree of school engagement for the adolescents.
Psychological engagement assessed students' sense of identifi cation with school, sense of membership at the school, and positive relationships with peers.Eleven items were adapted from the Psychological Sense of School Membership scale (PSSM; Goodenow, 1993).A sample item is "I feel proud of belonging to this school." Cognitive engagement (11 items) assessed students' thinking about academic tasks, processing information, and self-directed learning.Cognitive engagement is subsumed under metacognition which is defi ned as the conscious and periodic self-checking of whether one's goal is achieved and, when necessary, selecting and applying different strategies, i.e., planning, monitoring and regulating one's own learning (Pintrich & DeGroot, 1990;Pintrich, Smith, Garcia, & McKeachie, 1993).A sample item is "If I get confused taking notes in class, I make sure I sort it out afterwards."

Procedures
Access to the participating schools was granted by the Educational Policy and Research division (EPRD) of the Ministry of Education Malaysia and the Kedah state education department.When consent was obtained from the respective schools, appointments were made to meet with the school principal and the student counselor.They were briefed on the objectives of the study and the procedures for data collection.To obtain maximum cooperation from the students, it was felt that the questionnaire was best administered by the student counselor.In most cases, participants assembled in the school hall, where the questionnaire was then administered in groups.Prior to responding, students listened to a set of standard directions which were read aloud by the counselor.Participants received a token of appreciation (a faculty note pad and a ball point pen) upon completing the task.

RESULTS
The study employed both exploratory and confi rmatory factor analyses procedures to examine the underlying hypothesized factor structure of the Multidimensional Student Engagement Scale (MSES).The individual items were used as measured or observed variables to defi ne its respective latent variables.

Reliability
Internal consistency was operationalized as Cronbach's alpha.Table 1 presents the results of the internal consistency reliability, and mean and standard deviation for the total score and each subscale.
Cronbach's alpha ranged from .75 ('Psychological' subscale) to .82('Cognitive' subscale) for the three dimensions.Thus, the MSES evidenced acceptable internal consistency in this study.

Exploratory Factor Analyses
Since this is the fi rst time we examined the psychometric properties of the Bahasa Malaysia adaptation of the multifaceted school engagement instrument, an exploratory factor analysis was carried out to examine the factorial validity of these newly reconceptualized constructs.We used exploratory factor analysis using maximum likelihood extraction with oblique rotation.Since the study was also testing hypothesis about the factors to be extracted, we set a priori criteria based on the number of factors extracted in previous studies.This method is useful and justifi ed when a researcher is testing a theory or replicating previous work such as in this study (Hair, Anderson, Tatham, & Black, 1998).
For the school engagement scale, examination of the eigenvalues and scree plot supported a three-factor model.The fi rst factor accounted for 13.58% of the variance, whereas the second and third factor accounted for 11.11% and 10.66% respectively.These factors represent cognitive, behavioural, and psychological engagement respectively.However, four items were dropped in the fi nal analysis.One item (i.e., "How often do you participate in class?") was deleted because it loaded on both cognitive and psychological engagement.Three additional items (i.e., "How often do you pay attention to class work?", "During class I often miss important points because I'm thinking of other things.",and "How often do you participate in extra-curricular activities?") were removed because they did not load on the factor they had been designed to represent.Therefore the twenty-nine remaining items represent general school engagement with three subscales.Table 2 displays the factor loading for the salient items (factor loading >.30), as well as eigenvalues, the percent of variance accounted for by each factor, and the Cronbach's alpha internal consistency reliability measure.

Item parceling
Following Floyd and Widaman's (1995) advice that "…factor structures are diffi cult to determine when the measured variables are individual items from a questionnaire especially if this means more than fi ve to eight items are free to load on each latent factor," (p.293) we used item parcels to conduct confi rmatory factor analysis procedures.Thus, in this study, item parcels (Marsh, Hau, Balla & Greyson, 1998;Rindskopf & Rose, 1988) were created based on randomly selected items of the same domains.An item parcel or testlet is derived by averaging the sum of several items that presumably measure a similar construct (Brown, 2006).Three parcels were created to represent each domain of the MSES, i.e., Behavioural, Cognitive, and Psychological engagement.In most cases, the item parcels comprised three measured items.Table 3 shows item parcel descriptions.FB2 When reading for this course, I make up questions to help focus my reading. .64 FB3 When I become confused about something I'm reading for this class, I go back and try to fi gure it out.

Item
FB9 When studying for this course I try to determine which concept I don't understand.
.68 FB10 When I study for this class, I set goals for myself in order to direct my activities in each study period.
.66   : Steiger & Lind, 1980).Incremental fi t statistics were also chosen for their ability to evaluate different aspects of model fi t.
The two incremental fi t statistics chosen were the Tucker Lewis Index (TLI: Tucker & Lewis, 1973) and the Comparative Fit Index (CFI: Bentler, 1989).For the chi-square tests, a signifi cant value relative to the degrees of freedom indicates that the model does not adequately fi t the data.Thus, a good fi tting model is indicated by non-signifi cant results from these tests.However, it has been noted that chi-square tests are heavily infl uenced by sample size and are easy to reject as the sample size increases (Tanaka, 1993).Since the chi-square is infl ated by sample size, we report the model's χ 2 /df ratio merely for informative purposes.The NFI, TLI and CFI vary along a 0 to 1 continuum.Values greater than .90and .95refl ected an acceptable and excellent fi t to the data, respectively.Finally, the RMSEA values at or less than .05and .08refl ect a close and reasonable fi t respectively (Schumacker & Lomax, 1996).Others consider RMSEA values of .05 or less to indicate a "good" fi tting model (Browne & Cudeck, 1993;Rigdon, 1996).The RMSEA provides answers to "How well would the model, with unknown but optimally chosen parameter values, fi t the population covariance matrix if it were available" (Browne & Cudeck, 1993, pp.137-138).
The Chi-Square Goodness-of-fi t statistic with 41 degrees of freedom was 230.58 (p<.001).The Normed fi t Index (NFI), Tucker Lewis Index (TLI), Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), and the Root Mean Square Error of Approximation (RMSEA) were .97,.97,.98 and .04,respectively.As seen in Figure 1, all loadings of items on their targeted factors were high, statistically signifi cant, and nearing or above the 0.40 cutoff values typically used in exploratory factor analyses.Correlations among factors ranged from .30(cognitive and behavioural engagement) to .44 (cognitive and psychological engagement).Thus, it suggests that although the factors are interrelated and might form a general factor, each subscale is relatively independent of the other scales.

Test of a second order factor
A hierarchical factor structure was also hypothesized and tested.Results of the hypothesized second-order factorial structure are presented in Figure 2. Model fi t statistics comparing the two models are presented in Table 4.Both models met the criteria for good fi tting models although the chi-square tests rejected both models due to large sample size.The path coeffi cients for each MSES dimension in the hierarchical model were .57,.53,.77for Cognitive, Behavioural and Psychological, respectively.To the researchers' knowledge this is the fi rst study in school engagement literature that has examined a second order latent factor.The second-order factor reproduced results similar to the earlier fi rst-order factor.the instrument studied.All coeffi cient alphas were equal to or more than .70 for all dimensions of school engagement.The exploratory factor analysis demonstrated support for the factorial validity of the MSES.Additionally, confi rmatory factor analyses to provide tests of signifi cance regarding item loadings and disattenuated correlations, as well as assessment of the overall model fi t, yielded evidence of construct validity.Practically, the three subscales of the MSES have proven to be valuable instruments for studying school engagement among adolescents in Malaysia.Additionally, the data also provided support for a hierarchical model.
One major contribution of this study is the development of a more comprehensive school engagement measure.The confi rmatory factor analysis results suggest that each school engagement dimension can be used independently to measure the specifi c school engagement dimension or together as the composite score to describe adolescent school engagement in general.The multifaceted school engagement construct is an answer to the current debate in school engagement literature (see Fredricks, Blumenfeld, & Paris, 2004;Libbey, 2004).This instrument can be used by Malaysian researchers interested in examining adolescent social psychological factors.Each scale and its subscales can be used singly or together depending on the nature of the study.Taken together, these analyses suggest that the MSES reliably measures the constructs it was designed to measure.Generally, the psychometric properties of the instrument are very acceptable as most of the constructs equal or exceed recommended measurement levels.
readings are diffi cult to understand, I change the way I read the material .60FB5 Before I study new course material thoroughly, I often skim it to see how it is organized..57FB6 I ask myself questions to make sure I understand the material I have been studying in this class..68FB7 I try to change the way I study in order to fi t the course requirements and the instructor's teaching style..63FB8 I try to think through a topic and decide what I am supposed to learn from it rather than just reading it over when studying for this course.

Table 1 .
Summary Statistics for School Engagement Scale and Cronbach's Alpha

Table 2 .
Factor Loadings for School Engagement

Table 2 .
(continued)established by examining a combination of absolute and incremental fi t statistics.Absolute fi t statistics used in this study included the traditional chi-square test of exact model fi t, the chi-square/degree of freedom ratio (χ 2 /df), and the root mean square error of approximation (RMSEA

Table 4 .
Model Fit Statistics for Each Hypothesized Factor ModelCONCLUSIONFindings showed evidence of construct validity of the instruments, thus supported the utility of the instrument for local studies.The present results also suggest acceptable reliability and validity of