Evaluation Of Task And Contextual Performance: A Multitrait-Multimethod Approach

Task and Contextual Performance

According to Borman and Motowidlo (1997), there are two types of employee job performance behavior. These are role performance and extra-role performance. At the same time, role performance and extra-role performance are also called task performance and contextual performance respectively. Task performance expresses duties and responsibilities of a job which makes it different from others (Jawahar & Carr, 2006). Being related to expertise and mechanics of the job (Borman, 2004), task performance focuses on basic technical details in the job and means artifice of the task required to carry out a job successfully (Van Scotter & Motowidlo, 1996). Despite being mostly focused on task performance to reach organizational goals, in the long run researchers have realized that various activities contributing to actualization of organizational goals exist. Borman and Motowidlo (1993) called these activities which support implementation of the organizational activities successfully contextual performance.

Self-Ratings and Manager-Ratings of Performance

How and by whom the performance evaluation is made is intensely discussed in the literature. According to a variety of studies, people find it hard to analyze themselves objectively enough and to give accurate information about themselves (DeNisi & Shaw, 1977; Levine, Flory, & Ash, 1977). People tend to overvalue their own performances (Thornton, 1980). Some studies on this effect have revealed that this tendency has different aspects in western and eastern cultures. In eastern cultures, individuals are observed to rate their own performances lower than their superiors and this was called “humility effect”. In western cultures, the results show just the opposite (Farh, Dobbins, & BarSchiuan, 1991). In contrast to varying results in this study and this type of studies, Farh and Webel (1986) have observed that self-enhancement bias decrease in individual’s self-evaluation once he is informed that an objective performance criterion is used for comparison. Within the scope of this study, self-enhancement bias is believed to decrease since individuals know that information will be gathered through another sources as well and there will be a comparison. Other than that, studies in which self-evaluations are compared with different sources provide more accurate information about the validity of self-evaluation. When compared to performance in the past and peer-ratings and various psychological tests, self-rating is stated to be as much valid as these methods (Shrauger & Osberg, 1981). On the other hand, DeNisi and Shaw (1977), in their research with 144 university students, evaluated students’ skills via two methods (self-evaluation and skills tests). They have found out there is a low correlation between the results of self-evaluation and skills tests, and reported that self-evaluation method cannot replace skills tests. People’s ability to evaluate themselves, is subject to a great variety of factors which can affect the accuracy of evaluation such as intelligence, high success status, internal locus of control and self-enhancement bias. According to Thornton (1980), individuals have quite different opinions about their own job performances than people around them have. Conway and Huffcutt (1997) have concluded that different sources (individual, peer, supervisor) have quite different perspectives of performance and agreement within sources (peer and peer) is more notable than the agreement between sources (peer and self). In studies on task oriented teams whose members’ performances are evaluated by themselves, their supervisors, peers and a consultant who is a part of the team, Furnham and Stringfield (1998), on the other hand, have found out rater agreement is higher in specific and observable behaviors and lower in cognitive dimensions of performance. They have argued that evaluation of performance by a supervisor would be more effective than by a peer. The reason for this is supervisors are better educated to evaluate employees; from this point of view they are more reliable and less biased raters.

The Multitrait-Multimethod Model

In social sciences, there is known to be a number of measurement tools which measure a trait. In theory, the same trait which is measured by different methods is supposed to be correlated with one another. One of the best ways to carry out valid measurements for any observed variant is to use different methods to discover different traits. For this purpose, Campbell and Fiske (1959) have put forth multitrait-multimethod model (MTMM). This approach includes use of multiple methods and traits. Both convergent validity and discriminant validity studies are made in structure validity studies with MTMM and these studies show effect of applied methods (Campell & Fiske,1959). Convergent validity is put into perspective when there is one trait and two methods while discriminant validity is put into perspective with multiple traits and a single method (Höfling et al., 2009). Different structures are defined with the same methods, afterwards measurements for each structure are obtained with each method and their correlation with each other is calculated. Final correlation coefficients are defined as being one of either convergent validity coefficient or discriminant validity coefficient. Correlation between the measurements of the same structure is observed by using different methods in convergent validity coefficient and correlation is expected to be high. On the other hand, in discriminant validity coefficient, correlation between different structures is observed by using the same measurement method or different methods; correlation is expected to be lower than convergent validity coefficient (Campell & Fiske,1959).

Research Goal

Objective of this study is to analyze contextual and task performance evaluations by self-ratings and supervisor ratings via multitrait-multimethod analyses.

Research Hypotheses

H1: After controlling for the method factors, trait dimensions of performance which are named as the task and contextual performance can be differentiated.

H2: The measurement of task and contextual performance will be invariant across self-ratings and supervisor-ratings.

Sample and Data Collection

486 medical personnel and 61 supervisors who work for public hospitals in İstanbul constitute the sample of the study. Questionnaires were handed over and collected by the surveyor. Medical staff and staff in charge of the services in the hospitals were interviewed in person, were informed about the study, and how to fill the questionnaire was explained to them elaborately. Afterwards names and questionnaire numbers of every volunteer who accepted to participate in the study from each service were noted down on a small notepaper and mentioned paper was handed over to the superior. The supervisor wrote down the questionnaire number of whichever subordinate he would evaluate on the questionnaire form and destroyed this paper. By this way, data of the supervisor and subordinate were combined. In order to maintain confidentiality of the information, sealed envelopes were handed over to both the supervisor and the subordinates, and participants submitted the questionnaire form to the surveyor in a closed envelope. SPSS 23 and AMOS 23 were used for analysis of the data.

Measures

The scale which was developed to measure job performance by Goodman and Svyantek (1999) was used in the study. While subordinates evaluated their own performance through this scale, supervisors evaluated their subordinates’ performances through the same scale. Originally first 16 points of this scale constitute contextual performance while the last 9 constitute task performance. Contextual performance constitutes of two sub dimensions. However, only “altruism” sub dimension comprising of 7 points was used as part of this study. Here are a couple of points from the scale: “Helps other employees with their work when they have been absent.” and “Takes initiative to orient new employees to the department even though it is not a part of his/her job description.” On the other hand, task performance does not have any sub dimensions. “Achieves the objectives of the job.” and “Performs well in the overall job by carrying out tasks as expected.” are examples of the points in this scale. Likert Scale, which is originally a 7-point scale, was transformed into a 6-point scale as part of this study (1= Strongly Disagree; 6= Strongly Agree). Lower points from the scale show lower performance level while higher points show higher performance level.

Results

Firstly, we ran preliminary CFAs to test the factor structure of task and contextual performance separately for self-ratings and supervisor-ratings. We examined six models. These are:

• One-factor model where all task and contextual performance items loaded on a general performance factors for self-ratings and supervisor-ratings (Model 1 and Model 4).

• Two-factor model which included one task performance latent factor with the respective task performance items loading on this factor, and one contextual performance latent factor with the respective contextual performance items loading on this factor for self-ratings and supervisor-ratings (Model 2 and Model 5).

• A modified two-factor model where some changes were implied based on the modification indices for self-ratings and peer-ratings (Model 3 and Model 6).

These results are presented in Table 1 .

Table 1 presents that the proposed two-factor model fits the data better than the one-factor model both for self-ratings and for supervisor-ratings. However, some fit indices did not satisfy the criteria for good fit for the two-factor model for both methods. Therefore, we examined the modification indices for potential cross-loadings. Then, 2^th, 4^th, 6th, 7^th, 13^th, 14^th and 16^th items were excluded from analyses and these two factor modified models (M3 and M6) were used in the remaining of the paper.

After testing the factor structure of task and contextual performance separately for self-ratings and supervisor-ratings, we examined descriptive statistics, reliabilities, and correlations between the research variables.

Table 2 shows that descriptive statistics, reliabilities, and correlations between the research variables. All scales showed good reliabilities with Cronbach’s alphas values varying between 0,76 and 0,95. Self-ratings of contextual performance were significantly correlated with supervisor-ratings of contextual performance (r = .173, p< .01). Similarly, the correlation between self-ratings and supervisor-ratings of task performance was positive and significant (r = .220, p < .01). Also, Table 2 shows that supervisor-ratings of contextual and task performance correlate higher (r = .808, p < .01) than self-ratings of the same dimensions (r = .643, p< .01).

Secondly, we examined five models. These are:

• M1: All items loaded on a single performance latent factor.

• M2: This model included two correlated latent factors which were named as task and contextual performance. There was no differentiation between two measurement methods (self and supervisor ratings) for this model.

• M3: This model included two correlated latent factors which were named as self-ratings and supervisor-ratings performance. There was no differentiation between two performance dimensions (task and contextual performance) for this model.

• M4: This model included four correlated latent factors. Task performance items as rated by employees loaded on a self-ratings task performance latent factor, contextual performance items as rated by employees loaded on a self-ratings contextual performance latent factor, task performance items as rated by supervisors loaded on a supervisor-ratings task performance latent factor and contextual performance items as rated by supervisors loaded on a supervisor-ratings contextual performance latent factor.

• M5: MTMM Model. This model is shown in Figure 1 .

Table 3 shows that four-factor model and the MTMM model fit well to the data, ∆X² (22) = 40,664, p<0,01. Moreover, the estimated correlation between the trait factors in the MTMM model was Ҩ = .42 (p = .015). According to Höfling et al., 2009; if the estimated correlation between the traits in the MTMM model is moderate to low, it will demonstrate discriminant validity. Besides, the estimated correlation between the traits in the MTMM model (Ҩ = .42) is lower than their correlations in the four-factor model (Ҩ = .73 for self-ratings and Ҩ = .83 for supervisor-ratings). So, these findings support the discriminant validity in our research.

See Full Size >

In order to estimate the variance explained by each factor, we calculated the communality (R-square) of the item loadings in the MTMM model:

• The model explains a total of 14%–70% (M = 38%) of the variance in task performance items and a total of 12%–54% (M = 35%) of the variance in contextual performance items.

• For self-rated contextual performance items the trait factor explains an average of 17% of the variance and 71% of the variance for the method factor.

• For self-rated task performance items, the trait factor explains an average of 56% of the variance in items ratings, versus 50% for the method factor.

• For supervisor-rated contextual performance items the trait factor explains an average of 53% of the variance and 67% of the variance for the method factor.

• For supervisor-rated task performance items the trait factor explains an average of 19% for the trait factor versus 86% for the method factor.

The measurement of task and contextual performance was found to be invariant across raters, that is employees and their supervisors. However, there is more method variance in the supervisor-ratings task performance and in the self-ratings of contextual performance (Table 4 ).