Using Employee Perceptions for Organizational Performance Assessment

Abstract

The aim of this paper is to compare perceptual, objective and joint organizational performance measurement outcomes of nine Turkish public hospitals and provide empirical support for using perceptions of employees other than top-level managers on organizational performance. The data consisting of perceptual measures of hospital performance was collected by a survey instrument which was developed based on the dimensions of the performance assessment tool used in Turkish public hospitals. Objective performance scores were obtained from the hospitals’ annual performance assessment reports. A joint metric was then obtained by combining the perceptual measures and the objective measures of the hospital performance and comparisons were made using One-Way ANOVA. The results indicated that perceptual performance did not statistically differ between pairs of physicians, nurses and midwives, clinical support staff and administrative support staff. However, all of the four professional groups had a significantly different perception of performance compared to managers. Moreover, when the hospital groups were ranked based on their mean scores of perceptual performance, joint performance and objective performance the ranking did not change. Having similar results for different groups of hospitals in perceptual, objective and joint performance measures imply that using perceptions of employees from different departments of the hospital on organizational performance could indeed be a close proxy for objective measures.

Keywords: Employee perceptionsorganizational performancehospitalshealth care

Introduction

Organizational performance is one of the most important areas of research in strategic management field and researchers frequently face challenges in obtaining performance data. Often times, required performance data is not available publicly (Dess & Robinson, 1984). As a result, using perceptual measures of performance in the absence of objective measures has become common practice. When doing so, typically top level and/or mid-level managers are the main constituents and employees from other levels are not included in the studies. Hasson, Tafvelin, and von Thiele Schwarz (2013) argued that it was important to include perceptions of employees other than the top-level managers or human resource department ratings on organizational learning research. It was stated that employees from different professional groups could offer valuable information on organizational learning (Hasson et al., 2013). In similar vein, inclusion of employees’ perception other than the top-level managers on organizational performance could provide valuable insight. We argue that it may lead to biased results to include only managers as constituents in measuring organizational performance and other professionals from different levels of the organization should be included in order to obtain a more balanced and accurate measure.

The study aims to provide empirical support for the employment of perceptual measures and inclusion of different types of professional groups, not only managers for the measurement of organizational performance (OP) and proposes using a joint performance metric which comprises of objective measures as well as subjective to have a more balanced performance measurement framework.

Literature Review

Determining and understanding the sources of high organizational performance (OP) and sustainable competitive advantage is one of the most important research areas of strategic management field (Barney, 1991; Porter, 1985; Rumelt, 1984). It is seen in the organizational performance literature that there is a disagreement regarding the definition of organizational performance. Richard, Devinney, Yip, and Johnson (2009), argue that many researchers are not clear about the difference between organizational performance and organizational effectiveness which is a much broader concept. Organizational performance is defined mainly depending on financial indicators such as “financial performance (profits, return on assets, etc.), product market performance (sales, market share etc.), and shareholder return (total shareholder return, economic value added etc.). Organizational effectiveness on the other hand is considered to be “a broader concept capturing organizational performance as well as many other internal performance outcomes associated with more efficient or effective operations and other external measures that relate to considerations that are broader than those simply associated with economic valuation” (Richard et al., 2009, p. 725). On the other hand, one of the most popular organizational performance measurement tools—the Balanced Score Card (BSC)—was developed from the notion that reliance only on financial measures was not enough for managing complex organizations (Walker & Dunn, 2006). Kaplan and Norton’s (1992) BSC framework for measuring organizational performance captured areas related to customers, internal processes, learning and growth as well as financial. Organizational performance measurement in this study is a multidimensional construct and is based on the BSC framework which encompasses many different areas of the organization not only financial.

The studies in the literature of organizational performance reveal that it is a multidimensional construct (Combs, Crook, & Shook 2015). However, the dimensions identified vary depending on the method of analysis, data source and measures used in the study. In the OP literature it is possible to observe many different types of subjective and objective performance measures. There are varying views in the literature regarding the use of perceptual/subjective measures. By some researchers, objective measures of OP are considered to be more robust compared to subjective OP measures. However, even though some researchers question the credibility of subjective performance measures, mainly due to access restrictions of objective performance related data, subjective measures are frequently applied and in fact are used more often than objective measures (Singh, Darwish, & Potocnik, 2016). It is argued that managers may not be enthusiastic about revealing their organization’s weaknesses and consequently may overstate the performance of their organizations (Singh et al., 2016). Similarly, Meier and O’Toole (2012) argue that perceptual measures can lead to spurious results. They compared perceptual and objective measures of organizational performance and found common method bias to be an important problem when managers’ perceptions are used for performance measurement. Venkatraman and Ramanujam (1987) on the other hand, showed that managers are not as biased as they are believed to be and that performance data obtained via managers can be used as an acceptable measure of OP.

Dess and Robinson (1984) compared objective measures of performance with subjective measures. The participants in their study were only top managers representing their organizations. They concluded that subjective performance measures could be used as a substitute for objective measures when the latter was not accessible. In their research, McCracken, McIIwain, and Fottler (2001) compared the perceptions of hospital executives to the objective financial performance data. They found that return on assets and operating margin are the most valid subjective financial measures of hospital performance. Some researches on OP use both objective and financial measures and report that both measures are equally reliable and valid (Singh et al., 2016). Hult et al. (2008) investigated research papers published in top journals to assess OP measures. They found that subjective (primary) objective measures (secondary) are both valid and reliable measures of OP. Wall et al. (2004) compared subjective and objective performance measures in their study. Their study provides evidence on convergent, discriminant and construct validity of subjective and objective performance measures. They conclude that subjective measures of OP do not lead to overestimation. Harris (2001) performed a study on the managers of European hotel properties and found significant correlation between the subjective and objective measures of OP based on return on investment and sales growth.

Moullin (2004) on the other hand states that “delivering excellent health services requires a high standard of performance on a wide range of factors such as clinical care, patient satisfaction, short waiting times, cost control and learning from best practice elsewhere. Hence, he emphasizes that all areas must be considered when measuring performance and achieving a balance between subjective and objective measures” is crucial.

Research Method

The aim of the study is fourfold. First is to investigate if there is a statistical difference on the perception of hospital performance between different groups of employees. Second is to determine if there is a statistical difference on the perception of hospital performance among different hospital groups. Third is to compare the perceptual performance results and the objective performance results. Lastly, to use a ‘joint metric’ which consists of perceptual performance and objective performance to investigate whether there is a statistical difference on the perception of hospital performance between different groups of employees and different hospitals. The ‘joint metric’ is obtained by combining the perceptual measures and the objective measures of the hospital performance. A weighted score from hospitals’ overall objective performance score was calculated for each hospital which was multiplied by the survey responses to obtain a ‘joint metric’ of performance.

One-way ANOVA Analysis (Analysis of Variance) was used to analyze the data. To test the reliability and the validity of the survey instrument exploratory factor analysis (EFA) was used. IBM SPSS Statistics Version 25 was used for data analysis.

Data Analysis and Results

In the EFA the factor structure is determined based on the correlations among the variables in the data set. Principal components method of extraction and varimax rotation was performed on the survey items. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.907 indicating that the strength of relationship among the variables was high. Bartlett’s Test of Sphericity is another measure of sampling adequacy and tests the overall significance of the correlations among the items in the construct. The Bartlett’s test of sphericity was significant (chi-square= 1661.170, p<0.001) confirming sampling adequacy for the factor analysis. The factors loaded in a single factor and the loadings were above 0.5. The factor explained 63.28% of the total variance. Reliability analysis was performed to ensure consistency among multiple measurements of the variables. Cronbach’s alpha score of 0.899 was obtained indicating sufficient construct reliability. Commonly, values above 0.7 are considered satisfactory (Nunnally & Bernstein, 1994). The results of the EFA are shown in Table 01 .

414 participants were divided into five groups based on their professions: 1) physicians, 2) nurses and midwives, 3) clinical support staff (laboratory personnel, pharmacists, physiotherapists etc.), 4) managers, and 5) administrative support staff. The participating hospitals were also split into three groups depending whether they are 1) Specialty hospitals 3) Teaching and research hospitals or 3) State hospitals.

A One-way analysis of Variance (ANOVA) was used to investigate three questions:

• Whether physicians, nurses and midwives, clinical support staff, managers and administrative support staff have different perceptions of hospital performance. The independent variable is the type of profession while the dependent variable is the perceptual performance.

• Whether perceptual hospital performance is different among specialty hospital, teaching and research hospitals and state hospitals. The independent variable is the type of hospital while the dependent variable is the perceptual performance.

• Whether joint hospital performance is different among specialty hospital, teaching and research hospitals and state hospitals.

Discussion and Conclusion

Results of the data analyses revealed that perceptions of physicians, nurses and midwives, clinical support staff and administrative support staff regarding hospital performance did not significantly differ with each other. However, all the aforementioned professional group’s perception of hospital performance differed with the managers. Considering the frequent approach of utilizing managers’ perception to obtain organizational performance data this could have some important consequences. The main justification behind using managers’ knowledge as a proxy for organizational performance is that they will have a more thorough view of the organization and they have first-hand exposure to the organization’s financial operations. The findings of the study revealed that basing organizational performance measurement only on mangers’ perception could possibly lead to an upward bias. All the professional groups in the study had lower and significantly different perception of the hospital performance. The results imply that in order to obtain a more balanced measure of organizational performance different professional groups should be included in the study.

The ANOVA conducted for perceptual performance by type of hospital indicated that there was significant difference in perception of performance between different types of hospitals. Similar results were obtained when the analysis was repeated for ‘joint performance’. The results imply the importance of including different types of hospitals in a study when conducting a research since organizational structure could be an important determinant in organizational performance. Moreover, when the hospital groups were ranked based on their mean scores of perceptual performance, joint performance and objective performance the ranking did not change (see Table 05 ). Specialty hospitals had the highest performance scores in perceptual, joint and objective measures. State hospitals ranked second while teaching and research hospitals ranked third in each measure. Having similar results in perceptual, objective and joint performance measures may imply that using perceptions of employees from different departments of the hospital on organizational performance could indeed be a close proxy for objective measures.

Managers typically have a general knowledge of the operations which is undoubtedly crucial. However, it is possible that a general view can sometimes cause some important details to be missed. Clinicians and other medical staff may not have accurate information on financial performance of the hospital while they are expected to have a more accurate perception of the health services performance. So, it may be wise to obtain financial indicators from the managers while for the procurement of other performance data, other professional groups could be utilized in an effort to obtain more balanced and comprehensive performance data. The survey instrument used in this study to measure perceptual organizational performance was developed based on the general framework of the objective organizational assessment tool utilized by the hospitals in the study in order get a general perceptual view of performance. Future studies may consider a more detailed framework to get a more robust measurement metric. Hence, when it is not possible to obtain objective performance measures perceptual measures could be used instead. Moreover, when possible, conducting the research using both objective and perceptual measures should be preferred in order to obtain a more comprehensive and accurate view of organizational performance.

References

1. Barney, J. (1991). Firm Resources and Sustained Competitive Advantage. Journal of Management, 17(1), 99-120.
2. Combs, J. G., Crook, R. T., & Shook, C. L. (2015). The Dimensionality of Organizational Performance and Its Implications for Strategic Management Research. Research Methodology in Strategic Management, 2, 259-286.
3. Dess, G. G., & Robinson R. B. (1984). Measuring organizational performance in the absence of objective measures: the case of the privately-held firm and conglomerate business unit. Strategic Management Journal, 5(3), 265-273.
4. Harris, L. C. (2001). Market orientation and performance: objective and subjective empirical evidence from UK companies. Journal of Management Studies, 38, 17-43.
5. Hasson, H., Tafvelin, S., & von Thiele Schwarz, U. (2013). Comparing Employees and Managers’ Perceptions of Organizational Learning, Health and Work Performance. Advances in Developing Human Resources 15(2) 163-176.
6. Hult, G. T. M., Ketchen Jr., D. J., Griffith, D. A., Chabowski, B. R., Hamman, M. K., Dykes, B. J., … & Cavusgil, S. T. (2008). An assessment of the measurement of performance in the international business research. Journal of International Business Studies, 39(6), 1064-1080.
7. Kaplan, B. S., & Norton, D. P. (1992). The balanced scorecard: Measures that drive performance. Harvard Business Review 70(1), 71-79.
8. McCracken, M. J., McIlwin, T. F., & Fotter, M. D. (2001). Measuring organizational performance in the hospital industry: an exploratory comparison of objective and subjective methods. Health Services Management Research, 14, 211-219.
9. Meier, K. J., & O’Toole Jr., L. J. (2012). Subjective Organizational Performance and Measurement Error: Common Source Bias and Spurious Relationships. Journal of Public Administration Research and Theory, 23, 429-456.
10. Moullin, M. (2004). Eight essentials of performance measurement. International Journal of Health Care Quality Assurance, 17(3).
11. Nunnally, J. C., & Bernstein, I. H. (1994). The Assessment of Reliability. Psychometric Theory, 3, 248-292.
12. Porter, M. E. (1985). Competitive Advantage. Creating and Sustaining Superior Performance. New York, Free Press.
13. Richard, P. J., Devinney, T. M., Yip, G. S., & Johnson, G. (2009). Measuring Organizational Performance: Towards Methodological Best Practice. Journal of Management, 35(3), 718-804.
14. Rumelt, R. P. (1984). Towards a Strategic Theory of the Firm. In R. Lamb (ed.). Competitive Strategic Management, Englewood Cliffs NJ, Prentice-Hall, 556-570.
15. Singh, S., Darwish, T. K., & Potocnik, K. (2016). Measuring organizational performance: a case for subjective measures. British Journal of Management, 27, 214-224.
16. Venkatraman, N., & Ramanujam, V. (1987). Measurement of Business Economic Performance: An Examination of Method Convergence. Journal of Management, 13(1), 109-122.
17. Walker, K. B., & Dunn, L. M. (2006). Improving Hospital Performance and Productivity with Balanced Scorecard. AHCMJ, 2, 85-110.
18. Wall, T. Michie, J., Patterson, M., Wood, S., Sheehan, M., Clegg, C., & West, M. (2004). On the validity of subjective measures of company performance. Personnel Psychology, 57, 95-118.