Using Employee Perceptions for Organizational Performance Assessment

Abstract

The aim of this paper is to compare perceptual, objective and joint organizational performance measurement outcomes of nine Turkish public hospitals and provide empirical support for using perceptions of employees other than top-level managers on organizational performance. The data consisting of perceptual measures of hospital performance was collected by a survey instrument which was developed based on the dimensions of the performance assessment tool used in Turkish public hospitals. Objective performance scores were obtained from the hospitals’ annual performance assessment reports. A joint metric was then obtained by combining the perceptual measures and the objective measures of the hospital performance and comparisons were made using One-Way ANOVA. The results indicated that perceptual performance did not statistically differ between pairs of physicians, nurses and midwives, clinical support staff and administrative support staff. However, all of the four professional groups had a significantly different perception of performance compared to managers. Moreover, when the hospital groups were ranked based on their mean scores of perceptual performance, joint performance and objective performance the ranking did not change. Having similar results for different groups of hospitals in perceptual, objective and joint performance measures imply that using perceptions of employees from different departments of the hospital on organizational performance could indeed be a close proxy for objective measures.

Keywords: Employee perceptionsorganizational performancehospitalshealth care

Introduction

Organizational performance is one of the most important areas of research in strategic management field and researchers frequently face challenges in obtaining performance data. Often times, required performance data is not available publicly (Dess & Robinson, 1984). As a result, using perceptual measures of performance in the absence of objective measures has become common practice. When doing so, typically top level and/or mid-level managers are the main constituents and employees from other levels are not included in the studies. Hasson, Tafvelin, and von Thiele Schwarz (2013) argued that it was important to include perceptions of employees other than the top-level managers or human resource department ratings on organizational learning research. It was stated that employees from different professional groups could offer valuable information on organizational learning (Hasson et al., 2013). In similar vein, inclusion of employees’ perception other than the top-level managers on organizational performance could provide valuable insight. We argue that it may lead to biased results to include only managers as constituents in measuring organizational performance and other professionals from different levels of the organization should be included in order to obtain a more balanced and accurate measure.

The study aims to provide empirical support for the employment of perceptual measures and inclusion of different types of professional groups, not only managers for the measurement of organizational performance (OP) and proposes using a joint performance metric which comprises of objective measures as well as subjective to have a more balanced performance measurement framework.

Literature Review

Determining and understanding the sources of high organizational performance (OP) and sustainable competitive advantage is one of the most important research areas of strategic management field (Barney, 1991; Porter, 1985; Rumelt, 1984). It is seen in the organizational performance literature that there is a disagreement regarding the definition of organizational performance. Richard, Devinney, Yip, and Johnson (2009), argue that many researchers are not clear about the difference between organizational performance and organizational effectiveness which is a much broader concept. Organizational performance is defined mainly depending on financial indicators such as “financial performance (profits, return on assets, etc.), product market performance (sales, market share etc.), and shareholder return (total shareholder return, economic value added etc.). Organizational effectiveness on the other hand is considered to be “a broader concept capturing organizational performance as well as many other internal performance outcomes associated with more efficient or effective operations and other external measures that relate to considerations that are broader than those simply associated with economic valuation” (Richard et al., 2009, p. 725). On the other hand, one of the most popular organizational performance measurement tools—the Balanced Score Card (BSC)—was developed from the notion that reliance only on financial measures was not enough for managing complex organizations (Walker & Dunn, 2006). Kaplan and Norton’s (1992) BSC framework for measuring organizational performance captured areas related to customers, internal processes, learning and growth as well as financial. Organizational performance measurement in this study is a multidimensional construct and is based on the BSC framework which encompasses many different areas of the organization not only financial.

The studies in the literature of organizational performance reveal that it is a multidimensional construct (Combs, Crook, & Shook 2015). However, the dimensions identified vary depending on the method of analysis, data source and measures used in the study. In the OP literature it is possible to observe many different types of subjective and objective performance measures. There are varying views in the literature regarding the use of perceptual/subjective measures. By some researchers, objective measures of OP are considered to be more robust compared to subjective OP measures. However, even though some researchers question the credibility of subjective performance measures, mainly due to access restrictions of objective performance related data, subjective measures are frequently applied and in fact are used more often than objective measures (Singh, Darwish, & Potocnik, 2016). It is argued that managers may not be enthusiastic about revealing their organization’s weaknesses and consequently may overstate the performance of their organizations (Singh et al., 2016). Similarly, Meier and O’Toole (2012) argue that perceptual measures can lead to spurious results. They compared perceptual and objective measures of organizational performance and found common method bias to be an important problem when managers’ perceptions are used for performance measurement. Venkatraman and Ramanujam (1987) on the other hand, showed that managers are not as biased as they are believed to be and that performance data obtained via managers can be used as an acceptable measure of OP.

Dess and Robinson (1984) compared objective measures of performance with subjective measures. The participants in their study were only top managers representing their organizations. They concluded that subjective performance measures could be used as a substitute for objective measures when the latter was not accessible. In their research, McCracken, McIIwain, and Fottler (2001) compared the perceptions of hospital executives to the objective financial performance data. They found that return on assets and operating margin are the most valid subjective financial measures of hospital performance. Some researches on OP use both objective and financial measures and report that both measures are equally reliable and valid (Singh et al., 2016). Hult et al. (2008) investigated research papers published in top journals to assess OP measures. They found that subjective (primary) objective measures (secondary) are both valid and reliable measures of OP. Wall et al. (2004) compared subjective and objective performance measures in their study. Their study provides evidence on convergent, discriminant and construct validity of subjective and objective performance measures. They conclude that subjective measures of OP do not lead to overestimation. Harris (2001) performed a study on the managers of European hotel properties and found significant correlation between the subjective and objective measures of OP based on return on investment and sales growth.

Moullin (2004) on the other hand states that “delivering excellent health services requires a high standard of performance on a wide range of factors such as clinical care, patient satisfaction, short waiting times, cost control and learning from best practice elsewhere. Hence, he emphasizes that all areas must be considered when measuring performance and achieving a balance between subjective and objective measures” is crucial.

Research Method

The aim of the study is fourfold. First is to investigate if there is a statistical difference on the perception of hospital performance between different groups of employees. Second is to determine if there is a statistical difference on the perception of hospital performance among different hospital groups. Third is to compare the perceptual performance results and the objective performance results. Lastly, to use a ‘joint metric’ which consists of perceptual performance and objective performance to investigate whether there is a statistical difference on the perception of hospital performance between different groups of employees and different hospitals. The ‘joint metric’ is obtained by combining the perceptual measures and the objective measures of the hospital performance. A weighted score from hospitals’ overall objective performance score was calculated for each hospital which was multiplied by the survey responses to obtain a ‘joint metric’ of performance.

One-way ANOVA Analysis (Analysis of Variance) was used to analyze the data. To test the reliability and the validity of the survey instrument exploratory factor analysis (EFA) was used. IBM SPSS Statistics Version 25 was used for data analysis.

Sample and Data Collection

The study employs two different methods of data collection. The data consisting of perceptual measures of hospital performance was collected by a survey instrument. The objective measures of the hospital performance were obtained from the hospitals’ annual performance reports covering the years 2015 and 2016. The study was carried out in 9 public hospitals operating in Istanbul metropolitan area. The survey data was collected over a period of 5 months between October 2016-February 2017. In order to get a more comprehensive and balanced view, all of the hospital departments were included in the study and the participants were asked to provide their opinions on hospital performance. However, considering the fact that hospital performance is a concept not all employees may be sufficiently knowledgeable about, only employees who are in charge of their particular departments were selected as participants. In Turkish public hospitals most departments have an employee in charge—responsible (chief) personnel. The study’s sample consists of the 414 chief personnel from the population of 917 chief personnel in these 9 hospitals. During the course of the research a total 623 surveys were distributed. 431 surveys were returned resulting in a response rate of 69%. After the elimination of 17 surveys due to missing and unengaged responses and improper respondents the final sample size consisted of 414 chief personnel with a response rate of 66%. The names of the hospitals were kept confidential due to privacy concerns.

Measurement of variables

The performances of Turkish public hospitals are evaluated annually with a multidimensional performance measurement tool. The performance scores of the hospitals used in this study were the final scores that were calculated by the hospitals themselves. The performance measurement tool used in the Turkish public hospitals has 4 dimensions: 1) Health care services performance 2) Financial services performance 3) Administrative services performance 4) On-site performance assessment. Each dimension consists of a large number of indicators. Some of the indicators used to measure health care services performance are emergency room patient visit volume, volume of patients admitted through the emergency room, emergency room efficiency score, outpatient services efficiency score, readmission rate, patient admissions, inpatient services efficiency score, patient satisfaction rate, average length of stay, bed turnover rate, bed occupancy rate, operating room efficiency score, waiting times etc. Total assets to total accruals ratio, loan period, total expenditures, total income, total accruals, bank holdings etc. are some of the indicators used in the measurement of financial services performance. Some of the metrics used to measure the administrative services performance are employee training participation rates (hand-hygiene training, employee rights and safety training, patients’ rights training, etc.), medical waste per staffed bed, employee satisfaction rate, etc. On-site performance assessment covers a wide range of areas such as infrastructure and equipment, infection control and prevention, facility safety and management, patient safety and it is performed by the auditors of the Ministry of Health. Each dimension is measured on a scale of 1000 points and the dimensions do not have equal weight in the total hospital performance score. The weight of health care services performance dimension is 35%, financial and administrative services performance is 20% and on-site performance assessment is 25%. The final score is the weighted sum of the dimensions.

Perceptual data of hospital performance was obtained using a survey instrument. The survey instrument was developed following the performance assessment tool used in Turkish public hospitals. The survey questionnaire was developed based on the dimensions of the public hospital measurement tool. The participants (414 chief employees) were asked to provide their perceptions regarding the performance of ‘health care services’, ‘financial services’, ‘administrative services’, ‘teaching and training’ and ‘employee satisfaction’, ‘patient satisfaction’ and ‘overall performance’. The survey questionnaire consisted of 17 questions of which 10 was about the demographic characteristics of the participants and 7 was about their perceptions on the performance of their respective hospitals. A 7-point Likert-type scale was used. The range of the scale was 1= ‘Very Poor’, 2= ‘Poor’, 3= ‘Somewhat Poor’, 4= ‘Neither good or poor’, 5= ‘Somewhat Good’, 6= ‘Good’, 7= ‘Very Good’ with an option of ‘No Opinion’ at the end.

Data Analysis and Results

In the EFA the factor structure is determined based on the correlations among the variables in the data set. Principal components method of extraction and varimax rotation was performed on the survey items. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.907 indicating that the strength of relationship among the variables was high. Bartlett’s Test of Sphericity is another measure of sampling adequacy and tests the overall significance of the correlations among the items in the construct. The Bartlett’s test of sphericity was significant (chi-square= 1661.170, p<0.001) confirming sampling adequacy for the factor analysis. The factors loaded in a single factor and the loadings were above 0.5. The factor explained 63.28% of the total variance. Reliability analysis was performed to ensure consistency among multiple measurements of the variables. Cronbach’s alpha score of 0.899 was obtained indicating sufficient construct reliability. Commonly, values above 0.7 are considered satisfactory (Nunnally & Bernstein, 1994). The results of the EFA are shown in Table 01 .

Table 1 -
See Full Size >

414 participants were divided into five groups based on their professions: 1) physicians, 2) nurses and midwives, 3) clinical support staff (laboratory personnel, pharmacists, physiotherapists etc.), 4) managers, and 5) administrative support staff. The participating hospitals were also split into three groups depending whether they are 1) Specialty hospitals 3) Teaching and research hospitals or 3) State hospitals.

A One-way analysis of Variance (ANOVA) was used to investigate three questions:

  • Whether physicians, nurses and midwives, clinical support staff, managers and administrative support staff have different perceptions of hospital performance. The independent variable is the type of profession while the dependent variable is the perceptual performance.

  • Whether perceptual hospital performance is different among specialty hospital, teaching and research hospitals and state hospitals. The independent variable is the type of hospital while the dependent variable is the perceptual performance.

  • Whether joint hospital performance is different among specialty hospital, teaching and research hospitals and state hospitals.

One-way ANOVA for ‘perceptual performance’ by type of profession

Levene’s Test was performed and the results of the analysis revealed that the assumption of homogeneity of variance was not met (F statistic=6.787, p<0.001). The sample size of each of the professional group is unequal. Welch’s Test is considered appropriate when the sample size is unequal and the homogeneity of variance assumption is not met. Welch’s F statistic was significant at 0.05 probability level (Welch’s F=15.412, p<0.001) implying that perceptual performance differed significantly by professional group. In order to determine which pairs of the five professional groups’ means differed significantly Games-Howell post hoc test was performed. The results of the post hoc test are shown in Table 02 .

The results indicated that perceptual performance did not statistically differ between physicians, nurses and midwives, clinical support staff and administrative support staff (p<0.05). However, all of the four professional groups had a significantly different perception of performance compared to managers. In other words, perceptual performance differed between physicians and managers, nurses & midwives and managers, clinical support staff and managers, and finally administrative support staff and managers while there is no significant difference of performance perception between physicians, nurses & midwives, clinical support staff and administrative support staff. Managers had the highest perceptual performance score (M= 5.640, SD=0.754) while the physicians (M= 4.648, SD=1.053) which was very closely followed by nurses and midwives (M= 4.674, SD=1.163) had the lowest.

Table 2 -
See Full Size >

One-way ANOVA for ‘perceptual performance’ by type of hospital

Welch’s Test was performed to determine if there was difference among groups. The results revealed that there was statistical difference of perceptual performance among different types of hospitals (Welch’s F statistic=12.650, p<0.001). Games-Howell post hoc test was performed to obtain a detailed comparison between the hospitals. The results of the post hoc tests are shown in Table 03 . The results indicated that perceptual performance differed significantly between specialty hospitals and teaching and research hospitals (p<0.001), specialty hospitals and state hospitals (p=0.035). No evidence was found that teaching and research hospitals’ perceptual performance differed from state hospitals (p=0.119). Specialty hospitals had the highest perceptual performance (M=5.29, SD=1.063), followed by state hospitals (M=4.93, SD=0.992) and teaching and research hospitals had the lowest perceptual performance score. (M=4.66, SD=1.109)

Table 3 -
See Full Size >

One-way ANOVA for ‘joint performance’ by type of hospital

Because of unequal sample size and failing to meet the assumption of homogeneity of variance (F statistic=5.439, p=0.005) Welch’s Test was conducted. Welch’s F statistic was significant (Welch’s F statistic=22.929, p<0.001) indicating that there was a significant difference in ‘joint performance’ scores between type of hospitals. Games-Howell post hoc test was conducted to investigate which hospital types differed significantly. The results of the post hoc test revealed that there was a significant difference between all of the hospital types in joint performance scores. The comparisons are displayed in Table 04 . The specialty hospitals had the highest mean for joint performance (M=4.68, SD=0.783), followed by state hospitals (M=4.32, SD=0.903) and teaching and research hospitals had the least mean (M=4.03, SD=0.725).

Table 4 -
See Full Size >

As a last step, the hospital groups based on type (specialty, teaching & research, state) were ranked depending on their mean scores on perceptual performance, joint performance and objective performance. The results are displayed in Table 05 . As mentioned before, the perceptual performance measures were obtained via the survey instrument, the objective performance measures were obtained from the hospitals’ annual performance reports and the joint performance measures were obtained by combining the objective and the perceptual measures. This was done by calculating a weighted average score for each hospital (from the objective performance scores) and multiplying it with the survey responses. The results displayed in Table 05 show that the ranking of the hospital groups (specialty, teaching &research, state) does not differ depending on the type of performance measure (perceptual, joint or objective).

Table 5 -
See Full Size >

Discussion and Conclusion

Results of the data analyses revealed that perceptions of physicians, nurses and midwives, clinical support staff and administrative support staff regarding hospital performance did not significantly differ with each other. However, all the aforementioned professional group’s perception of hospital performance differed with the managers. Considering the frequent approach of utilizing managers’ perception to obtain organizational performance data this could have some important consequences. The main justification behind using managers’ knowledge as a proxy for organizational performance is that they will have a more thorough view of the organization and they have first-hand exposure to the organization’s financial operations. The findings of the study revealed that basing organizational performance measurement only on mangers’ perception could possibly lead to an upward bias. All the professional groups in the study had lower and significantly different perception of the hospital performance. The results imply that in order to obtain a more balanced measure of organizational performance different professional groups should be included in the study.

The ANOVA conducted for perceptual performance by type of hospital indicated that there was significant difference in perception of performance between different types of hospitals. Similar results were obtained when the analysis was repeated for ‘joint performance’. The results imply the importance of including different types of hospitals in a study when conducting a research since organizational structure could be an important determinant in organizational performance. Moreover, when the hospital groups were ranked based on their mean scores of perceptual performance, joint performance and objective performance the ranking did not change (see Table 05 ). Specialty hospitals had the highest performance scores in perceptual, joint and objective measures. State hospitals ranked second while teaching and research hospitals ranked third in each measure. Having similar results in perceptual, objective and joint performance measures may imply that using perceptions of employees from different departments of the hospital on organizational performance could indeed be a close proxy for objective measures.

Managers typically have a general knowledge of the operations which is undoubtedly crucial. However, it is possible that a general view can sometimes cause some important details to be missed. Clinicians and other medical staff may not have accurate information on financial performance of the hospital while they are expected to have a more accurate perception of the health services performance. So, it may be wise to obtain financial indicators from the managers while for the procurement of other performance data, other professional groups could be utilized in an effort to obtain more balanced and comprehensive performance data. The survey instrument used in this study to measure perceptual organizational performance was developed based on the general framework of the objective organizational assessment tool utilized by the hospitals in the study in order get a general perceptual view of performance. Future studies may consider a more detailed framework to get a more robust measurement metric. Hence, when it is not possible to obtain objective performance measures perceptual measures could be used instead. Moreover, when possible, conducting the research using both objective and perceptual measures should be preferred in order to obtain a more comprehensive and accurate view of organizational performance.

References

  1. Barney, J. (1991). Firm Resources and Sustained Competitive Advantage. Journal of Management, 17(1), 99-120.
  2. Combs, J. G., Crook, R. T., & Shook, C. L. (2015). The Dimensionality of Organizational Performance and Its Implications for Strategic Management Research. Research Methodology in Strategic Management, 2, 259-286.
  3. Dess, G. G., & Robinson R. B. (1984). Measuring organizational performance in the absence of objective measures: the case of the privately-held firm and conglomerate business unit. Strategic Management Journal, 5(3), 265-273.
  4. Harris, L. C. (2001). Market orientation and performance: objective and subjective empirical evidence from UK companies. Journal of Management Studies, 38, 17-43.
  5. Hasson, H., Tafvelin, S., & von Thiele Schwarz, U. (2013). Comparing Employees and Managers’ Perceptions of Organizational Learning, Health and Work Performance. Advances in Developing Human Resources 15(2) 163-176.
  6. Hult, G. T. M., Ketchen Jr., D. J., Griffith, D. A., Chabowski, B. R., Hamman, M. K., Dykes, B. J., … & Cavusgil, S. T. (2008). An assessment of the measurement of performance in the international business research. Journal of International Business Studies, 39(6), 1064-1080.
  7. Kaplan, B. S., & Norton, D. P. (1992). The balanced scorecard: Measures that drive performance. Harvard Business Review 70(1), 71-79.
  8. McCracken, M. J., McIlwin, T. F., & Fotter, M. D. (2001). Measuring organizational performance in the hospital industry: an exploratory comparison of objective and subjective methods. Health Services Management Research, 14, 211-219.
  9. Meier, K. J., & O’Toole Jr., L. J. (2012). Subjective Organizational Performance and Measurement Error: Common Source Bias and Spurious Relationships. Journal of Public Administration Research and Theory, 23, 429-456.
  10. Moullin, M. (2004). Eight essentials of performance measurement. International Journal of Health Care Quality Assurance, 17(3).
  11. Nunnally, J. C., & Bernstein, I. H. (1994). The Assessment of Reliability. Psychometric Theory, 3, 248-292.
  12. Porter, M. E. (1985). Competitive Advantage. Creating and Sustaining Superior Performance. New York, Free Press.
  13. Richard, P. J., Devinney, T. M., Yip, G. S., & Johnson, G. (2009). Measuring Organizational Performance: Towards Methodological Best Practice. Journal of Management, 35(3), 718-804.
  14. Rumelt, R. P. (1984). Towards a Strategic Theory of the Firm. In R. Lamb (ed.). Competitive Strategic Management, Englewood Cliffs NJ, Prentice-Hall, 556-570.
  15. Singh, S., Darwish, T. K., & Potocnik, K. (2016). Measuring organizational performance: a case for subjective measures. British Journal of Management, 27, 214-224.
  16. Venkatraman, N., & Ramanujam, V. (1987). Measurement of Business Economic Performance: An Examination of Method Convergence. Journal of Management, 13(1), 109-122.
  17. Walker, K. B., & Dunn, L. M. (2006). Improving Hospital Performance and Productivity with Balanced Scorecard. AHCMJ, 2, 85-110.
  18. Wall, T. Michie, J., Patterson, M., Wood, S., Sheehan, M., Clegg, C., & West, M. (2004). On the validity of subjective measures of company performance. Personnel Psychology, 57, 95-118.

Copyright information

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

About this article

Publication Date

30 October 2019

eBook ISBN

978-1-80296-070-9

Publisher

Future Academy

Volume

71

Print ISBN (optional)

-

Edition Number

1st Edition

Pages

1-460

Subjects

Business, innovation, Strategic management, Leadership, Technology, Sustainability

Cite this article as:

Zaim, S., Tarim*, M., & Ozgun, A. (2019). Using Employee Perceptions for Organizational Performance Assessment. In & M. Özşahin (Ed.), Strategic Management in an International Environment: The New Challenges for International Business and Logistics in the Age of Industry 4.0, vol 71. European Proceedings of Social and Behavioural Sciences (pp. 451-460). Future Academy. https://doi.org/10.15405/epsbs.2019.10.02.41