Using Integral Indicators In Evaluation Of Project Implementation

Abstract

When implementing any project, the question arises about the use of indicators for evaluating its effectiveness. Since the project often has a long-term implementation period, it is important to monitor the effectiveness of management decisions not only upon the implementation of the project, but also in dynamics. Assessment of the level of implementation should consider a large number of indicators, which greatly complicates the analysis. To reduce the multidimensionality of analytical operations, it is suggested to calculate the integral indicator of the level of project implementation. This quantitative value will give a generalized description of the effectiveness of the project at each stage. Any number of indicators can be included in an integrated assessment of project performance. Each of them will be brought to a comparable form to track its effect on the final value. The calculation of the proposed integral indicator should be carried out several times at each individual stage of the project. It will allow timely identification of discrepancies in actual results to the planned ones and implementation of corrective and regulatory activities. In this option, the project manager is provided with timely operational analytical information on the progress of the project. The proposed integral indicator of the effectiveness of the project acquires practical significance because the algorithm for calculating this value allows tracking the impact of indicators included in its assessment related to individual subject groups (generalized categories). This approach allows assessing the strengths and weaknesses of the project, identifying the risks of its implementation.

Keywords: Integral assessmentproject implementationproject managementproject subject groups

Introduction

The issue of developing mechanisms for evaluating a project not only upon its completion, but also during the period of its implementation, is urgent in the context of the effectiveness of management methods. Timely identified problems can be corrected with timely management decisions. It is important to provide the project manager with relevant, reliable and easily interpreted information (Kozin & Astarkina, 2010).

Integrated approaches to assessing the effectiveness of the project were considered earlier in some works of Russian and foreign authors. For example, Khlynin and Khoroshilova (2011) propose using cluster analysis to identify subject groups of indicators included in the integral assessment. A peculiarity of this method will be its specific application for each individual project, since depending on the initial data the set of subject groups will be slightly adjusted each time.

Kutsenko (2015) describes the mechanism for constructing integral assessments using investment projects as an example. This integral indicator proposed by the author includes both qualitative and quantitative characteristics. It allows considering the economic interests of all stakeholders of the project.

The modern project management methodology provides detailed information on the principles of project organization, the role status of project participants, the progress of its implementation and completion (Green, Welsh, & Dehler, 2003). The project, as a rule, has a long-term nature. If at the first stage of its implementation all participants have information about the level of its implementation, then over time, the clarity of the data may be lost. It is important to develop a mechanism for quantifying the effectiveness of a project at each stage of its implementation.

Problem Statement

It is extremely important for the project manager to have a real idea of the implementation characteristics at each stage. The difficulty lies in the large volume of indicators that evaluate any project. GOST R ISO 21500-2014 singles out ten subject groups; each of them includes individual study components and is characterized by a specific set of indicators. They reflect the effectiveness of management decisions in each block of the project.

As a rule, it is not easy to bring together a large number of multidirectional data. That is why the assessment of the overall project effectiveness at each stage is a complicated process. An integral indicator calculated using primary indicators of each subject group will give a generalized characteristic of the level of effectiveness of the project (Vlaar, 1998).

Research Questions

The use of integrated characteristics in assessing the effectiveness of project implementation will reduce the multidimensionality of primary data for analysis (Grishakina & Zaretskaya, 2013). Based on the dynamics of the integral indicator, one can judge the overall level of implementation of all project characteristics.

Since the level of project implementation is a multifaceted indicator, it is proposed to include in the construction of its integral value several generalized categories (hereinafter - GC) reflecting various processes during the implementation of the project. Each GC will be characterized by its own set of indicators. The principle of the GC formation is determined by the goals and objectives of monitoring the level of implementation of each specific project.

According to GOST R ISO 21500-2014, ten GCs of indicators in each project: “Integration”, “Interested Parties”, “Content”, “Resources”, “Timing”, “Cost”, “Risks”, “Quality”, “Purchasing”, and “Communications” are distinguished.

In each GC (subject group), primary indicators for evaluation of the level of project are identified.

Purpose of the Study

The generalized integral indicator of the effectiveness of the project should meet the following requirements:

• be universal. Reflect the level of effectiveness of the project at each stage;

• be objective. The growth of this characteristic should reflect real positive changes in the course of the project;

• be easy to interpret. This value should not only reflect the dynamics of the effectiveness of the project, but also answer questions about the factors ensuring this growth.

Research Methods

To calculate the integrated generalizing characteristics of the level of implementation efficiency, a technique is proposed that allows comparing selected indicators in dynamics over a period of time.

Each indicator must be brought to a comparable form by the following transformations:

• for indicators describing positive processes during the implementation of the project (level of communication of project participants, etc.) the following formula is applied:

$P_{ij}^t=\mathrm{}\raisebox{1ex}{$X_{ij}^t$}\!\left/ \!\raisebox{-1ex}{$X_{ij}^S$}\right.$.

Fraction is a relative value (an indicator of advantages) of the i-th attribute (i = 1, 2, ..., n) included in the t-th period in a generalizing category j (j = 1, 2, ... 10); The fraction numerator is the value of the i-th attribute in the t-th year for the j-th generalizing category; The denominator of the fraction is the value of the i-th attribute, taken as the basis of comparison, for the j-th generalizing category.

The higher the calculated value obtained is, the higher is the level of effectiveness of the project implementation relative to the level of the indicator taken as the basis for comparison.

• for indicators describing negative processes (terms of project implementation, level of project costs, etc.) the calculation formula is as follows:

$P_{ij}^t=\mathrm{}\raisebox{1ex}{$X_{ij}^S$}\!\left/ \!\raisebox{-1ex}{$X_{ij}^t$}\right.$,

where j – is the number of the particular generalized indicator;

P – is the relative value (indicator of advantages) of the i-th attribute (i = 1, 2, ..., n), included in the t-th period in j - a generalized category (j = 1, 2, .. 10);

i – is the number of features characterizing the generalized category j

If the project has positive dynamics of the initial basic indicators, the benefit indicator will have a value greater than 1, if the situation is the opposite, then the indicator value tends to 0.

For the comparison base, it is recommended to use the desired (reference) value of indicators at each stage. This variant makes it possible to track the level of effectiveness of the project at each stage in regards to the planned data.

The conversion of primary a priori indicators removes many of the difficulties of analyzing the integrated assessment of the level of project effectiveness, as it solves the problem of unification of scales and contralateral units of measurement.

Particular general indicators (for each generalized category) are calculated as the geometric mean value from the number of advantage indicators forming these generalized categories:

$f_j=\mathrm{}\sqrt[i]{PR\mathrm{}{P}_{ij}^t}$,

where j – is the number of the particular generalized indicator;

P – is the relative value (indicator of advantages) of the i-th attribute (i = 1, 2, ..., n), included in the t-th period in j - a generalized category (j = 1, 2, .. 10);

i – is the number of features characterizing the generalized category j,

PR – production of indicators.

Further, the layout is carried out similarly for each of the ten generalized categories.

The integral indicator of project implementation efficiency for the t-th period relative to indicators taken as a comparison base is calculated for all ten generalized categories according to the formula:

$K_{ef}^t=\mathrm{}\sqrt[\mathrm{10}]{PR\mathrm{}{f}_j^{}}$,

where K – is an integral indicator of the effectiveness of the project for the t-th period.

As a result of the implementation of this series of mathematical algorithms, integral estimates for each generalized category (subject group) and a generalized integral indicator of the effectiveness of the project will be obtained (Grishakina & Zaretskaya, 2018).

Findings

As an example of the use of this integral indicator of the effectiveness of the project, we give a case of its calculation in the project for organizing an international conference.

For the calculation, ten subject groups were singled out in accordance with GOST R ISO 21500-2014. In each of them, primary indicators for analysis were designated. The project implementation included three stages. At each of them, reference (desired) values were determined for each primary indicator.

To bring the primary indicators to a unidirectional form, the formulas shown in Figures 1 and 2 were used. If the growth of the indicator reflected the positive dynamics of the project, then its actual value was compared with the reference, and if, on the contrary, the growth of the indicator ascertained negative dynamics, then the reference value was compared with the actual. Further, in each subject group, generalized categories (GC) were calculated. They characterize the effectiveness of the project in each individual group of goals and results. Using the geometric mean formula, the integral indicator of the project implementation efficiency was calculated (Table 01 ).

According to the dynamics of the integral indicator, we can conclude that in the first two stages of the project there was a slight lag in the indicators from the intended reference values (the integral estimate is slightly less than 1). This fact was due to very low values in the subject groups “Content”, “Quality” and “Purchasing”.

In the subject group “Purchasing”, over the entire period of the project implementation, actual indicators only reached the normative ones by half. The project manager should analyze this fact separately. In this case, either ineffective activity of the project participants in the procurement sphere is noted, or the benchmarks for this subject group were incorrectly set (too high). In the last stage, the integral indicator of the project is more than 1, so we can conclude that it was successfully completed. When closing a project, it is also necessary to analyze the dynamics of indicators for each generalized category.

Conclusion

The use of integral indicators in the practice of evaluating the effectiveness of any project is of great importance. When moving to this generalized category, as a rule, instead of 20-30 indicators, we select only one for analysis. It displays the generalized dynamics of each primary indicator. It greatly facilitates the analysis of project performance (Suwardy, Ratnatunga, Amrik, Sohal, & Speight, 2003).

It is important that the integral indicator of the effectiveness of project implementation does not exist separately. Its dynamics is determined by the values of each generalized category included in it (subject group). In this case, the project manager can evaluate the activities of all project participants (Bakshi, Sinharay, Sarkar, & Sanyal, 2016). If the estimated value of the generalized category for any subject group is lower than the normative, then the specialists responsible for this area do not show proper competence.

The calculation of the integral indicator at each stage of the project allows tracking the effectiveness of its implementation in dynamics. It is extremely important because of the possibility to introduce corrective actions in a timely manner (Misakov, Khamzatov, Temrokova, Misakov, & Dikareva, 2013).

References

1. Bakshi, T., Sinharay, A., Sarkar, B., & Sanyal, S. K. (2016). Introduction to soft-set theoretic solution of project selection problem. Benchmarking: An International Journal, 23(7), 1643-1657.
2. Green, S. G., Welsh, M. A., & Dehler, G. E. (2003). Advocacy, performance, and threshold influences on decisions to terminate new product development. Academy of Management Journal, 4(46), 419.
3. Grishakina, N. I., & Zaretskaya, A. S. (2013). Integral assessment of food security in the Novgorod region. Vestnik NovSU, 74(1), 20-26. [in Russ.].
4. Grishakina, N. I., & Zaretskaya, A. S. (2018). Theoretical aspects of simulation. In T. Yu. Chernysheva (Ed.), Modern decision-making technologies in the digital economy: Proceedings of the All-Russian Scientific and Practical Conference of students, graduate students and young scientists (pp. 16-18). RF, Tomsk: National research Tomsk Polytechnic University. [in Russ.].
5. Khlynin, E. V., & Khoroshilova, E. I. (2011). Formation of the concept of an integrated assessment of the effectiveness of investment projects based on the use of the cluster approach. Economic analysis: theory and practice, 47(254), 22-31. [in Russ.].
6. Kozin, M. N., & Astarkina, N. R. (2010). An integrated methodology for evaluating the effectiveness and choice of an investment project in small and medium-sized enterprises. Audit and financial analysis, 2, 281-287. [in Russ.].
7. Kutsenko, E. I. (2015). Aspects of evaluating the effectiveness of an innovative project. Bulletin of the Orenburg State University, 8(183), 73-78. [in Russ.].
8. Misakov, V. S., Khamzatov, V. A., Temrokova, A. Kh., Misakov, A. V., & Dikareva, I. A. (2013). Strategic management of innovative agro-industrial projects. Amazonia Investiga, 14, 16-23.
9. Suwardy, T., Ratnatunga, J., Amrik, S., Sohal, A. S., & Speight, G. (2003). It projects: evaluation, outcomes and impediments. Benchmarking: An International Journal, 4(10), 325-342.
10. Vlaar, P. J. G. (1998). Target zones and realignment risk: an integrated approach. De Economist, 1(146), 91-116.