Effectiveness Of Implementing Practice-Oriented Higher Education Programs Based On Stakeholders Feedback
The article covers the issues of higher education development in the framework of innovative economy and the role of practice-oriented education technologies in this process. It presents substantial characteristics of successful international practice-oriented models, which potential can be used to solve problems that occur in the national system of education. It is shown that the successful implementation of practice-oriented programs involves the development of various forms of interaction between business and education, including: requirements for the quality of specialists’ training, holding of training sessions for students by the employers’ representatives¸ integration of professionally-oriented education technologies, creation within universities of innovative forms of professional employment in accordance with the learning profile, formation of professional skills of students through their immersion in professional environment, project integration. To meet labor market needs, programs are to consider the opinion of all stakeholders. In this paper, the authors first identify key stakeholders to be considered in the process of study programs development. Then they argue that online surveys are a preferable method of data collection for this purpose. Secondly, the authors focus on these online surveys being the main object of this research. They outline selection parameters which should be taken into account when planning such surveys by asking questions and giving guidelines based on practices and experiments within the universities. Thirdly, the authors identify some common pitfalls to be avoided when designing such surveys. The described survey's methodology can be accordingly included in the content of study programs in management and pedagogy.
Keywords: Higher educationpractice-oriented educationinteraction with the employerstakeholderonline survey
One of the key trends of higher education in the innovation economy is the provision of an optimal combination of fundamental and professionally-applied training, transfer from transmission of knowledge technologies to practice-oriented educational technologies and getting learning experience.
Higher education cannot evolve as a closed system. The main complaint of employers to universities is the isolation of knowledge obtained by graduates from practice, psychological unpreparedness for the realities of labor activity, the rules of conduct in a business environment even with a good theoretical level of education. The gap between theory and practice is to be necessarily resolved by the company for which the provision of qualified personnel is a topical issue. It should be emphasized that the lack of practice is causing frustration, which is expressed not only by the employers but also by the students themselves as stakeholders (Konovalova, 2008; Remorenko, 2011; Wilson, 2014). In the state program "Development of education" for 2013-2020, it is emphasized that problem-solving in the national system of higher education requires the understanding of the experience gained in this area by foreign countries and support of approaches that facilitate the adaptation of Western practices of mass, regionally-oriented and practice-oriented higher education to the national system.
In European countries, different forms of practice-oriented education programs are applied, the effectiveness of which depends on the direction of training.
The model of applied baccalaureate involves the implementation of educational programs, corresponding to the International standard classification of education (ISCED), level 5 B, described as a practice-oriented program of the first stage of tertiary education with the duration from two to four years. The programs provide both fundamental knowledge in a particular subject area and also a professional skill to work with complex technology. In the countries of Western Europe, they are being implemented at universities along with the academic programs of a bachelor degree, and in specialized institutions of professional education. In Russia, such programs are implemented, for example, in the Financial University under the Government of the Russian Federation, National research University "Higher school of Economics", The Russian Presidential Academy of National Economy and Public Administration Saint Petersburg state electrotechnical University, the Ural Federal University.
The model of liberal arts and sciences involves the implementation of individual interdisciplinary educational programs, which results of exploration are: breadth of vision, the ability for independent judgment and decision making, the ability to quickly and flexibly navigate in a constantly changing world and to defend their own point of view, using reasoned arguments and a willingness to apply their knowledge in a wide range of fields (Van der Wende, 2011). In Russia, such programs are implemented, for example, on the Faculty of Liberal Arts and Sciences of St. Petersburg state University.
The model of «corporate University» involves the creation of multifunctional centers of qualification certification, the creation of corporate and educational organizations established to implement the educational programs of professional education, developed on the basis of professional standards (qualification requirements) (Blass, 2005; Shaw, 2005). At the first stage, corporate universities in Russia arose only with the business structures that eventually led to the fact that the idea of the corporate University often began to be substituted by a set of trainings that were aimed at improving the skills of the individual managers in finance, marketing or sales. In this connection, there is a necessity of creation of corporate universities in the leading higher education establishments of the country and radical improvement of the education quality corresponding to international standards in such universities as: Higher business school, Graduate school of public administration, Graduate school of innovative business and Graduate school of management and innovation at Moscow state University named after M. V. Lomonosov, Higher school of management at St. Petersburg state University (Trifonov et al., 2014; Leontyeva, 2012).
The model of cooperation between schools of different levels and business enterprises (SEED) (the program of the Swedish Foundation for Enterprise Education) involves the usage of practical methods of entrepreneurship education aimed at developing students’ spirit of enterprise and creativity, working in a real company directly with the head of the organization, who becomes a mentor for a student. In Russia such programs are implemented, for example, at Arkhangelsk state technical University, Herzen State Pedagogical University of Russia, Tomsk national research Polytechnic University.
The "CDIO (Conceive – Design – Implement – Operate) global initiative" model for reforming the basic engineering education involves the implementation of practice-oriented programs of engineering and technology focus, the objective of which is to ensure the influx of highly professional, ready for responsible and independent work engineering and technology personnel. The CDIO standards – a comprehensive approach to engineering education: a set of general principles for the creation of educational programs, its logistics, recruitment and training of educators (Mills & Treagust, 2003; Edström et al., 2014; Jesiek et al., 2010; Chuchalin, 2011). In Russian programs based on the CDIO, standards are implemented, for example, in Tomsk national research Polytechnic University, Astrakhan state University, Moscow Institute of Physics and Technology.
The model of dual education is an innovative type of professional training, which involves coherent interaction between educational and industrial spheres for the training of future professionals with a high degree of mobility in the labor market. The dual education system is practiced in many countries, notably in Germany, Austria, Switzerland, Denmark, the Netherlands, France, the last few years in China and other Asian countries. The dual education model is based on the active participation of employers in a preparation process: training at the production sites with the involvement of production experts, the incorporation of students into the corporate norms of the organization of production and production relations, the participation of students in solving real production problems (Aili et al., 2015; Graf, 2017; Gonon, 2009; Powell et al., 2012). In Russia, such programs are implemented, for example, at Moscow Institute of Physics and Technology, Kazan National Research University.
It should also be noted that the practice-oriented masters' programs in the sphere of technological entrepreneurship are implemented at several universities in Moscow, St. Petersburg and Tomsk. The formation of the basic courses of masters' programs uses Western methodologies such as Customer Development, Lean Startup, Lean Canvas, Business Model Canvas, which are used directly by the master students in the development of their projects.
Despite the existing experience of the successful use of practice-oriented models of higher education, the problem of a discrepancy between the pace of change in business and in education still remains, namely, the delay in the application of the knowledge obtained at the university and its rapid obsolescence. Many Russian employers are still unable to formulate the requirements for a professional profile of a University graduate; they are not ready for a dialogue with the educational system and for transition to other, more flexible educational models focused on the rapid changes in the market environment. Higher establishments also need radical changes in the set up of the teaching process.
Solving of these problems requires regular interaction with all stakeholders. Where it is possible to establish a dialogue, the interaction between universities and employers is realized in the following forms:
requirements for the quality of specialists’ training;
holding by the employers’ representatives of training sessions for students, including lecture courses, seminars, master classes, business games, workshops, etc.;
integration of professionally-oriented education technologies, aimed at formation of significant professional activity knowledge, abilities, skills of future professionals;
creation within universities of innovative forms of professional employment for students to solve their scientifically-practical and experimental-industrial tasks in accordance with the learning profile;
formation of professional skills of students by their immersion in a professional environment during their practical trainings;
involvement of employers in final state certification of graduates;
teaching and consulting the educators, training the educators at real workplaces;
professional retraining of the employers’ representatives at the universities;
involvement of employers in scientific and practical conferences, educational projects, scientific sessions, science days at universities, etc.;
project integration (joint research, creation of joint high-tech small companies, etc.);
employment of University graduates.
For the development of a dialogue between business and education, it is necessary to improve the effectiveness of existing forms of cooperation between stakeholders and to search for new ones.
Purpose of the Study
Universities have an incentive to identify key stakeholders that are important to raise universities’ attractiveness and to involve them in the continuous process of study programs development and improvement (Butin, 2010). Thus, it is very important not only to identify the key stakeholders to be considered in the process of study programs development, but also to develop adequate tools to obtain feedback from them.
Prospective employers, recent alumni, university faculty, current students can be mentioned as the important stakeholders aimed at improving the quality of education.
Here the first two of the mentioned are external stakeholders who are primarily interested in what they teach at the university, that is, in the structure of study programs and as a result in learning outcomes. At the same time, the last two are internal stakeholders who are primarily interested in how to teach, that is, the instructional design and tools are used. In the process of improving the study programs, universities must take into account the opinions of all groups of stakeholders, which requires getting feedback from stakeholders on a regular basis primarily through surveys (Chatterton et al., 2000).
Although the authors propose using surveys for retrieving stakeholder feedback (Aghashyan et al., 2014), other methods like interviews and focus groups can be helpful at early stages of the research to get insights into which questions to ask in the survey and investigate particular aspects in more detail. Experience shows that to obtain a sufficiently high proportion of meaningful response, it is preferable to focus on online surveys using well-known services (Google Forms, QuizMaker, ProProfs, etc.) or creating a piece of specialized software (Duffy et al., 2005). In all cases, the results of these activities will largely depend on the quality of the relevant questionnaires. General advice how to choose an appropriate research design and particular survey design can be found in a number of available publications (De Vaus, 2013; Fraenkel et al., 2012; Creswell, 2013; Creswell, 2014).
Key Preliminary Questions when Preparing a Stakeholder Survey
Indeed, when it is necessary to gather information from large groups of stakeholders, the best way to do so is to send surveys. The main problem lies in the considerable complexity of the development of the survey. There are many ways of asking questions, and if one incorrectly formulates the question, one will not get the feedback one needs. That is why it is useful to consider the technique of asking the right questions so that preparing survey becomes both painless and productive.
First of all, it should be noted that before surveying others, one needs to make an honest self- survey. If one is getting ready to send an online survey, one probably already has some questions in mind for the stakeholders. Obviously, the questions will be given to the stakeholders must be well thought out. However, not less important are the preliminary questions that are helpful to ask oneself before one prepares the questions for stakeholders. Basic questions that one should ask oneself (Why, Who, What, How, When is it necessary to ask?) usually do not get as much attention as the questions to be included in the survey. If the person spends enough time to answer these questions, without any doubt, results of the future survey will be much more useful.
The most important self-question is: Why is it necessary to ask? In other words, the most important step in a survey is figuring out what actually should be known. It is important to make personal objectives really clear up front otherwise, it will be difficult to implement next steps. It is necessary to concentrate on the end goal, namely: What decision is to make or what is to be changed based on the feedback got from the stakeholders?
The second important self-question is: Whom to ask? It is very important to choose the right list of stakeholders for getting valuable feedback. In essence, this question proceeds from the main objectives, and correctness of the answer to this question largely determines the effectiveness of the survey.
The next significant self-question is: What to ask? Every person has their objectives and their sample list of stakeholders identified. But sometimes a personal study remains empty for a long time and the problem requiring solution is still not solved. It can be a very difficult problem for which the correct short answer is not enough. In this context, it is necessary to ask additional questions which answers will help to reach the goal. If the person clearly submits what kind of feedback from stakeholders is necessary to achieve to reach the goal, then it will be easier to determine what to ask.
The main technical self-question is also important enough: How to ask? There are many forms of questions a reasonable choice from which enables us to obtain reliable and useful answers (Collins, 2003). For a complete understanding of the forms of questions which can be asked, for example, look at Google Forms (docs.google.com/forms) types of survey questions can be added and how they will appear in an actual survey. Thus, it is very important to strive to ensure that questions were possibly direct, simple and concise. This will make it easy for stakeholders to understand what is asked and will make it easy to analyze the feedback.
And finally, last but not least self-question is: When to ask? In order to understand when it should be conducted in a survey is necessary to consider many factors, such as convenience for most involved stakeholders, the time slot for the corresponding reforms in the university and others. Assigning too close deadline for the survey can exclude from the consideration stakeholders, who are a little slower in making decisions or busier. As a result, the conclusions drawn from the survey could be less useful.
Thus, it can be recommended using aforementioned 5 questions to guide, and the survey will be off to a good start before writing it.
Guidelines for Designing the Survey
Next, when writing and carrying out a survey, it is necessary to consider some of the basic recommendations and warnings, which are derived from the authors’ experience and analysis of existing sources (De Vaus, 2014; Collins, 2003; Rea & Parker, 2012).
An effective survey should flow in an orderly fashion, help to stimulate recall (if necessary) and motivate the respondent to reply. That is why it can be regarded as a good practice to adhere to the following guidelines:
Clearly define the goals. Foremost remember “Why”, namely the entire reason conducting a survey in the first place? Communicate that to stakeholders so they can identify with the purpose. If the survey does not have specifically formulated goals it is unlikely to get a large number of responses.
Watch for shortness of surveys. Writing short surveys helps ensure that stakeholders start and finish the survey. Stakeholders, as a rule, are much more receptive to partaking if they know the survey will not take them much time to complete. They take more time to respond to each question when taking shorter surveys as well, which means to be more likely to collect accurate feedback.
Start with the overall and gradually move on to details. Very important is the most overall, logical and concretized placement of questions in the survey, because a sequence of questions largely determines the extent and usefulness of the responses to the survey.
Preferably use words instead of numbers. If there is a need to ask stakeholders about some choosing does not overload them with too many options. For evaluation (rating) questions are necessary to limit the number of respectively labeled options up to five, and only in exceptional cases, up to ten. If there is a need to measure stakeholders attitudes or behavior preferred to use questions with Likert scale (Bertram, 2012). This is essentially the most convenient and popular method to evaluation opinion from one extreme to another (for example, “Extremely likely” to “Not likely at all”).
Use unlimited questions moderately. Unlimited (or so-called open-ended) questions, such as “What is a favorite tool for penetration testing and why?”, mainly can be acceptable while qualitative assessment of study program. In the development and improvement of the study program, such questions are not very comfortable, as they are unfocused and therefore imply obtaining very diverse responses which are extremely difficult to analyze. So, the authorsour recommend having no more than 1-2 unlimited questions per survey to save the time in the analysis stage.
Allow for shades of answers. Sometimes it can be tempting to choose questions that provide with a simple “Yes” or “No” answer because they are really easy for stakeholders to answer, and it would seem as if the 50/50 responses would make for easier analysis later on. Even though “Yes” or “No” questions are appealing, try to use them moderately for two major reasons. First of all people’s views and behaviors can be fuzzy, so their opinions about what is measured do not always coincide with one of two predefined answers. Secondly, writing questions that allow stakeholders to respond with some shades from “Agree” to “Disagree” (for example Likert scale questions) actually guarantees that will be obtained answers quite suitable for statistical analysis, as well as simply more informative answers because they include quantifiable evaluation.
Only ask questions that give the answers to act on. The whole idea of conducting a survey in this case is to find out a key piece of information in order to actually change something in education, such as study programs. If in this case, ask questions, for answers to which there is no a concrete action plan, it is essentially wasting the time and the time and energy of stakeholders.
In general, it should be noted that a good survey is like a good conversation in that it is written in a personable, logical fashion, in a tone stakeholders can relate to and it allows them to feel listened to. In many ways, this is the main reason why stakeholders agreed to participate in surveys, first of all, they want to be heard. If stakeholder took the time to answer questions, it is a good idea to continue the dialogue and to inform him what improvements have been made in the study program based on his answers. Perhaps, it will take a lot of time but will contribute to the further fruitful cooperation with stakeholders.
Summing up the discussions, it can be said that surveys need to be very carefully and thoughtfully created. Doing the hard work up front will allow stakeholders to easily complete the survey, and thereby to be provided with quality data. The survey must have a meaningful purpose, flow in a logical order, contain uncomplicated language, and avoid tedious question types (e.g. ranking or matrix questions). If the survey is simply too difficult to understand, for whatever reason, it will be just annoying to the stakeholders and end up with unusable data, with which nobody can deal, which defeats the whole purpose of conducting a survey in the first place. It needs to be remembered that bad data can be worse than no data at all. Incorrect data will lead to poor decision making that could eventually destroy the educational process, instead of improving its quality.
World practice developed diverse and effective models of cooperation between business and universities, the elements of which could be used by Russian universities. However, the process of dialogue establishment between the educational and business communities also was not easy and took a long time. Many universities considered themselves and their educational activities in fundamental science above the real economy, and considered the contacts with businesses as "submission" to it.
For the modern Russian practice of building effective and diverse forms of interaction between educational and business communities, it is crucial to enhance the practical orientation of educational programs.
Application of the proposed methodology surveys to gain feedback from stakeholders in the implementation of programs of higher education will help to:
increase motivation for employers to dialogue with the education system, to form a new culture of their interaction;
update the content of educational programs in accordance with the latest achievements in the corresponding area of activity, improve the quality of the educational process;
form the graduates’ professional competences, providing them with a high level of demand in professional environment and a successful business career;
significantly reduce the period of adaptation of graduates to the conditions of the practical activities and specific needs of employers.
- Aghashyan, R., Margarov, G., Rapp, C. & Mitrofanova, E. (2014). The Role of Stakeholders in the Process of Study Program Improvement. Proceedings of International Conference «Education in the 21st century: Problems and Challenges». (рp.31-33). Yerevan State University, Armenia.
- Aili, С., Nilsson, L.-E. (2015). Dual learning – a challenge for higher education in the new landscape of governance. Tertiary Education and Management, 21 (4), 277-292.
- Bertram, D. (2012). Likert scales. Calgary, Alberta, Canada.
- Blass, E. (2005). The rise and rise of the corporate university. Journal of European Industrial Training, 29 (1), 58-74. doi: 10.1108/03090590510576217.
- Butin, D.W. (2010). Service-Learning in Theory and Practice: The Future of Community Engagement in Higher Education. Palgrave Macmillan.
- Chatterton, P., Goddard, J. (2000). The response of higher education institutions to regional needs.
- Chuchalin, A.I. (2011). [Modernization of bachelor engineering programmes considering international standards of engineering education]. Vysshee obrazovanie v Rossii [Higher education in Russia]. 10, 20-29. (In Russ.)
- Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12 (3), 229–238.
- Creswell, J.W. (2013). Qualitative Inquiry and Research Design: Choosing Among Five Approaches.
- Creswell, J.W. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage. De Vaus, D. (2014). Surveys in Social Research. Routledge.
- De Vaus, D. (2013). Research Design in Social Research. Sage.
- Duffy, B., Smith, K., Terhanian G. & Bremer, J. (2005). Comparing data from online and face-to-face surveys. International Journal of Market Research. 47 (6), 615–639.
- Edström, K., Kolmos, A. (2014). PBL and CDIO: complementary models for engineering education development. European Journal of Engineering Education, 39(5), 539-555. doi:10.1080/03043797.2014.895703
- Fraenkel, J.R., Wallen, N.E., Hyun, H.H. (2012). How to Design and Evaluate Research in Education.
- Gonon, P. (2009). ‘Efficiency’ and ‘vocationalism’ as structuring principles of industrial education in the USA. Vocations and Learning, 2, 75–86. 10.1007/s12186-008-9016-z.
- Graf, L. (2017). Work-based higher education programmes in Germany and the US: Comparing multi- actor corporatist governance in higher education. Policy and Society, 36(1), 89-108. doi: 10.1080/14494035.2017.1278872.
- Jesiek, B.K., Borrego, M., & Beddoes, K. (2010). Advancing global capacity for engineering education research: relating research to practice, policy and industry. European Journal of Engineering Education, 35(2), 117-134.
- Konovalova, V. (2008). Global trends in the development and training of personnel and the situation in Russia. Kadrovik, 9, 61-69.
- Leontyeva, Y.G. (2012). Corporate university as innovative corporate training institution model. Tomsk State University Journal, 361, 144-147.
- Mills, J.E., Treagust, D.F. (2003). Engineering education - is problem-based or project-based learning the answer? Australasian Journal of Engineering Education, 9(1), 2003, 2–16.
- Powell, J. J. W., Bernhard, N., & Graf, L. (2012). The Emerging European Model in Skill Formation: Comparing Higher Education and Vocational Training in the Bologna and Copenhagen Processes. Sociology of Education, 85, 240–258.
- Rea, L.M., Parker, R.A. (2012). Designing and Conducting Survey Research: A Comprehensive Guide.
- Remorenko, I.M. (2011). Transition to an Innovative Economy: Opportunities and Limitations for the Education System. Educational Studies, 3, 54-73.
- Shaw, S. (2005). The corporate university: Global or local phenomenon? Journal of European Industrial Training, 29 (1), 21-39. doi: 10.1108/03090590510576190.
- Trifonov, Y., Zharikov, A., Shiryaevа, J. (2014). The development of innovative education on the basis of corporate universities. Professional Education in Russia and Abroad, 4 (16), 103-105.
- Van der Wende, M.C. (2011). The Emergence of Liberal Arts and Sciences Education in Europe: A Comparative Perspective. Higher Education Policy, 24 (2), 233 – 253.
- Wilson, J. W. (2014). Co-operative education: A means of relating work and education. In D. J. Waddington (Ed.), Education, industry and technology (pp. 349–353). New York, NY: Elsevier.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
About this article
18 December 2019
Print ISBN (optional)
Business, business innovation, science, technology, society, organizational behaviour, behaviour behaviour
Cite this article as:
Margarov, G., Mitrofanova, E., Konovalova, V., Mitrofanova, A., & Trubitsyn, K. (2019). Effectiveness Of Implementing Practice-Oriented Higher Education Programs Based On Stakeholders Feedback. In I. B. Ardashkin, N. V. Martyushev, S. V. Klyagin, E. V. Barkova, A. R. Massalimova, & V. N. Syrov (Eds.), Research Paradigms Transformation in Social Sciences, vol 35. European Proceedings of Social and Behavioural Sciences (pp. 906-914). Future Academy. https://doi.org/10.15405/epsbs.2018.02.107