Software For Responsive Evaluation Of Community Work As A Learning Tool

Community Work

One of the tools of social inclusion is community work, which emphasizes people's participation, sharing and redistribution of power, mutual solidarity and assistance, mobilizing their sense of belonging, and building a mobilized and sustainable community (Schuringa, 2007; Henderson & Thomas, 2007). Hautekeur and Henderson (2008, p. 101) refer to community work as a “participatory approach to collective problems” aimed not only at improving the living conditions of community members, but also at strengthening and empowering the community. Schoenberg (Henderson & Thomas, 2007) considers the community to be fit when it is able: a) to create mechanisms for negotiating and enforcing agreements, b) to establish formal and informal organizations in a place that have the ability to define various interests of the community in relation to its surroundings, (c) to maintain communication with public and private resource holders; and (d) to create mechanisms supporting a dialogue between competing interests and groups within the community.

Working conditions of community workers

Gojová et al. (2018) conducted extensive research on all community work recipients in the Czech Republic. The results of the research show that one community work project most often requires full-time jobs of 1.0 to 2.0 which are most often occupied by 1 or 2 persons. On the basis of the above, it can be stated that part of the community workers are forced to manage difficult situations themselves or with a co-worker. The authors further state that in most cases the accumulation of working hours is quite common (i.e. the community worker works with the same target group also in social services or the municipality social work). The accumulation of roles increases the workload which is related to the amount of activities performed and complicates the conditions for self-reflection.

Yet such a demanding performance of the job is expected from workers with the basic education in nearly half of the cases according to Gojová et al. (2018). Most of them, however, have the support of a methodologist or supervisor. The staffing of the monitored community projects is relatively stable and changes in the position of community workers are rare. The employer of the interviewed community workers is most frequently non-governmental non-profit organizations, or municipalities or church organizations. According to the above-mentioned authors, almost half of the respondents start working in 50–150-member communities, with a significant proportion of families with children. In most cases they identify themselves as Roma people and the community is considered socially excluded.

Evaluation as a tool for self-reflection and learning

What follows from the above-mentioned is a high need for reflection and self-reflection of community workers. Reflection and self-reflection form the central basis for professional competence development (Schön, 1983). Self‐reflection is crucial for understanding how power and oppression shape the professionals' sense of themselves and their approach to work (Heron, 2005). According to Hilzensauer (2008), self-reflection can be understood as an ability to recognize one's own strengths and weaknesses and at the same time to act critically, while recognizing one's own learning difficulties or possibilities. Slepcevic-Zach and Michaela Stock (2018) state that a self-reflective attitude should be promoted through reflection, evaluation, and, if necessary, through learning behaviour regulation.

Within the community development, not only the result but also the process leading to this development is important. That is why it is necessary to broaden the view from a single product to a process. Nolan, Raban and Waniganayake (2005) state that greater chances of a positive change in the organization arise when the employees' development is ensured over a longer period of time and they are involved in assessing their own learning, having an opportunity to apply their new knowledge and skills in practice and having a trusted 'another person' to discuss their work. Therefore, it is important to develop tools and practices that support the professional development and address the issues of quality and evaluation of community work. One such tool could be responsive evaluation which emphasizes a comprehensive monitoring of programmes to support local efforts for improvement. The objectives of responsive evaluation include, in addition to improving the practical work, the development of self-reflection of participants in social situations (Smith, 2007). The emphasis is therefore put on developing the evaluation skills of all participants (Alkin, 2004). The responsive evaluator favours methods that allow new questions and issues to emerge during the evaluation, focusing on both programme activities and achieved goals and responding to user learning needs. Stake (2004) also emphasizes the role of the context in which the evaluated intervention is being conducted. Research has shown that the process of change is significantly faster in social programs when using participative evaluation and measurement (Burns, MacKeith, & Graham, 2008; Killaspy, White, & King, 2012).

Research sample

The basic research sample included all community workers in the Czech Republic who were identified through the results of a subsidy programme of community work support. It was the subsidy programme “Prevention of Social Exclusion and Community Work”, which was announced by the Government Council for Roma Minority Affairs in cooperation with the Office of the Government of the Czech Republic. In 2017, there were a total of 55 entities (including 4 municipal offices, 1 regional authority, 13 church organizations, and 37 non-profit organizations). All the organizations were asked for cooperation so the research participants were purposefully selected by institutions, when the main criterion was the experience with the implementation of community projects in a socially excluded locality.

Research method and data collection technique

The research was carried out using the method of the Constructivist Grounded Theory. The data collection technique was semi-structured focus groups, their scenario consisting of seven key areas: 1) understanding evaluation in community work, 2) reasons for evaluating community work, 3) the expected outputs of the evaluation, 4) the users of the evaluation, 5) the current state of evaluation in community work, 6) evaluation development areas, 7) quality evaluation criteria. The organization of focus groups was organized in cooperation with the Community Work Platform in the Czech Republic, which is organized by the Agency for Social Inclusion. Between May and June 2018, there were 6 focus groups with 48 participants in total.

Data analysis method

The discussions were transcribed verbatim, the data being anonymized. The transcripts of focus groups were analysed using constructivist grounded theory procedures (Charmaz, 2006). The aim of the analysis was to search in the data for categories, i.e. significant classes of behaviour, persons or events, and then to define their specific properties and compile a set of relations among these classes. The data analysis was carried out using the two main types of coding that Charmaz (2006) describes as follows: 1) initial line-by-line coding and strategies that encourage a detailed study of data and creation of concepts; 2) focused coding that allows large amounts of data to be separated, sorted and synthesized. As part of triangulation, the results were discussed with community workers at the first national meeting of the Community Work Platform organized by the Agency for Social Inclusion. The research results were used to develop software for responsive evaluation of community work.

Responsive evaluation as a means of strengthening professional autonomy

The results of the research show that community workers primarily expect strengthening of their professional autonomy from the community work evaluation software. Professional autonomy is the central category related to all other found categories, as shown in Figure 01 .

See Full Size >

According to the participants, an important condition for conducting community work is the research on the legitimization of the method, which, according to the community workers surveyed, may be helped by a) the fact that the community work is theoretically and methodically grounded: “... that we do not do this like intuitively, but we do it according to some rules and some principles and some theories…” (FG2), and b) the shared agreement of community workers on the so-called good practice: “what we agree on, what we would like to see in community work” (FG2).

Legitimization results in autonomy, which allows community workers to choose work practices. The choice is based on the cyclical process of reflection, planning, and action. “It's about capturing the process” (FG4); “… reflecting the cyclical process…” (FG2). One of the highlighted elements of the whole process is planning. “… planning, because in community work planning is particularly important, the whole team's thinking where to go… To assess what has been done successfully and what has failed” (FG4). From the above-mentioned statement it is clear that one of the important factors of planning is an analysis of past results. On the other hand, community workers resist the imperative of results, emphasizing the quality of the work process. “It's not about the result, but about how we achieve it” (FG6). They call for their autonomy in evaluating and planning their work: “... let us work for some time, don't require these outputs ... being community workers we badly need evaluation, but for other purposes… for evaluating designs, plans, and steps to make me aware of the areas that I have forgotten or missed, or that are too weak in the whole complex… ” (FG3).

To understand the process, it is necessary to be able to identify one's own actual position in the process and specific tasks related to it. “Here we were in the first phase ... here we enter the second phase ... and even have some prospect of that development ... which topics we have and where we are in those lines… it was the first eye-opener for me… this is my job… to move through all those lines, see where the topic is, how I do it, what needs to be done to get into another spiral… how I move in what the community development theory says… ” (FG2).

Another fundamental instrument for the process of reflection and planning is, according to the research participants, the existence of an adequate data collection instrument. However, in their statements, they stress that the act of interpreting data should remain within the competence of the community workers themselves. “… I have a problem with evaluation… it was a success, it was a failure… I'd be very happy if it was done descriptively, if there was a description of those changes, and only then an evaluation” (FG3).

Part of the reflection is an identification of factors that support or prevent the process of community work: “… what was a success, what was a failure, why it was a failure … what influenced that, what's behind it…” (FG3). According to research participants, the identification of factors should be carried out in a contextual framework that affects the process and results: “... that work completely depends on people and on the place where it is done .... the place, the place ... the assignment for the software is to describe as much as possible the place of work… setting the software in a particular place can have a considerable effect…” (FG3).

Another important requirement of the research participants for the process of analysis, reflection and planning is a) its consistency and flexibility: “… it is no use to do the planning once a year. More or less it should be done permanently… flexibly respond and have the information with you all the time…” (FG4); b) the possibility of comparing: “And for example to watch the fluctuations… beware, there is something different in this area than usual… in March it was like this, in June it was like that… watching the development…” (FG4).

According to the research participants, the cyclical process of analysis, reflection, and planning is a source of self-confidence and motivation not only for community workers, but also for community members they work with: “... so that people could see what they have achieved and could gain self-esteem... that is for the person himself (said by a community worker) a positive thing that now I can increase my self-esteem by telling others what we have done or what I have been able to help do…” (FG6); “… where and how he has shifted forward, that it is very motivating. Because people are forgetting, few people go through their history to see what a shift they have done…” (FG, 4), “… to motivate a worker… like to go back two, three years and look at what was useful…” (FG2).

Professional self-esteem and motivation are, according to the researchers, a prerequisite for the ability of self-reflection of community workers: “... to be able to discuss with himself, how he moves forward, how it develops him...” (FG4).

Evaluating the results and process of community work

According to the participants in focus groups, evaluation software should offer instruments to evaluate both the results of the work and the process itself. In their opinion, only then the work is not reduced to results or the process itself and both quantitative and qualitative indicators should be used to describe the results of community work. According to the researchers, the evaluation of community work results is important: a) when working with the target group, as demonstrating concrete results reinforces the self-confidence of community members, deepens their motivation, enables them to work with other community members and provide a feedback, b) when working with the public, when the results achieved can support a change in the community perception, c) when legitimizing community work as a method of work with authorities and those involved in social work and services, and d) when supporting the motivation of community workers and preventing the burnout syndrome.

In addition to the results, the software should also enable the evaluation of the community work process, which provides a better understanding of the process. It leads, according to the research participants, to a) more accurate identification of the current position in the process, b) understanding the context of processes and results achieved, c) creating a strategy for further work.

Research participants identified five processes to be evaluated by community workers and proposed results for each process to verify the success of the process. An overview of individual processes and results is presented in Table 01 .

The process of solving specific problems of the community members is, according to the research participants, an opportunity to develop organizational and educational capacities and to create a social network. In addition, according to the research participants, the individual activities should be linked to the objectives set and kept up to date.

The process of organizing the community involves creating an organizational structure for the community, whose main unit is the local group. A partial indicator of the success of the process organizing and, at the same time, a partial result of the community organizing process is the fact that community members are adequately informed about the work of the local group, participate in decision-making about goals and their achieving. The quality of cooperation of the local group is also significant according to the research participants. Its main features include joint decision-making by the local group on the division of tasks and organization of work and good relations among its members. The local group is also a place for the leadership development.

The process of the community members' development, according to the research participants, includes all forms of support for community members' strengths, mobilizing their resources and acquiring new skills. The success of this process is characterized by the participation of the community members in all the processes carried out, as well as their understanding of the nature of the community work method.

The creating of a social network is a process of creating partnerships that are the source of achieving goals. According to researchers, the quality of a social network means that its members are actors relevant to achieving the goals set and their willingness to cooperate.

According to the researchers, support provided by a community worker is an important process. It is characterized by the declining of the support, increasing quality of community relations (especially the growth of trust), the ability of self-reflection of community workers and the prevention of their burnout. An important instrument for the prevention of burnout syndrome is, according to the research participants, the monitoring of their burdens that a systematic evaluation would allow.

According to the researcher participants, the result of community work should be the social inclusion of the community members, which brings with it a higher quality of life. The indicators of social inclusion according to the research participants are presented in Table 02 . These include, in particular, the level of satisfaction with life in the locality, which is related to its image among the public, and the appearance of the site. The quality of life is influenced by the quality of relationships with community members as well as by relationships outside the community and the level of access to resources and services.

According to the research participants, the community organizing process results in a stable and functional organizational structure. One of the indicators is the ability of the community to solve problems together, a prerequisite of which is a stable local group, the ability to organize themselves and the existence of leaders who have established sufficient social contacts.

According to the research participants, community work results in the development of human resources. Above all, it is the development of communication skills, the growth of critical thinking and self-confidence. Critical thinking is, according to the researcher participants, a prerequisite for the ability to set realistic goals and strategies for achieving them, but also the resilience of the community members to manipulation. An important indicator of the human resources development is the growth of self-confidence, which is a prerequisite for activation and ability to act.