Participative Evaluation. Surveying Children and Young People as Participants in Cultural Programmes.

Vera Popper

In the evaluation of cultural projects and cultural programmes, surveys of the actual target groups, namely children and young people, are very rare. The reported effects of such cultural initiatives are often based solely on the suppositions and observations of the adults involved.

In order to avoid this weakness, the evaluation of the programme p[ART]-Partnerships between Schools and Cultural Institutions was performed by means of a participative procedure that made it possible to take into account the specific needs of all the participants. In a workshop with the adults responsible for the programme, it was first determined which indicators could be used to recognise whether the cultural programme had achieved its intended objectives with respect to the participating pupils, so as to provide a basis for subsequently developing a questionnaire to be used in surveying the young people themselves. The questionnaire was built around psychological constructs, and particular care was taken to formulate the questions in such a way as to make them easily understandable for the target group of children and young people.

At the methodological level, Kirkpatrick’s time-tested “four-level” model (2006) – reaction, learning, behaviour and results – was used to ascertain whether and to what degree p[ART] had achieved the anticipated effects. In this model, an effect at the first level, i.e. a “reaction” (acceptance, satisfaction), is the precondition for being able to detect effects at the higher levels.

One of the constructs that were measured in the context of the evaluation of the reaction level was “interest”. The emotional aspect of interest (“I enjoy p[ART]”) was taken into particular account here, because this is highly significant where new activities are concerned: In order for participants to really get involved with the content of a programme, they first have to react positively to it. The evaluation results showed that the children and young people reacted very positively to p[ART] (96% of 296 pupils). The objectives of the p[ART] programme were to teach certain things (which varied depending on the partnership) and, at the same time, to have the children and young people participate in shaping the partnership and also to promote their talents. The opinions of the children and young people show that these goals were achieved: Over 80% stated that in p[ART] they were able to co-determine the project and do something themselves. The majority (90%) reported that they had had new experiences and learned new things (ca. 75%).

Examples like these show that it is certainly possible to survey children and young people in the context of evaluations in order to examine the effects of cultural programmes, as long as such surveys are based on the latest findings in educational psychology and evaluation research.

Vera Popper has worked as an evaluator, work psychologist and organisational psychologist since 2000. She heads evaluation projects in the fields of e-learning, psychological interventions, change management and personnel development. She has taught project management, evaluation and systemic consulting at the University of Vienna since 2002. She also works at teacher education universities in the context of in-service programmes and holds seminars on evaluation at various educational institutions.