Originally posted on 20 December 2012 #LMGBlogArchive #LMGBlogArchiveProject
Evaluation is good… isn’t it?
Before exploring some of the reasons why we might not be sharing, or joining up our thinking around evaluation across the sector, it’s useful to think about why we are doing it, and whether or not it’s a good thing.
In order to give evaluation value, we must, of course, value audiences. In the last 20 years the museums sector has come a long way in its relationship with its audiences. Learning is now at the heart of everything we do; consultation and participation underpins our interpretation strategies, the object is no longer ‘king’ but a focus for conversation between the museum and its visitor, museums are no longer places to passively receive packages of knowledge and perceived wisdom but rich arenas for skills-building, nurturing understanding and inspiring creativity. They are welcoming, social spaces where everyone can happily enjoy a cup of coffee and a chat without the pressure of reading every label, aren’t they?
Sometimes they are – and there are many wonderful examples of this, but often they are not and, it is often coherently argued, neither should they be. As a fully paid up member of the fraternity[1] my vote is always audience-centred. But I cannot, hand-on-heart, say with full confidence that this is the view of all my colleagues across the many departments here, or across the sector as a whole. And, if we are still debating the place for learning, the audience and participation, then perhaps we are a long way from being able to adopt a united approach to evaluation.
Are we still learning how to evaluate?
Let’s assume we do, as a sector, value evaluation. There is, after all, much evidence to support this, with strong investment in evaluation strategies and consultative planning and development from many leading institutions. But it appears that expertise and confidence in these disciplines is still growing and has yet to be fully embedded as standard practice across the sector. Many of us are, all too often, going back to first principles and replicating practice every time we begin a new piece of work when the answers are often already there if we know where to look for them. I wonder…
How many institutions have a permanent evaluation team?
How many institutions prioritise audience and evaluation research over content and collections research?
Why national projects such as the Arts Council England funded Schools and Museums initiative require partners to undertake individual evaluation rather than evaluating the project as a whole?
Do we value practice-based expertise as highly as research-based expertise?
This is not an exercise in finger-pointing, simply a reflection that perhaps we are still learning how to evaluate.
Sharing is good…isn’t it?
Why aren’t we sharing..?
So, if we are still learning how to evaluate as a sector, can we be expected to know how to share as a sector? Or to have the confidence to share something we don’t fully understand or embrace? Is it that we won’t share, or that we simply can’t? Or – most likely – a bit of both?
At this point, it is important to point out that there is actually some sharing going on. Strands of recent MAand GEM conferences, the Visitor Studies Group training and academic studies from leading centres in evaluation and Museology [such as Leicester and Warwick Universities], as well as individual organisations have all had a good go. In my own field of museum learning I have found colleagues from other institutions only too willing to share both their practice and their findings – a quick email or phone call has often yielded fantastic results and, in many cases, built new working relationships. However, a quick Google search has been less successful. Perhaps all this points not to a lack of willingness to share but, perhaps, to a lack of opportunity or structure for doing so in a coherent and sustainable way.
Here at the Royal Museums Greenwich [RMG is made up of: National Maritime Museum, The Queen’s House, Royal Observatory, and the Cutty Sark and has a 2 million-strong collection] we have gone on a bit of an evaluation journey [or ‘voyage’ – we are maritime museums after all] over the last few years. In fact, we are still on it, still learning, and long may the voyage last.
Even within my own organisation we have struggled to share: Did your evaluation ask the right questions for my project? Do I need to fully understand the project before I can fully understand the evaluation? But that’s a digital project and this is a whole gallery! Have I got time to look into this in depth with everything else going on?
The answer is, of course, a Royal Museums, Greenwich-wide evaluation strategy which all of these findings feed into and out of. To make sense of all our visitor research and map it in such a way that evaluation for the Maritime Art Collection and the Queen’s House yields useful information for the Royal Observatory and Astronomy; that evaluation outcomes for digital interactives in the National Maritime Museum are useful for Cutty Sark.
And if that is indeed the answer, then can the wider museum sector effectively share evaluation without a nationwide strategy?
An interesting ‘sharing’ analogy came recently during the research and front-end evaluation [oh yes!!] phase of a forthcoming Children’s Gallery project here. In a workshop with Jo Graham [Learning Unlimited] we learned about the characteristics of different types of play and the evolution of ‘sharing’ in children. Very young children don’t share – they play in parallel, often the very same game with the very same toys, each brumming the same red car along parallel pieces of carpet. As pre-schoolers they become more social and friendships begin to form. By Key Stage 1 they are developing a strong sense of friendship and are often well-versed in the joys and advantages of cooperation and sharing [well, some of them…]
Perhaps, as a sector, we are in our infancy when it comes to evaluation. Perhaps as we grow into confident, well-adjusted, joined-up evaluators we will cease working in parallel – following the same methods and asking the same questions as each other – and begin to cooperate and collaborate.
Or is it all far less interesting than all this – is it simply a question of resource? Which brings us back to ‘value’ again. Do we value evaluation highly enough to invest the necessary time, money and people in strategic, effective, long-term, joined-up evaluation? In this age of austerity for the cultural sector it is still, sadly, commonplace for learning to bear the brunt of the cuts. If this is an indication of where our priorities lie then perhaps that says something significant about how much, as a sector, we value evaluation and the voice and opinion of our audience.
On the bright side, however, key funding organisations value evaluation extremely highly, demanding ‘evidence’ of need for any funding application and ‘a fully comprehensive evaluative report’ at the end of a project – often withholding actual cash until they see it in black and white!
And there other questions about sharing evaluation:
Is it because evaluation is not always valued as ‘proper research’?
Is it because we still value the object over the audience, or our own expertise and knowledge over that of the audience?
Are we afraid that we will expose our failings and be found woefully lacking?
Should we be sharing beyond the sector and, if so, how far? How about our recent secondary evaluation? Should we share it with the Times Education Supplement? Teachers’ and Headteachers’ conferences? Ofsted?
Are we seen as valuable contributors to the wider [formal] learning debate?
I feel I could go on and on and on….
So how should we share?
I have recently been involved in two significant evaluation processes for the National Maritime Museum with Nicky Boyd. One to support the development of the Museum’s forthcoming ‘Nelson, Navy Nation’ gallery, and one to, once again, try to solve the sticky problem of why we don’t get more visits from secondary schools. Both evaluations yielded fascinating results, which I am keen to share. But shouldn’t sharing go beyond simply posting the reports on GEM or sticking them on our website? And are the results really that fascinating – or just fascinating to us and the particular issues we face? If evaluation has taught me anything, it’s to ‘ask the audience’ so, answers on a [digital] postcard please: would you be interested in the findings of these evaluations and how would you like them disseminated? And have you got any interesting stuff I could have a look at?!
Author | Anna Salaman, Head of Formal Learning, Royal Museums Greenwich(incorporating the National Maritime Museum, Royal Observatory, Greenwich, Queen’s House and Cutty Sark
[1] This is not an official organisation – simply a way of expressing that I am professionally dedicated to museums learning
コメント