Post from Bev:
How do you show that a network’s activities have changed the performance of its members in their practice? And, how do you use that information to feed into a vision of where the network should go next?
That’s how we used a new version of our value-creation framework for a learning network whose members come from different countries in Southern Africa.
For this network one project cycle was coming to an end and a new one was beginning. Not only had we collected many value-creation stories to show the connection between network activities and changes in public finance management in their respective countries, but we also used the value cycles in the framework to look at what the project had aspired to in the beginning, what happened, and what should happen in the next phase of the project.
The report I produced for the project summarizes the new version of our framework (based on the original framework, but now substantially expanded) and demonstrates how this new framework can be used to provide a strategic tool for evaluating and shaping a big social learning project.
Highlights on using this updated framework:
- What was really powerful in this session was the combination of value creation stories and indicators at each cycle. This is what the framework is about – not just value-creation stories as is often interpreted.
- In a meeting that was potentially fraught with political tensions the framework gave a shared language for understanding what had happened and for negotiating the way ahead in a productive and transparent way. In particular the strategic and enabling value cycles helped to surface some of the most serious places of disconnect in the functioning of the project.
- We collected value-creation stories for nearly four years. During this time most network members felt that collecting these stories was a hoop to jump through to keep project funders happy. But when they were put together into a strategic document based on the framework, network leaders finally realized the power of their stories for their own network learning.
- The idea of feedback loops was extremely helpful for showing how learning comes from talking about successes and failures and that both need to be integrated into the discourse of the network.
Cautions
- It is very time and resource intensive to collect and refine good value-creation stories and the data for the different cycles. We collected hundreds of partial stories; it took persistence to follow up with people to complete them. There were only about fifty stories that went through all cycles (often at a stretch) and around twenty stories that were really good.
- Next time it should be easier because people will now have some good models of stories and see the value of collecting them. But it will take discipline to turn the telling and recording of stories (and the feedback loops) into a regular network practice.
- In the real world – certainly the one I live in – the process of collecting data is much more messy than people realize. And it’s certainly not linear. It was more of a bricolage as we went back and forth between people, stories and indicators, scraping out time, improvising, and making up tools as we went.
WoW!
Great evolution of this framework. I saw some of earlier prototypes and was so pleasantly surprised at the non linear new user interface of latest spiffy model.
Thanks for walking the talk in polishing this. As a practicing evaluator working with non profit organizational systems struggling with complex social issues. I appreciate this tool and its lens on value”. It is a great alternative/adjunct to Logic Models. Will be sharing this with evaluation peers. Good work Team B!
PS – Also seems to be a “fit” somewhere with the new Collective Impact movement sweeping the landscape. Hallie Preskill @ FSG leading that systems evaluation effort.
Great work
Joann back from Mongolia ; )
Welcome back Joann. We’d love to hear what happened in Mongolia. You’re right, the framework has evolved – we’re excited by the way it’s going. It would be good to know how or evaluation professionals take it. And thanks for pointer to FSG and Collective Impact. There certainly does seem to be a fit, given their emphasis on learning. – B+E
Hi – this looks like an interesting framework. Is there a publication which explains the details of the framework and how to use it?
thanks, Stefan
Hi Stefan, our publication in 2011 tells you more about how to use it. We have developed the framework a lot since then, and will be publishing the new framework later this year. If you want to be informed, you can sign up to this group here: be-vcf@be.wenger-trayner.com. Stay tuned!
Hi Team BE,
Just wondering if the version of the framework at https://www.wenger-trayner.com/resources/publications/evaluation-framework/ is the new version or the original 2011 version. Thanks
Hi, Tony. No, the link you have in your comment is still the old 2011 version. The report on this page uses the new version, which we are currently writing up as a small book.
Thank you Team BE
Hello Team BE,
I was curious to know whether the book you refer to in this comment, and which contains the new version of this framework for the strategic evaluation of networks, has been published yet? It doesn’t seem to be listed on your “Books” page…
Nearly….!
Hi Etienne and Beverly,
I am now starting a PhD research focusing on how networks fulfill a purpose as instruments of learning – so as you can imagine, I was quite thrilled to discover your value-creation framework for analysis, both the 2011 and the 2014 version!
Beverly, you mentioned a couple times that you have been writing a book centering on the new version of your framework. I was curious to know at what stage of writing/publishing this book currently is?
I would be very eager to provide feedback and questions if you wanted to share a draft with me. Please let me know.
Hi Dorian… thanks for connecting. We are in the final version of the book and preparing it for publication. Hopefully it will be out in early 2020. We can send you a draft if you are interested.
Pingback: Planning and evaluating social learning | Wenger-Trayner
Pingback: Community of Practice Evaluation Strategy | Michelle Ockers
Hello, I am currently conducting research for my Masters on the ‘Value of Digital Storytelling as a learning activity for high school students in community museums.’ As it turned out, my research participants were limited to the community museum members, and it meant my data set from the semi structured interviews I conducted was different to what I expected. I have only just discovered your value matrix and realised that my data was really about the museum community learning, instead of student learning. Now I see you have extended this framework, and the strategic and enabler elements add further clarity to my data. When might more be published about your updated framework? Would love some further information about it – theory, case studies …
Hi Janis, thanks for sharing what you are doing. This report and the video (https://www.wenger-trayner.com/resources/planning-and-evaluating-social-learning/) are the only public documents sharing the new version of the framework.
We are currently working on the new publication, which, hopefully, will come out this year.
Pingback: Social learning – is there any other kind? | Maarifa - Communications and Knowledge Management
Do you know of anyone that has used this new framework in an educaton organization? I have worked to create a prototype of an oCoP using human-centered design that included a variety of stakeholders, so very similar to the process you described. I LOVE that this framework has the learning loops but also includes the technology (enabler) and the strategy (backbone). If I’m misusing or misunderstsnding it, then please let me know. Otherwise, I may try to make out a matrix like you did for the World Bank. Awesome work!
Are you aware of anyone using the updated framework for planning, designing, and evaluating a prototype for a national-level education network? I saw that Connected Educator used your earlier one to describe the types of value, but I LOVE that this one now includes the enabling value (such as technology) and the strategic value (such as the backbone). If I’m misunderstanding the intended use, then please let me know. This is awesome work! Thank you for sharing. And, if you know of others working on similar things, I’d be grateful if you could direct me. You are fabulously inspiring.
Sarah,
You are representing the work properly. We now intend the framework to be used for planning and sustaining social learning as well as for evaluating. We don’t know of any national education project that is using this new version of the framework for planning a network. We will be using it ourselves. to work with some regional education communities of practice, but this work is just incipient at this stage. We would like to know if you find anyone else doing it.
Thanks for your good words about our work.
Great work up there Team B.
If I may ask, how possibly could one reference this page of yours?
I am on with a work on Communities of practice…
I will appreciate a prompt feedback on this request of mine.
Thanks a million!