Evaluation framework

As communities and networks go more mainstream there is an increasing demand from organizations to have ways of monitoring their value. How can we make the connection between the activities of a community or network and the improved performance of an organization, institution or even a country?

In our value assessment framework, published by the Open University of the Netherlands, we identify five levels of value creation of a community or network:

  • Cycle 1. Immediate value: the activities and interactions between members have value in and of themselves
  • Cycle 2. Potential value: the activities and interactions of cycle 1 may not be realized immediately, but rather be saved up as knowledge capital whose value is in its potential to be realized later.
  • Cycle 3. Applied value: knowledge capital may or may not be put into use. Leveraging capital requires adapting and applying it to a specific situation.
  • Cycle 4. Realized value: even applied new practices or tools are not enough. A change in practice does not necessarily lead to improved performance, so it is important to find out what effects the application of knowledge capital is having on the achievement of what matters to stakeholders …
  • Cycle 5. Reframing value: this happens when learning causes a reconsideration of how success is defined. It includes reframing strategies, goals and values…

 

For a reliable picture of how the community is creating value we would have to follow the value creation across these different cycles. To do this we collect “value creation stories”. Value creation stories and indicators (qualitative or quantitative) at each cycle are complementary. The idea is to generate a value-creation matrix that combines qualitative and quantitative data into a rich picture of the value created by learning in these social contexts.

Sponsors, evaluators and leaders of communities and networks should be able to use the framework and accompanying toolkit.

Contents

  • clarifications about the distinct learning value of communities and networks respectively
  • a conceptual framework to understand the value from such learning in terms of five cycles of value creation
  • examples of indicators for each cycles
  • a template for telling stories that account for value creation
  • a toolkit for collecting data

 

Download

You can download the full text in PDF format here:


Download the evaluation framework 

Citation

Wenger, E., Trayner, B., and de Laat, M. (2011) Promoting and assessing value creation in communities and networks: a conceptual framework. Rapport 18, Ruud de Moor Centrum, Open University of the Netherlands.

 

See also

There are some resources on evaluating communities of practice and networks on the CPSquare wiki and on the KM4dev wiki. Nancy White looks at three different perspectives on evaluating communities on a video on her blog.

The first four cycles in this framework are an adaptation of the four-level model of Donald Kirkpatrick (1976, 1994), which has become a standard in the training and program evaluation literature (the fifth cycle is an addition we have made specifically for the work of communities). An early version of this adaptation appeared in chapter 8 of the book Cultivating communities of practice.

38 thoughts on “Evaluation framework”

  1. Pingback: Communities versus networks? | Wenger-Trayner

  2. Pingback: | Wenger-Trayner

  3. Pingback: Planning and Evaluating Your CoP (amplifying and other good stuff) | Full Circle Associates

  4. Pingback: Social learning at Deutsche Bank | Wenger-Trayner

  5. Pingback: Beneath the surface | Open Learning House

  6. Pingback: Working Smarter in Online Communities – Etienne Wenger at Tulser « Technology, Innovation, Education

  7. Pingback: Virtual Canuck » Blog Archive » Comments on Promoting and assessing value communities and networks

  8. Pingback: Assessing value creation? | Wenger-Trayner

  9. Pingback: A Practitioner’s Dilemma: How can I calculate the value of communities of practice? « « DediCommDediComm

  10. Pingback: Meten = Weten: Hoe kun je online leren evalueren? | En Nu Online

  11. Pingback: #betreat12 Communities of practice and learning « Jenny Connected

  12. Pingback: How does a MOOC demonstrate it’s value? « Jenny Connected

  13. Pingback: MOOC and its value | Learner Weblog

  14. Pingback: Value Creation in Communities of Practice – further insights « Jenny Connected

  15. Pingback: Digital Storytelling in the US Army « Jenny Connected

    1. This is an eye-opener of how Kirk Patrick model has been innovated to include communities of practice. I will try these ideas in my health-related community of practice.

  16. Pingback: The Messiness of Rhizomatic Learning – Words Steal My Intent | Jenny Connected

  17. Pingback: COPASAH as a Global Learning Community | copasah

  18. Jaideep SOLANKI

    I wanted to do some quality work and research on Toastmasters International, which I see is, now after reading about Communities of Practice, a best example of CoP. I am preparing for my first Pitch about my PhD to the University of Pune. Trust that all your good work would come to my help in channelizing my Thought Process. In case if I need help from you, would you extend the same ?

    1. hello Jaideep SOLANKI
      i am a toastmaster and a masters student in MBA. I will love to read your research on toastmasters as i am work on the organisation as my case study.
      Please mail me @ ubonginyang32@yahoo.com thank you.

  19. Pingback: » Strategic evaluation of network activities

  20. I found your evaluation framework through Jane Bozarth’s article http://www.learningsolutionsmag.com/articles/1019/nuts-and-bolts-assessing-the-value-of-online-interactions . She uses a chart of the framework with permission from yourselves. I would like to write a clue creation story for my own participation in a specific community and publish as a blog article. Can I please have permission to use the same chart that Jane has used in her blog article.

  21. Pingback: Strategic evaluation of network activities | Wenger-Trayner

  22. Pingback: Articulating the value of our learning activities | CEF Knowledge Hub

  23. Pingback: Augmenting a project LogFrame to better convey stories on value created | CEF Knowledge Hub

  24. Pingback: How I am collecting value creation stories | CEF Knowledge Hub

  25. Your works really provide a useful inside in assisting me with my Master degree.Allow me to adapt your frameworks and works in my local context.

  26. This is some of the most valuable information that i have found about communities of practice. in the past, i have wrung my hands on CoPs that didn’t last long. i now have useful knowledge that i can put into practice to ensure long-lasting CoPs that provide short-term/long-term value to both the individuals and the organization. i’m also reading the “cultivating communities of practice” book.

  27. Pingback: Measuring value creation in digital enterprise communities | Sei Mani

  28. Pingback: » In full bloom: thoughts on learning and community

  29. Pingback: Thoughts about Evaluating Communities of Practice & Social Learning - Action Evaluation Collaborative

  30. Pingback: KM: Event KM4Dev Seattle – Thoughts on evaluating COPs – RadarBlog

  31. Pingback: In full bloom: Thoughts on learning and community | BCcampus

  32. Pingback: Meten = Weten: Hoe kun je online leren evalueren? - Ennuonline

  33. Pingback: Communities of Practice Measurement: Focusing on the Right Metrics to Grow Your Company’s Knowledge Capital

Leave a Comment

Your email address will not be published. Required fields are marked *