Developing the Research Impact Canada (RIC) Logic Model and Evaluation Framework: A Case Study Example

by Anne Bergen 

In 2017-2018, I helped the RIC Evaluation Committee develop a logic model and evaluation framework. This brief case study, originally posted at Research Impact Canada, describes that project.

Summary

This brief case study example describes a project working with Research Impact Canada (RIC) to create an evaluation framework, beginning with co-creation of a logic model.

Research Impact Canada (RIC) is a network of 16 universities dedicated to using knowledge mobilization to create research impact for the public good. In this project, a small group (the RIC evaluation committee) collaborated through four online workshops to define an evaluation framework that represented the shared knowledge mobilization activities of the larger collaboration, and aligned the framework with goals from RIC’s strategic plan.

This work expanded RIC’s past evaluation approach that focused on monitoring through counting outputs (the products of RIC activities). The current framework explains how activities link to short-term outcomes of improved knowledge, skills, and attitudes, which contribute to longer-term changes of improved individual practice, organizational capacity, and systems support.

Key Messages of this case study

  • Work together in a small group to create a collaborative and iterative logic model and evaluation framework, where the group represents the diverse perspectives of a partnership initiative.
  • Create a logic model that maps out target audiences, common activities, what success looks like (outputs, outcomes, impacts), and enabling conditions.
  • Then use that logic model to guide decisions around what and when to measure. Instead of trying to measure everything, focus what’s important and what’s feasible.

Read the full case study: http://researchimpact.ca/developing-the-research-impact-canada-ric-logic-model-and-evaluation-framework/

Secondary Dissemination

This past year, I taught a number of training workshops about knowledge mobilization, applied research, and evaluation. I have some plans to combine and collate these presentations into something exciting, but in the meantime I’m posting some of the workshop materials here as open access resources.

by Anne Bergen

The first set of workshop slides come from two full-day workshops hosted by Gambling Research Exchange Ontario (formerly the Ontario Problem Gambling Research Centre). These workshops were part of a series of capacity building sessions for graduate students and postdoctoral researchers.

The fall 2014 workshop “Skills and Ideas for #ProblemGamblingKTE” was designed  to give participants tools to carry out knowledge translation and exchange (KTE) activities. Students in this workshop:

  • practiced identifying target audiences and stakeholder groups and analyzing contextual factors relevant to KTE activities;
  • created and presented plans for engaging and working with stakeholders and decision makers;
  • translated key research messages using plain language writing and selected communication channels and social media tools to maximize research access and impact.

The winter 2015 workshop “Evaluating Problem Gambling KTE” provided participants with tools for monitoring and evaluation of research impact from knowledge transfer and exchange (KTE) activities. During this workshop, students:

  • created an evaluation theory of change or logic model for problem gambling KTE;
  • learned to consider goals, values, and context to decide on an appropriate evaluation approach;
  • identified indicators and measures for KTE evaluation. Continue reading

Worst Case Scenario: An Exercise

To get better, sometimes you have to think about the worst case scenario. This blog post is about a facilitated workshop activity that considers worst practices to support personal and collaborative change. Worst KTE practices

by Anne Bergen

What is the worst way that we can mobilize knowledge?

Knowledge mobilization* is about making information useful. It’s about:

  • helping information and knowledge flow among individuals and groups,
  • making decisions based on the best available evidence, and
  • creating relationships among people and networks.

The loftiest goals of knowledge mobilization are transformational, not just transactional. We want people not just to accept parcels of knowledge, but also to change their thoughts, feelings, and actions based on that knowledge. We want people to think and behave differently than they did before.

But there’s a problem. During knowledge mobilization training, it’s possible to get stuck on multi-layered definitions and conceptual models, leaving participants unable to answer the question:

“What does good knowledge mobilization look like?”

Yes, knowledge mobilization is about transformation of research knowledge into policy and practice. But non-specified “policy and practice” are dull. It’s hard to muster sustained enthusiasm for “policy and practice”, and it’s hard for trainees to be able to think about those terms in ways that are relatable and relevant.

Instead, a great way to start off a knowledge mobilization workshop is to think about worst practices, not best practices. That is, what could the participants do to get a really, really terrible result? What does knowledge immobilization look like?

Continue reading

Health Jam 2014: Lessons Learned for Knowledge Mobilization

Photograph of Upstream's prototype

A musical “jam” is improvised collaboration among musicians. Musical improvisation – creating something new without extensive preparation or practice – has been proposed as a model for innovation and social change (see for example the multi-disciplinary Improvisation, Community, and Social Practice project funded through the SSHRC Partnership program).

Happily, those of us who lack the skill for musical improvisation can still jam. The “rules’ of non-musical jamming come from design thinking methods and create a framework for rapidly developing prototypes of ideas designed to address a particular problem or issue. Design thinking is a “human-centred” approach to innovation – a form of structured brainstorming, testing, and iteration that repeatedly tests prototypes against the needs and attributes of end-users.

The global series of GovJam events are two day workshops aimed at building collaborative and innovative solutions to public sector problems. In turn, GovJam events were inspired by ongoing Global Service and Global Sustainability jams, with the tagline: “48 hours to save the world”. Jamming appears to be a rhizomatic practice, like the stem of a plant that grows through laterally spreading roots and shoots, with past Jam participants spreading outwards to create new Jam events in different places and with different areas of focus.

Continue reading