Posts

Don’t waste efforts: how to make social innovations produce robust (knowledge and) evidence

[Editor’s Note: This post has been written by Wenny Ho, researcher and facilitator based in the Netherlands. This post is based on her article titled “Evidence creation through Knowledge Integration“.]

Increasing the variety of perspectives has proven to deliver more sound knowledge. Knowledge Integration (KI) is the process of creating a common representation of a subject by bringing together knowledge coming from different perspectives and parties. Designing this process in a methodological, theory-rooted way, the knowledge produced can also serve as more robust evidence.

knowledge integrationIn recent years, there has been a growth in social innovation initiatives undertaken by actors including public agencies and civil society organisations. Generally, the issue at hand is of a complex nature, with an increasing diversity of actors involved, and of approaches used. By nature, a social innovation initiative has an experimental character. Complexity theory then highlights the importance of following an emergent approach, both in design and in implementation. Simply stated, instead of attempting to impose and follow through a pre-designed course of action, an emergent approach requires a gradual adjustment of the course of action based on a deepening insight regarding the path to a goal. Beware: this does not imply a ‘laissez faire’ approach. On the contrary, an emergent approach is characterised by a spiraling process that should be both systematic and purposeful.

Nevertheless, notwithstanding the growth in social innovation initiatives, the validation of the approaches used is lagging behind. I have been working for many years in M&E and social learning, but unfortunately, have seen few organisations and networks that follow sturdy principles in their quest to achieve evidence-based change and innovation. Generally, staff responsible for relevant organisational systems including M&E, Learning, and Knowledge Management are not tuned to, by means of validating approaches and methods, systematically moving initiatives from an experimental or pilot stage to a more institutionalised or upscaled level. This is especially the case, when multiple actors are involved making it necessary and worthwhile to follow knowledge integration (KI) or knowledge co-creation (KCC) approaches.

The working paper Creating robust evidence through Knowledge Integration (Ho, 2013) [1] deals with this multidimensional challenge: emergent design, multi-stakeholder change process, validation and evidence. Firstly, it describes how to undertake this tightrope walk of deliberately phasing an intervention process while avoiding the trap of prescribing next steps. Secondly, it takes as its basic premise the focus of a multi-stakeholder Knowledge Integration approach, that bringing together perspectives from different actors can generate more robust knowledge. Thirdly, to produce knowledge that is robust, the paper then sets out to describe an emergent process of intelligent, intentional sequencing. This sequencing constitutes of the merging of two helixes that draw from theoretical streams, thus providing it with a theoretical foundation:

  1. The organisational learning helix: Single-, and double-loop learning processes (Argyris, C& D.A. Schön, 1978);
  2. The organisational change helix: Freeze-rebalance-unfreeze (Weick, K.E. & R.E. Quinn, 1999).

Emergent design of a multi-stakeholder social innovation process, by using knowledge integration principles may still be somewhat unfamiliar. Nevertheless, the methods and instruments used in KI approaches have already passed through various cycles of testing.  Examples are negotiation tools, open space methodology, meta-reviews or participatory appraisal instruments. Using these endows a social innovation process with tried stepping stones. It allows one to concentrate on the more unfamiliar aspects: how to lay out those stepping stones in an emergent way in order to create more robust knowledge that can also count as evidence in a complex situation. Hence, fourth- and lastly, the paper proposes to use criteria to judge the robustness of scientific knowledge to design a process that generates evidence which can also be graded in those terms. The criteria used are: relevance, corroboration, veracity and validity (Rieper et al., 2012: 2)[2]. This purposeful sequencing produces lessons that can be used as rigorous evidence, because it is created by methodically learning from different knowledge perspectives. For example, methodical sharing or consulting with a set of selected stakeholders contributes in corroborating a previously isolated experience or enhancing the external validity of a conclusion reached under one particular set of conditions.

The potential offered by such a theory-based, emergent process is significant: connecting learning with changing can generate both knowledge and evidence; doing this systematically can generate knowledge and evidence that are both more robust, especially when the desired change concerns involves multiple stakeholders and complex societal issues.

Argyris, C. & D.A. Schön, 1978. Organizational Learning: A theory of action perspective. Reading MA: Addison-Wesley.

Rieper, O., F. L., Leeuw & T. Ling (editors), 2012. The evidence book: concepts, generation, and use of evidence. Comparative Policy Evaluation, Volume 15. Transaction Publishers, New Brunswick, New Jersey.


[2] Rieper et al. (2012: 2):

  • Relevance: in relation to a given assertion
  • Sufficiency: in sense of corroboration with other instances of the same kind of evidence or other kinds of evidence
  • Veracity: the process of gathering evidence has been free from distortion and as far as possible uncontaminated by vested interests
  • Validity: internal validity refers to how true inferences are regarding cause-effect or causal relationships; external validity refers the degree to which the conclusions in your study would hold for other places, persons, times etc.