01. MACRO CONTEXT

Macro context refers to overarching forces at the national level that establish the “bigger picture” in which policy is made. These forces compose the general external context for each policymaking institution, including political, economic, social and cultural systems that surround it. They shape opportunities and threats for state agencies in terms of using research to inform policy in two main ways:

(1) structural factors, which very rarely change in a significant way and could be regarded as the more constant and regular outside setting of policy institutions; and

(2) circumstantial factors, which emerge with particular weight every once in a while and open up very specific windows of opportunity for change.

02. INTRA AND INTER INSTITUTIONAL LINKAGES

Still part of the macro context, these are two particular types of relationship that exert significant influence over how knowledge interacts (or not) with policy. One is related to internal relationships between the public institution and other related government agencies. The second one relates to the interaction with relevant users and producers of knowledge who can affect or be affected by policy design and implementation.

03. CULTURE

Culture is the set of shared basic assumptions learned by a group. These assumptions have worked well enough to be considered valid and are therefore taught to new members as the correct way to perceive, think, and feel the organizational problems. This culture creates the daily context for practice, including incentives and motivations that affect what research can mean for policy processes.

Several diverse (and sometimes conflicting) cultures can be in operation within an organization, implicitly reflecting different values or worldviews. These opposing cultures may lead to dysfunctional or suboptimal working relationships, which can clearly affect how and what research is used. Differences frequently emanate from contrasting views on how the state should function.. Indeed, the state is not a monolithic entity behaving in a unique way with state representatives thinking similarly; on the contrary, heterogeneity among public servants is the prevailing rule and this explains the difficulty of their interactions with third parties which are also consequently different and sometimes even opposed.

Organizational culture is a powerful motivating force: by embodying the values sanctioned by the organization, the culture frames the boundaries of acceptable attitudes and behaviour and creates a shared ethos.

04. ORGANIZATIONAL CAPACITY

Organizational capacity is the ability of an organization to use its resources to perform. It is composed of two core components: human resources (including leadership and management); and the legal framework or set of rules that determine how resources can or cannot be used, and establish interactions between individuals that are part of the institution. Internal capacity plays a pivotal role in making the use of research possible (or not) as well as how it is taken up.

05. MANAGEMENT AND PROCESSES

Daily work in an organization is largely dependent on ongoing processes and policies, and how routine decisions are made. Organizational management and processes refers to how each governmental institution organizes its work to achieve its mission and goals, from planning to evaluation. The way it is managed and the processes and mechanisms that are established to enable members to fulfil their roles and responsibilities can open up (or not) chances for evidence to interact with policy discussions and decision-making. Management and processes are more prone to promote the use of evidence if there are long-term and consistent policy, regulatory, budgetary frameworks that support domestic research and development institutions.

06. OTHER RESOURCES

Refers to the set of resources that are critical to an organization’s performance and to achieve its goals, including budget, time, infrastructure and technology.

Leadership

The role of leadership is one of the main enabling factors for change in terms of enhancing the use of research in policy. Continuity and stability of high-level leadership who are clearly engaged with facilitating the use of knowledge in policy appears to be critical. For example, how much those at the top appreciate research determines the investment in research units or human resources.

Leadership and organizational culture are intertwined: cultures begin with leaders who propose their own values and assumptions on a group. If that group is successful and the assumptions come to be taken for granted, we then have a culture that will define for later generations of members what kinds of leadership are acceptable. But as the group runs into adaptive difficulties, as its environment changes to the point where some of its assumptions are no longer valid, leadership comes into play once more. Leadership is now the ability to step outside the culture that created the leader and to start evolutionary change processes that are more adaptive. This ability to perceive the limitations of one’s own culture and to evolve the culture adaptively is the essence and ultimate challenge of leadership (Schein 2004).

Political leadership is crucial to promote changes in terms of the role of knowledge in policy. Without commitment and drive from them, changing the internal rules of government (formal and informal) is not possible. Leaders who have understood and addressed a wide range of incentives and pressures related to using evidence in a policy processed, have been key in terms of enabling this change. Leadership also emerged as critical when building skills, fostering expectations for using knowledge to inform policy, providing “moral messaging” throughout change processes, reinforcing norms, and conducting participatory decision-making.

Senior Management

Leadership should be distinguished from management or administration; one can argue that leadership creates and changes cultures, while management and administration act within a culture. There are clearly leaders emerging from senior management who influence how and if knowledge is used within policy, and thus contribute to generating a new culture. These are more likely to emerge when higher leadership in government or a political party are already pushing for a broader use of evidence in policy. Senior managers often play an important role in terms of interpreting the evidence as well as deciding who advises them and how. For instance, some opt to form expert advisory committees while others rely more on informal relationships with experts they trust.

Human Resources

Government agencies do not always have adequate resources to hire and retain the best talent with the adequate skills and capacities to conduct or identify the type of knowledge needed at different stages of decision making and policy implementation.. This includes the capacity of bureaucratic agencies to manage programmes successfully, including expertise, personnel, political (elite) support, and other resources.

Among human resources, a key role is played by advisors who bring practice-based knowledge. Compared to research-based knowledge, advisors usually bring knowledge related to the different operating contexts, and experiences of implementation at different government levels. This type of knowledge becomes increasingly useful in the implementation of large policies, since there is usually a need to adapt them to local priorities and capacities.

Abilities to use evidence are usually related to the educational background of policymakers, which has also been positively correlated with their use of evidence. Officials with a graduate diploma or postgraduate degree are typically more likely to use research in policy related work, and at the same time present a greater capacity for acquisition and assessment of research evidence for policymaking than those with lower academic qualifications. The profession of the policymakers also influences their predisposition to accept and incorporate empirical evidence. Additionally, in some countries, where local research is still scarce, the ability to access research in other languages (mainly English) can also significantly affect policymakers’ capacities to use it.

Political and communications skills have also been proven to contribute to the effective use of research. Policymakers who are politically savvy and find effective ways to convey evidence, by identifying the right messages, moments and entry points, are usually more successful in convincing others about the need to consider research for policy decisions.


On the other hand, high staff turnover hampers efforts to institutionalize the use of knowledge. This is especially relevant for the use of existing data as well as information-generating processes. A lack of permanence may imply the loss of valuable information for the decision making process due to the fact that it was not computerized or communicated in good time. Also, incoming policymakers, may disregard existing evidence for the sole reason that it was generated or commissioned by another ruling party. At the same time, however, changes are and should be viewed as opportunities, as the new administration may take more interest in information generation and use.

Legal Capacity

Policymakers are constrained or enabled by regulations in the commissioning and conducting of research. Often regulations are so intricate and complex that researchers and smaller research institutions cannot fulfil all steps and requirements, and thus cannot become regular suppliers

Motivations

Personal drivers are also relevant in terms of the organizational culture and how incentives are taken up or not. People are motivated from within, by interests, curiosity, care or abiding values along with external factors such as reward systems, grades, evaluations, or the opinions they fear others might have of them. Conditions supporting the individual’s experience of autonomy, competence, and relatedness arguably foster the most volitional and highest quality forms of motivation and engagement for activities, including enhanced performance, persistence and creativity. Organizational culture plays a significant role, by determining the degree of consistency and interaction between incentives and motivations.

Among main motivations the following are worth highlighting:

Competence and relatedness. There are many concerns policymakers have as they develop, implement and evaluate policies besides having evidence on whether they will work or not. For instance, cost, acceptability, distributional effects, risk, and the need to maintain good relationships with civil society, the private sector and other government institutions.

Within a government organization, managers will be both co-operating and competing with one another as they press for personal and professional advantage, while at the same time trying to advance the policy on which they are working. Internal perceptions of colleagues and other government agencies also matter. Policymakers take into account how colleagues and superiors value the evidence and are also concerned about the perceived quality, relevance and usefulness of this research.

Ownership. A sense of ownership of the evidence produced emerged as a very important driver for policymakers engaged in promoting its use. When policymakers are involved in the design phase of research projects, this increases the likelihood of the research being used by securing early “buy-in”. It can also help to tailor the findings to the policymakers’ needs. When demand for evidence arises within a country’s political economy as opposed to structures external to the system (such as from donors), there is increased ownership. On the other hand, attempts to build upon imported or “home-grown” evidence are likely to encounter resistance if a sense of ownership does not exist in those expected to adopt the change.

Openness to change

Openness to change refers to the extent that an organizational culture may enable critical inquiry, curiosity, and support risk-taking and innovation. Bureaucratic logic often prevails: bureaucrats reinforce the idea that everything is fine (“It has always been done this way”), and that there is no need for change or innovation (there is often significant interest in maintaining the status quo, which benefits a specific group of stakeholders). This gives preference to existing frameworks in understanding policy problems and, therefore, favours only evidence confirming the efficiency of current practices.

The political economy of change must not be underestimated. Does the organization have a sufficiently flexible structure to enable the development of new groups or units, which will be effective in seeing through a policy change? Does the institutional environment allow any restructuring? Do resources exist within an organization, or can they be gathered, to respond to a new way of working?

Finally, openness to change is closely linked to a government’s willingness to admit failure. This is particularly relevant for evidence arising from M&E efforts. It is also related to the overall culture of critical thinking.

Beliefs and values

Beliefs and values play a crucial role at two levels: (1) how evidence/knowledge and its conveyor is listened or not according to the policymakers’ existing set of values related to the specific policy issue; and (2) the overall appreciation within the organization of the role of knowledge in informing decisions (which is also influenced by the social appreciation of science as noted among the macro-contextual factors).

In terms of the first level, we need to acknowledge the weight given to prevailing narratives and discourses on a specific policy issue. These, along with societal norms, can lead to policymakers’ reluctance to acknowledge an issue because it is stigmatized by prevailing societal norms and values implied in these narratives. Similarly, history and tradition can bestow legitimacy on a situation whereby existence of current practice can be deemed evidence that it should continue to exist. The extent to which it is socially acceptable to challenge power structures is also a strong driver for change or the lack thereof.

Regarding the second level, it is important to consider the extent to which an institution “values” evidence. There are agencies that, due to tradition, the will of politicians involved in their operation, or personnel characteristics, have developed a higher preference for processes that allow for a more efficient information management – from its creation to its use, and including processing and communication. On the contrary, the collection and appraisal of research is in some settings regarded as “non-work” amongst those who have needed to appear to be taking action. Indeed, an organizational culture of doing can become a barrier; enabling staff to undertake and familiarize themselves with research requires a balance between thinking and doing.

Incentives

Values and beliefs bear weight on existing incentives within an organization. Incentives depend entirely on the promise of something external (while motivations are internal). An activity can have a motivation of its own, called “intrinsic motivation”, which leads to the execution of the activity in the absence of external rewards or incentives. Furthermore, intrinsic motivation may be diminished by extrinsic incentives such as performance cmonetary rewards. In this way, mid-level policymakers will be more likely resort to evidence to the extent that it is considered important to their workspaces – that is, to the dynamics of decisions and to their leaders of their organization.

Promoting an institutional culture of learning from mistakes, rather than one where mistakes are punished, can help overcome a compliance-based system and encourage departments to make use of evidence to actively solve problems (rather than avoid them). If research is valued by leadership and senior management, they will establish processes that require, enable and/or reward civil servants that commission or use it, including the creation of specific job positions or roles and responsibilities within job positions. The role of leadership and senior management is crucial in the determination of the rest of the concrete incentives.

The lack of performance management indicators, which monitor service provision, and/or ongoing evaluation make efforts to promote the use of research more challenging. Additionally, some organizations or officials often do not share information for fear of it being used to assess their performance.

The existence or absence of electoral incentives to use research evidence along with political costs also bear significant weight.

Remuneration is another strong driver. Low remuneration and the dependence on allowances do not only affect individuals: they also have serious negative impacts on work practices and workplace relations, and tend to entrench the status quo. New ways of doing things are resisted in favour of what will maintain the expected levels of remuneration, even if inefficient. This is exacerbated in institutions with an incentive structure that encourages researchers to scramble for short-term consultancy work from donors and government rather than focusing on longer term projects that may provide the opportunity to strengthen their research skills along the way.

Opportunities to upgrade knowledge, skills and qualifications through capacity building have also been highlighted as concrete incentives to promote the use of research.

Monitoring and evaluation

Monitoring and evaluation (M&E) are two increasingly implemented processes or practices that help generating internal knowledge. Both are key aspects of public policy management because they make knowledge accumulation and organizational learning easier. The types and uses of evidence in M&E are more specific and more clearly delimited among government institutions compared with other potential uses of research, such as framing a policy problem, evaluating policy alternatives, etc.

However, the role of motivations and incentives in terms of how M&E takes place should not be underestimated: research results that inform policy design stages are seen as valuable inputs, while evaluation results – when negative – can create a great deal of conflict for public servants, implying that part they are not doing their work properly or efficiently. Consequently, they tend to be reluctant to adopt or use those evaluations results. This is linked with the overall organizational culture and how it genuinely embraces learning or not. More recently, many governments in developing countries have achieved a better understanding of the value of M&E in helping to accurately determine which investments and public interventions work, which don’t and why.

However, the remaining challenge is the institutionalization of these systems in the general dynamics of public sector information management. That is to say, there is still a significant way to go towards system creation, suitable operation and intensive use. Further, evaluating policies does not ensure the results of the M&E will be used to change policies. It is a great step forward to have M&E units in the public sector –and to have special committees or commissions with private and public sector representatives choosing the type of evaluations that a country needs, or independent bodies conducting the evaluations to ensure its independency and assuring its quality. But when evaluation results emerge and are shared with policymakers and public officials, they are not obliged to use these results or to adopt the recommendations. Unless there are processes in place to ensure the use of M&E findings, policymakers have to be convinced about the power of such recommendations in order to use them.

The recommendations also need to be feasible within the bureaucratic rules of the public sector; most evaluations end with recommendations that then need to be translated into operational actions within the public sector, and in that process, sometimes they lose their goals. The adoption of a recommendation implies changing operational rules in the public sector and such processes imply time, coordination (within the sector, but also with other sectors or governmental levels), negotiation and validation of its legality and validity.

Communication processes

Communication processes are critical for relationships with other stakeholders. Often due to the variable quality of research outputs, or the dispersed systems for research commissioning and production, communication and coordination of research within each institution is poor. Another crucial point about communication is the frequent lack of knowledge and coordination among different ministries and agencies to jointly establish common research topics. Doing so would mean that all could be engaged early on in the sharing of what will be produced or commissioned, as well as in making decisions about how and when to involve external stakeholders throughout the process.

On the positive side, some institutions are very effective in communicating their research activities to diverse stakeholders, including periodically producing formal research reports as well as attractive documents with evidence of analysis and interpretation. Others have developed concrete and sophisticated mechanisms to interact effectively with a diverse group of stakeholders who can support the use of knowledge in policy, actively listening to their demands, priorities, proposals, etc. Moreover, although some formal communication processes can help towards further use of available knowledge, the role of key people of influence should be carefully considered, including how to build multiple-level dialogues with them.

Positions, including division of work and roles and responsibilities

Another crucial aspect of process is linked to how roles and responsibilities are defined and distributed – for example, whether they contain references to an expected use of research. In this sense, when there is an established functional specialization of policymakers, those within those roles are expected to be specialized experts in the substance of the policy domain and are consequently more likely to generate or listen to research findings.

Certain divisions of responsibilities within government bureaucracies limit the use of evidence, arguing that individual civil servants are compelled to focus on small, specific areas of policy activity, making it extremely difficult for them to engage with ideas beyond their immediate area of responsibility.

On another hand, the creation of positions such as Policy Analyst, can increase the demand from government for knowledge and promote the incorporation of a new cadre of well-trained policy staff. This in turn could improve uptake of evidence into policymaking. The effectiveness of this, however, could be undermined by problems with inappropriate incentives born out of the division between administrative and functional staff, which has been identified as a major constraint to the use of knowledge in policy.

Degree of systematic planning

The existence of overall national strategies and plans encouraging the incorporation of evidence, or that have strong M&E components, will push institutions at different levels (including subnational and sometimes even local) to use evidence and perform some type of monitoring and/or evaluation. This is especially applicable to the planning stage. The planning processes within a specific institution can also open up opportunities to incorporate research – for example, when requiring diagnostics or baselines to support a new programme or policy. A mid-term plan can also provide sense and clarity for potential research agendas, and the need to systematically gather information to monitor and evaluate it. It can also enable continuous feedback loops between research, policy, implementation and monitoring.

However, in some cases government’s planning and resourcing model can work against the ability of government research units to be responsive and flexible. Links between planning and budgeting are also crucial in terms of opening up windows of opportunity to fund and commission new research.

Existence of formal processes to access and use evidence in policymaking

Another key factor is the existence of concrete and external processes that promote the use of evidence, such as evidence-based peer-review processes for internal policy briefings, parliamentary committee inquiries that require parliamentarians to gather evidence to scrutinize government policy, and requirements for spending bids to be supported by an analysis of the existing base.

In terms of formal processes, it is more frequent to find existing systems that aim to continue generating and using data that already exists (for example, national household surveys) than formal processes to commission new and/or dedicated research. The use of information tends to be more institutionalized for existing and systematic data gathered by the state while, in contrast, research produced by external stakeholders is used to inform a specific policy discussion, design or an aspect of its implementation.

Time availability

Lack of time also inhibits policymakers’ capacity to use evidence. Indeed, timescales for decision-making are frequently incompatible with those for research. This will vary according to policy-decision methodologies and cycles. There is often a lack of synchrony – for instance, when evidence comes after the decision has been made and has missed the window of opportunity. This can only be avoided when there is a constant churning of evidence, to ensure that information is available for when the opportunities for use arise.

Availability of time will therefore vary according to the type of decisions and how much agreement there is already in place in terms of what is the policy problem and which are its potential solutions.

It is also worth distinguishing between systematic and ad hoc opportunities for research use. An example of systematic use is during election cycles, when many policy decisions are made and evidence has the chance to significantly influence the election manifesto. Ad hoc decision-making, on the other hand, happens when a specific policy issue or problem scales up in the political agenda due to a crisis, a social movement, pressure by a special citizens’ group or media campaign, etc. This leads to many decisions being made in haste.

Technology

Technology plays a pivotal role in enabling the flow and production of policy-relevant knowledge. For example, internet access is poor or unreliable in many low- and middle-income countries. Additionally, many states have not yet computerized all the information they have available and departments operate as silos of information. Having digital tools to manage information could allow for a more efficient and timely use of knowledge but the availability of this infrastructure does not necessarily mean it will be effectively accessed or used. There is therefore also the need for a comprehensive capacity-development effort to ensure the use of new technology.

Budget committed to research

One of the factors most cited as a restriction on the degree of knowledge generation and utilization is the limited funding available for research. This limitation leads to inadequate methodological capacities and weak analytical skills among public servants, combined with the inability to hire highly qualified researchers or research centres to conduct specific studies. This is worsened by a frequent divide between the research and the policy agenda within the same governmental agencies.

Even when funders recognize the importance of budget support – which often carries high hopes as a motivator – there are other issues that frequently limit the effectiveness of financial support as catalyst for reform. Often those countries where donors have the most financial leverage have the furthest to go to improve institutions, but the least capacity to implement change. An uncoordinated set of donors with diverse and less focused demands and priorities, each providing financial support to suit their own individual priorities, does not allow for concerted and integrated strategies to promote the use of knowledge in policy. Sometimes tight schedules for commitment and disbursement of lending hinder government institutions in developing a sound plan that aligns its need with the existing offer and opportunities.

Budget flexibility and availability is not only key to the generation of required evidence but also for applying findings and results, as lack of resources significantly affects how useful these can be. For example, even if evidence suggests a concrete programme is working and it should be scaled up, governmental agencies often do not have the resources to do so.

Existence of a knowledge infrastructure

The existence of a good knowledge infrastructure in government departments is likely to affect the levels at which policymakers consult and use research. This may be in the form of research units within government, knowledge management strategies that provide some sense and order to the existing information, or the existence of an electronic knowledge base of past policy papers. Indeed, in public agencies’ hallways there is a lot of circulating information, which is not systematic and thus hinders it being better leveraged. This is also frequently linked to obsolete processes or tools.

For this data to be available to decision making processes, it needs to be stored in some kind of system that orders and makes sense of it. These structures, often called information systems, are instruments for knowledge generation and transfer, which support programme and project planning, monitoring and evaluation. Information systems are usually used as support for other areas, queries (administrative acts, controls, reviews) or, in the end, for filing. For these systems to be efficient, they must address users’ needs. The usefulness of information systems lies in their: (1) reliability – that is, decisions are made based on the generated information; and (2) timeliness – that is, information must be available when decision-makers need it. Another significant value is the feasibility of compiling data, what is not only related to an attribute of existing information systems but also to the technical and administrative ability of chief operating officers and institutions to use them.

Degree of power distribution in the political system

The degree of power distribution in a society involves factors such as the level of centralization versus decentralization of policy design and implementation, the checks and balances (or their lack of) between the executive, legislative and judicial branches, and the roles and responsibilities of some specific government agencies, which can enable more open entry points for research to be involved in policy making processes or, on the contrary, represent several constraints for it.

For instance, research is most likely to be used in open, transparent, democratic contexts with strong academic and civil society institutions, and free media. On the contrary, centralized political systems are likely to be less open to the uptake of research findings than decentralized systems. Moreover, the concentration of power prevents pluralistic debate and thus the need for evidence to support competing views.

Another important factor is the weight of congress/parliament in the decision making process: a key determinant of the effectiveness of legislators in using research is the extent to which other important power holders –most importantly executives and parties– cede, lose, share, exchange or let slip the power they hold.

The distribution of power among political parties and the degree of consolidation of the party system are also very important: institutionalized systems are associated with programmatic parties and a programmatic policy is more easily linked with the use of research than a non-programmatic policy.

Consultation and participation in policy processes and accountability

A greater degree of democratic openness generally leads to the creation and institutionalization of norms on consultation and participation in policy processes, which directly influence how evidence impacts policy. Increased consultation with policymakers has been identified as helping to establish a “conducive environment” for research use. These norms are often present in political systems with greater levels of accountability: when policymakers are held accountable for the “quality” of their decisions and scrutinized by other state or civil society organizations the need to inform decisions with available evidence from different sources is greater.

When policy processes are more open and transparent, there is a greater role for interested parties – including campaigning and lobbying groups, coalitions, and citizens – in adding to and contesting evidence, as well as their perceptions or interpretation of this evidence. This also includes the role of the media in overseeing policies, requesting evidence and framing the parameters of the debate.

Another relevant aspect is state regulation with regard to lobbying and advocacy, and how this enables or not diverse stakeholders to legally and equally access important discussion and decision-making spaces.

Knowledge regime

Knowledge regimes are the organizational and institutional machineries that generate data, research, policy recommendations and other ideas that influence public debate and policymaking. Within this, there are several key aspects to consider.

• Availability of public data and information. Evidence-based policies require quality data, a critical contribution to informing decision makers, improving public debate, and tracking the actual implementation of policy. Where there is shortage of basic statistical and other data, which are fundamental to drawing reliable conclusions, policy arguments may be more likely to turn on issues of power and prejudice rather than evidence. This is also linked to the existence of laws on public and private access to information. Related to this, is the degree to which existing research is available, accessible and relevant to policymakers.

• Funding of the knowledge sector. In countries that have invested highly in research and development, the government is not the only source of demand nor is it the only source of supply. Demand stimulates domestic capacity, both government and private sector, to provide dynamic and expert research capability. On the contrary, a low level of expenditure can result in government dissatisfaction with the volume, quality or applicability of research, and lead to excessive reliance on foreign technical advice.

• Labour market. The degree of development and involvement in policy of those working at universities, private research organizations and think tanks should be taken into account when looking at the entire knowledge regime. Many developing countries face a significant lack of individuals with skills in analysis and interpretation.

• Critical thinking. Low levels of evidence literacy in policymaking institutions have been linked to low levels of evidence literacy in critical inquiry in students. Lack of global South–South or South–North research exchange and “brain drain” can also damage or limit the production and dissemination of research.

• Capacity to conduct policy relevant research. Even though there might be a critical mass of research centres and universities in a particular context, if the knowledge they produce is distant and disengaged from policy problems, the chances of it being used are very scarce.

• Social valuation of science. In countries where a more rationalist culture predominates, the demand for research tends to be more intense and recognizes that science can be neutral. In countries characterized by political cultures that are skeptical of expert knowledge, there will be less demand for scientific knowledge and knowledge will not be used as a tool toward political ends. Moreover, if the level of confidence a country has in science is low, it is more probable that the level of aggregated ability of society’s players (universities, civil society organizations, companies, the state itself) to produce quality, relevant information for the decision making process is lower. This makes it more difficult for evidence to become a deciding factor in the decision-making process.

Strategic planning culture

The existence of large strategic planning processes, generally at the national level, often represents an important framework for the generation and use of policy relevant research. These frameworks are spread from the very top down to all governmental institutions, to establish priorities for subsequent planning, reforms and capacity-building efforts. As a consequence, demand for related evidence is clearer and more concrete. However, sometimes planning can also act as a barrier to introducing new ideas emerging from research: they may be quickly disregarded if they do not fit under previously defined goals or if they could limit the implementation of existing officials’ plans.

Discretionary decision making and corruption

Governance arrangements will play a key role in how a certain technical process/mechanism is taken in and implemented. Oversight mechanisms (checks and balances) exist to guard against discretionary decisions and to ensure accountability in the use of public resources, which create a better environment for research to be used in policy making. On the contrary, arbitrariness, corruption, rent-seeking, cronyism and “influence peddling” significantly diminish the potential for research to be used in policy.

Prevailing policy narratives and ideas

The way in which findings and proposals emerging from research are taken in and interpreted are heavily influenced by dominant policy discourses. Several authors have highlighted how policy problems and potential solutions are socially constructed; the way that policymakers and others frame them is usually linked to how public issues are understood, prioritized and discussed. For instance, how decision-makers define state interests and formulate policies to deal with complex and technical issues can reflect the manner in which the issue is presented by the specialists to whom they turn for advice in the face of uncertainty. Moreover, there is a need to understand how policy problems are defined, and what public sentiment about it is. Indeed, public sentiment (or “philosophy”) and the problem definition will limit the extent to which a solution is considered appropriate and feasible.

Circumstantial factors

In addition to these aforementioned factors, there are some macrocontextual factors that emerge every once in a while that disrupt the regular decision-making landscape and open up opportunities to include research and evidence in policy debate and decisions.

Popular Pressure

Popular pressure and the desire for faster economic growth and improved public services frequently motivate reforms, which sometimes can point in the direction of a new demand for research. However, public opinion does not necessarily have an impact on the uptake of research: many times it is largely a reflection of ideas, priorities and values rather than a result of information and analysis.

Crises and transitions

Crises and transitions also open up unique windows of opportunity for incorporating research into policy. Many lead to short-lived “policy windows” during which an institution is temporarily more receptive to research uptake. For example, regime changes might create spaces for new attitudes towards knowledge and a more conducive environment for research use. The occurrence of natural disaster, for instance, can also give rise to demand for evidence. However, the urgency to reach a consensus for decision-making in such a setting often hinders the possibility of resorting to new sources of information.

Electoral processes are also key milestones when it comes to the use of evidence: it is important to look at the tone of the electoral campaign, and whether electoral candidates and their teams are open or averse to incorporating evidence into their discourses. Transitions at the government administration are other moments in which some policymakers are interested to show “how well” their programs or agencies have been working, looking to survive in the next period. Lots of evaluations are conducted at the end of one government administration or at the beginning of a new one; different diagnoses or policies and also organizational designs are requested.

Usual factors

There is a set of factors that are usually acknowledged in literature on how the macro context affects efforts to promote a better use of knowledge in policy. These are forces that define the big picture for how governmental institutions behave and can or cannot enable change for better use of knowledge. These factors are well beyond the sphere of influence of those who intend to strengthen how research is used in policy, and include:

- Extent of democracy/political freedom
- Extent of academic freedom
- Extent of media freedom
- Extent of development commitment of ruling elite
- Extent of culture of evidence use
- Extent civil society groups have an input into the making of policy
- Extent of political volatility
- Extent of conflict or insecurity
- Political competition
- Government effectiveness
- Priorities and transparency
- Intellectual and civil society environment

Flow of information between jurisdictions and levels

The flow of information between jurisdictions and levels is usually more complex in those countries with federal government structures. In them, national agencies’ access to information generated by local agencies and vice versa is often limited by the degree of political affinity or distance between parties. There are also horizontal challenges: different national government agencies tend to share more or less information depending on their political affinity.

Degree of capacity for use evidence in different sections and departments

Some sections and departments include high-level researchers and have significant budgets while others may lack expertise. Policymakers may also be less willing to use research when they were not involved with from the start, or when it comes from other agencies; engaging with policymakers at all stages of research production could, then, help overcome limited capacity or willingness to use research.

Another challenge is that sometimes research design and/or outputs are not directly linked to the policy agenda; some research works are commissioned for a different purpose and may not necessarily have been translated into a possible solution for a particular policy problem. This sometimes makes it more difficult for policymakers in other institutions to access, understand and use it.

Support from governmental agencies that produce data and research

Support from other governmental agencies refers to the level and depth of interaction and trust between the government institution and other governmental bodies such as national research and science councils, institutes of statistics and policy analysis, and strategy and planning units, departments and directorates, etc. Whether these relationships are hierarchical or horizontal, rigid or flexible, etc. will influence how relevant knowledge is shared and produced. Institutional silos can limit access to research and evidence use; government departments do not always have
easy and free access to evidence generated by such agencies.

Coordination among agencies

In some situations, a lack of coordination between agencies can significantly deter sharing of research. For instance, where coordination is demanded in policymaking due to their integral approaches (for example, social protection), “co-production” of knowledge with different government departments and also with other research institutions, as well as transdisciplinary research will have increasing windows of opportunity. Some potential “spread effect” can also take place if other public agencies note that similar institutions are successful or have good reputation in the media due to how they publish their data; they are encouraged to imitate them.

Policy domains

Policy domains also account for differences in how government institutions interact and share or not knowledge. For example, policy areas such as social services or health generally present an intense level of knowledge utilization. However, the policy domains that use more evidence depend on other factors such as the stamp of the policy makers, the amount and quality of available research addressing relevant issues, among others.

Existence and types of policy forums and epistemic communities

Ongoing and institutionalized or formal interactions between policymakers and researchers enable the development of trust and can positively influence policymakers’ views of evidence. Within these interactions, the existence of “epistemic communities” (that is, colleagues who share a similar approach on an issue and maintain contact with each other across their various locations and fields create new channels for information and discussing new perspectives) are believed to be particularly effective if they include prominent and respected individuals.

Formal channels of interaction with researchers and research institutions

When there are limited systemic or institutional channels for policymakers and researchers to interact, engagement, collaboration or communication is less frequent and chances for dissemination and discussion of research diminish. Even when a channel is created, both sides face significant challenges in investing time to participate in these due to tight agendas and researchers having to commit time to paid projects. Management of such spaces is still challenging.

Links with other training or educational institutions should also be considered among factors that can enable more interaction between research and policy, as too should the existence of intermediaries who facilitate interactions between policymakers and researchers. Moreover, international experts as well as regional and global programmes linked with policy-relevant research, often influence both policymakers, and influence (and finance) local researchers. However, there is an increasing trend to emphasize how this external help should be guided by local agents, as owners of the change process.

Number and type of civil society actors involved in decision processes and degree of vested interests

Research is unlikely to be used if the required reforms go against the interests of important political players. The interests of various stakeholders –including religious groups, pharmaceutical companies and health professionals– may often run counter to research findings and hinder the use of evidence in policymaking. Strong vested interests have a significant impacton policies (including how much budget is allocated) and tend to decrease evidence use and limit the scope of possible policy reform for policymakers.

On the other hand, the existence of an educated and aware public with the capacity to understand evidence may enable a broader and more frequent use of knowledge to make decisions. Links have been found between higher literacy rates and greater use of evidence. Indeed, incentives to support decisions with information lessen if citizens do not demand their political leaders to justify the decisions they make.

The type of relationships with non-state actors and their participation in policy-making processes are very diverse. For instance, the interaction with knowledge provided by business
chambers will differ significantly from evidence emerging from marginalized groups due to the differences in available resources, roles and interests between these stakeholders.

Status of consensus on the policy base

When available research is complemented by the views of people, it is more likely to be used.

Relationships with donors

In many developing countries donors have taken on a long-term functional role in the provision of knowledge to the policy process. Policymakers frequently have to anticipate the responses of donors when developing policy, which may result in them failing to consider local research even when it is available. Also, when political leadership does not request any analytical input, technocrats find it very difficult to take advantage of knowledge produced with the support of external donors. Moreover, as many donors need to take care of relationships with governments, showing negative results is frequently avoided.