Understanding success in research-for-development projects

Teak agroforestry in Indonesia. Photo credit: ACIAR/ANU/Tony Bartlett

A new study argues that there are five key success factors that need to be considered during the design of collaborative research-for-development projects and a further 10 key success factors that should be considered during project implementation, in order to increase the prospects of success.

The study, published in 2018 in International Forestry Review, synthesized results from case studies of collaborative forestry research projects, 10 each in Viet Nam, Indonesia and Papua New Guinea. The relative success of projects was evaluated and the factors affecting success were investigated. Differences were found in the relative success of projects, between and within countries, and between related projects in a long-term program.

Only one-quarter of the 30 projects studied had high achievements and high impacts; none of which occurred in Papua New Guinea. Interviews with 90 project participants identified 37 success factors that can enhance or diminish project success, of which 15 were considered generally applicable to research for development projects.

Acacia improvement in Viet Nam. Photo credit: ACIAR/ANU/Tony Bartlett

The two most important success factors were collaborative scoping and design, and the scientists’ commitment, collaboration and focus. Some relationships were apparent between relative success, the success factors and contexts at the national, local and project levels, including the importance of linking research to impact pathways.

The study was conducted by Tony Bartlett of the Australian Centre for International Agricultural Research as part of doctoral work at the Fenner School of Environment and Society of the Australian National University. He will be presenting his findings at the World Congress on Agroforestry in Montpellier, France, 20–22 May 2019.

 

‘I have also completed an analysis of the “relative success” of 15 different agroforestry projects [as distinct from a wider range of forestry projects] funded by ACIAR, including some of the ones led by ICRAF,’ said Bartlett. ‘I have analysed the responses from project staff about the factors that have contributed to enhanced or diminished success in these agroforestry projects. The analysis will form the content for most of my keynote talk.’

The talk will also consider the role of scaling-up activities in achieving success and examine the role of research-for-development projects in achieving transformational development outcomes.

In the synthesis, Bartlett identified new factors among the 15 key success factors that had been compiled from earlier research, one of which was also the most frequently identified success factor: ensuring there is effective collaboration between the proposed project’s partners during scoping and design stages. Other newly identified factors were related to the adequacy of funding for the planned research; the importance of capacity-building activities; the design and location of field trials; monitoring and review processes that support implementation flexibility; and dissemination of research results.

Table 1. Success factors

‘For the success factor, “donor influence on design”’, noted Bartlett, ‘divergent outcomes are possible, depending on the way in which the factor is addressed. Generally, project leaders expressed the view that donor interventions during the design process — to broaden the skills

within the project team or enhance the project’s focus on particular activities — improved a project’s success. However, in two projects, donor interventions related to combining two themes of research and/or substantially reducing funding without reducing the expected activities, reduced success.’

Some of the success factors — such as a deteriorating security situation, political interference, ineffective policies or markets, and community or land disputes — also significantly reduced the success of individual projects.

Landowner with balsa trees in Papua New Guinea. Photo credit: ACIAR/ANU/Tony Bartlett

‘While such factors may be very difficult for an individual project to manage,’ said Bartlett, ‘it’s important that both the project team and the donor are aware of them and the constraints they pose for implementation and impact.’

Further, he argues, in some situations where these factors are identified, it could be appropriate to terminate or modify the project, particularly, if success is likely to be severely compromised.

‘It’s also critical to understand that multiple factors need to be addressed at the same time,’ he stressed. ‘While this knowledge could help reduce the risk of poorly performing projects, it should not be used to preclude higher-risk, research-for-development investments. In reality, some outcomes and impact are unpredictable, others are context specific, and there is unlikely to be a definitive list of factors that will guarantee success for all project investments. Rather, it is more important to have a good understanding of the breadth of factors and have processes during project design and implementation to address all the success factors relevant to the project.’

To assist with understanding the relationships between the factors and the relationships with project contextual factors, Bartlett developed a theoretical framework consistent with findings from other researchers. The framework shows that the 10 success factors that can be influenced during design and the 20 that can be influenced during implementation have a central influence on success. However, these 30 success factors in themselves are not the only determinants of a project’s success. Every research-for-development project is implemented within a broader and variable context that can influence success.

While there are many external factors that contribute to the operational context, they are represented in the theoretical framework by four categories: 1) enabling policies; 2) research capacity; 3) stakeholder support; and impact-pathway links. Each of these four categories, while often being largely outside the control of a project, does need to be taken into account during the design, implementation and evaluation of research-for-development projects.

                                                  Figure 1. Theoretical framework for considering success in research-for-development projects

The Framework has been informed by the work of other researchers, such as Coe et al (2014), who identified the importance the importance of developing appropriate service-delivery mechanisms, markets, and institutional contexts (impact-pathway links) and co-learning amongst research, development and private-sector actors (stakeholder support); Gritten et al (2015), who identified regulatory barriers that restrict enhanced livelihoods from the sale of timber and timber products from community forests (enabling policies).; Clark et al (2016), who examined factors that help ensure knowledge for sustainable development is useable, including co-production relationships between knowledge- and decision-making (enabling policies) and the need for effective stakeholder collaboration (stakeholder support).

Bartlett’s study identified the level of institutional and individual capacity amongst the developing-country partner researchers as an additional aspect of context that can influence success, confirming the finding of Ika and Donnelly (2017) of the importance of the capability of a beneficiary institution.

‘The implication of the study is that donors need to consider organizational capability both when designing projects and when comparing the results of inter-country evaluations,’ he said.

Further research is needed to improve understanding of why similar projects achieve quite different results in different locations.

Read the synthesis study

Bartlett AG. 2018. Understanding and evaluating success in international forestry research projects: experience from ACIAR projects in Vietnam, Indonesia and Papua New Guinea. International Forestry Review 20(3):274–295.

Rob Finlayson

Robert Finlayson is the Southeast Asia program's regional communications specialist. As well as writing stories for the Centre's website, he devises and supervises strategies for projects and the countries in the Southeast Asia region, including scripting and producing videos, supervising editors and translators and also assisting with resource mobilization.

You may also like...