Impressions from HSR 2018: Developmental Evaluation – an approach whose time has come?

Related Project:
Coordinating Implementation Research to Communicate Learning and Evidence (CIRCLE)
Author(s):
Ana Claudia Franca-Koh, Deputy Director/Research Advisor, CIRCLE Project

It is  great to be here in Liverpool at the Fifth Global Symposium on Health Systems Research. We kicked off the week by hosting the Developmental Evaluation (DE) skills building satellite session based on our early experience implementing a DE in Tanzania. Jointly led by Dr. Beati Mboya, the Principal Investigator of CIRCLE’s* DE team,  and Dr. Ashwin Budden, a DE expert, the session aimed to apprise the audience about the fundamentals of the DE approach. The diverse audience representing  NGOs, government, research institutions, and funding organizations mirrored the Tanzanian panelists – Rose Kyando (USAID/Tanzania), Dr. Bakari Salum (Government of Tanzania), and Albert Ikonje (Deloitte, Inc.), who ably led the audience through the nuts and bolts of Tanzania’s DE. 

Participants reflected on the need to improve on traditional evaluations which produce results at the end of a program cycle, which is often too late to inform subsequent interventions. 

Photo credit: Ant Clausen

“I was looking for new ways of doing things. The traditional ways are not working well, due to rapidly changing environments. DE tries to capture the needs of our times. We aren’t really getting what we need from traditional evaluations, in terms of the need to adapt our projects.”  

– Ryan Guinaran, DITENG Inc. and Benguet State University, Philippines.

What DE is and is not

  • DE is not a tool or a method, it is a broad approach to evaluation that supports different kind of methods.
  • DE focuses on learning and program improvement, not on judging whether a program succeeded or failed.
  • DE is ideal for complex programs that are developing and or operating in dynamic contexts. 

    Photo credit: Ant Clausen
  • DE is flexible in terms of how it is designed and implemented. But this also means it needs a flexible enabling environment. This may be challenging for organizations that have defined reporting periods.
  • DE is not the same as process evaluations. Although DE does focus on process, it can also target outcomes and especially unintended and emergent outcomes that are not part of a predefined results framework.
  • DE is not the same as performance reviews. Performance reviews focus on examining indicators and to what extent targets have been met. DE goes further; it looks at broader contextual issues by asking more questions: For example, why are we seeing these changes?  Are there factors that are affecting these results? Where are the opportunities for course correction or innovation?

This approach of continuously informing stakeholders, in particular those implementing the programs, about areas of improvement is something that we need.”

Habib Hussain, Contech International, Pakistan

Stakeholder engagement 

  • DE employs a continuous stakeholder engagement process, implemented from the beginning. This takes time, effort and commitment.
  • All stakeholders need to be included. Mapping them out from the beginning to decide who to engage, when and how is important.
  • Getting stakeholder buy-in is only the beginning. DE keeps stakeholders involved throughout the process, from developing evaluation questions and frameworks together, to embedding evaluators that work alongside implementers.
  • Embedded evaluators are key to ongoing engagement and cultivating trust relationships with stakeholders. 

 

To learn about our activities at HSR click here, and for more information about DE, read other recent blogs at www.harpnet.org.

Stay tuned for more about implementation research in MNCH at HSR 2018.

 

*USAID’s Coordinating Implementation Research to Communicate Learning and Evidence (CIRCLE) Project