My comments (based on 20+ years of experience in humanitarian and development work) focus on the following excerpt from the Executive Summary, which states "...the evidence base on programme impact in conflict-affected situations tends to be of low quality, with outputs often privileged over impacts and questions of internal design frequently taking precedence over the experiences of beneficiaries. More fundamentally, many studies and reports fail to include adequate information on their methodologies or data sources, making it difficult to accurately appraise the reliability of their conclusions and recommendations. Given the primary purpose of these studies – to provide practical policy guidance on ways to support livelihoods and engineer growth – this silence is particularly concerning."

Let us first consider the issue of low quality evidence base: this I think presupposes that humanitarian workers working in conflict-affected and complex environments have everything at thier disposal to generate the evidence base from the programs they implement. This is not to use the difficult situation argument as an excuse, but to ground the discussion on reality. The fact that humanitarian programs do not have the required number of staff to do the job right (at any given time) is a huge challenge in itself. Added to this, because of the difficult nature of humanitarian response work in conflict situations, humanitarian agencies face huge challenges of high staff turnover and recruitment for difficult to fill-in positions takes a considerable amount of time sometimes between 12 and 18 months for some senior positions. Under this situation, available staff work under huge pressure with the resultant effect of stress and burn-out, which results in high staff turnover. In this regard, it is not surprising that the evidence base from humanitarian programs is limited.

Second, on outputs previlaged over impacts: humanitarian response needs to first focus on addressing needs. This means reporting on number of people provided with supplies of food and non-food items, those protected from any form of violence, number of affected people who have accessed first line health services, etc. Under such situations, the available data would be at output level rather than effect and impact level. In fact, expecting the latter two from an emergency humanitarian response program is unrealistic. This is because emergency humanitarian programs are primarily designed to save lives. Note that effect and impact can be achieved when supporting local institutions are strong and when they function well to take on the work that has been started by a project. Such institutions are non-existent or are weak. The reason for external assistance is justified, promoted and implemented because of weak institutions that cannot provide basic life saving services when disaster strikes. Strong and viable institutions are crucial for lasting change to be realized. Short-term programs/projects can catalyze change, but cannot deliver long-term impact on their own.

Third, on questions of internal design taking precedence over the experience of beneficiaries. The question that needs to be raised is: what is driving such practices? It is important to realize that projects are designed in response to donors' polices and requirements. Furthermore, humanitarian agencies are accountable to their donors. This includes not only implementing a project as agreed with a donor, but also minimizing all sorts of technical and administrative risks. The time that donors provide for designing projects doesn't also allow adequate opportunities to go through a consultative process involving beneficiaries and stakeholders. Often, projects are designed not based on what humanitarian agencies and beneficiaries know best, but based on what donors want. Overall, to fully incorporate the experience of beneficiaries in a project design, donors must let go of their strict policies and guidelines which may not always be relevant to a particular context. In this regard, flexibility is key to allow humanitarian agencies to align the project design to local realities.

Last, but not least, methodology and data source with regard to reliability of conclusions and recommendations. In my view, all of the above issues need to be addressed to address issues of methodology and recommendations.