Home Blogs

A longitudinal panel survey in a conflict-affected situation! Are you crazy?

19 December, 2012 : 17:13

"Are people right to be so scathing or sceptical about us trying to use quantitative survey methods in the Secure Livelihoods Research Consortium, or about us attempting a panel?" 

SLRC Research Director, Rachel Slater

SLRC is doing a quantitative survey in its seven focus countries to explore how livelihoods recover following conflict and how access to basic services affects perceptions of government.  Not only are we trying to do a representative, statistically significant survey, we aim to make it into a longitudinal panel by returning to the same households in three years time.  We have just completed the first round of the survey in DRC, Nepal, Pakistan and Sri Lanka, are currently in the field in Uganda, and will go to the field soon in Afghanistan.

When I tell people that we are doing a quantitative panel survey, I get three reactions from people. The first is a frown. It may be accompanied by the second reaction – (often fierce) shaking of the head.  The third (definitely the worst!) is a reaction which feels like I'm being patted on the head and told ‘never mind, my dear, you’ll soon learn’.  So, are people right to be so scathing or sceptical about us trying to use quantitative survey methods in the SLRC, or about us attempting a panel?

We certainly recognise that our approach brings risks – especially the security of our respondents and enumerators – but after spending the time last month with the SLRC Nepal survey team, in Ilam district in the far west of the country, I'm more convinced that our approach is justified. This is why:

 
Enumerator(s) picking their way across a small landslide in Eastern Nepal
 

It starts with the journey into Ilam.  We fly to Biratnagar and then on by road.  At one stage we stop for tea (a famous Ilam product) and across the valley see a huge landslip scarring the mountainside.  On closer inspection, we realise that it is the landslip of 30th September.  I can feel the heart of Bishnu (SLRC’s leader in Nepal) grow heavy as we contemplate the 22 bodies forever entombed in the mud, soil and rocks.  A few days later, I find myself gingerly picking my way across smaller landslip, urging the team of ten enumerators in front of me not to dawdle and to stop chattering so they can hear any rock fall coming from above.  All this reminds me that, as a team, we frequently catch ourselves sliding into what we've called ‘conflict exceptionalism’, where we automatically assume that conflict is the cause, or outcome of all that we find.  Here in Nepal, I'm reminded at every turn of the path winding from one household to the next that there are numerous threats to people’s livelihoods and well-being and that our survey will help to capture that multiplicity.  

We also face other challenges in the survey that are not directly related to conflict: enumerators walking up to five hours between households; difficulties preventing a small (or large!) crowd gathering to listen to (and sometimes attempt to join in with) the interview; very elderly respondents who, although we’d love to know more about how they perceive the government, cannot understand our questions well.  In Bardiya district, where young women and girls were being kidnapped and taken across the border into service in India, we had to reorganise our enumerator teams to ensure that all our female enumerators worked far from the border.

So, should we avoid doing quantitative work in conflict-affected places because of these challenges? I’m left feeling that conflict should not be an automatic excuse for not doing quantitative survey work and that we need to look at situations on a case by case basis.  Nepal may currently be safer than Pakistan or DRC and these will certainly present other challenges, but nothing we have confronted so far is a reason not to try and deliver a high quality survey.

I find myself coming face to face with the second question – should we try a longitudinal panel? - in the first Ward where I accompany enumerators.  Beside the logistical challenges (for example attrition in our sample), we’ve been told by many people that that in three years we’ll never see enough shifts, either in people’s access to health, water, education and social protection, or in their attitudes towards government, for us to say anything useful about how far delivering services can contribute towards state legitimacy and state building.  And the truth is that we won’t and can’t know what change we’ll get until 2015.  Until now, we've been arguing that if we find very little change in access to services or to attitudes, this will at least contribute to a recognition of the sorts of timelines that are required to get changes in people’s perceptions of their governments.  Then, en route to our first interview, I'm taken by Suresh Prasain to look at the local health post.  It’s a small, dilapidated building but right next to it the foundations are being laid for a new 20 bed inpatient facility.  So here, at least, we are likely to see either some change in access to health service provision.  And if construction should grind to a halt, or the facility is not staffed, we can expect to hear our respondents voice their frustrations in three years’ time.

So, my visit to Ilam has left me feeling that in Nepal we can deliver a high quality survey and there is huge value in tracking change over the next 3 – 4 years.  We face significant logistical constraints, especially tracking down our respondents in the future, but we've a strong plan to tackle them.  Finally, the survey has thrown up some new analytical challenges for us to deal with – not least how we analyse ‘cross-border perceptions’ – but Bishnu and I will tell you more about that next week!

Share this via Twitter         Follow @SLRCtweet

7 comments

Joe
Re: "Not only are we trying to do a representative, statistically significant survey, we aim to make it into a longitudinal panel by returning to the same households in three years time." You cannot TRY to obtain statistically significant findings. Statistically significance does not say anything about the quality of the data, it is about whether how likely the findings are found due to chance.
Mao
...conflict should not be an automatic excuse for not doing quantitative survey work and that we need to look at situations on a case by case basis. I share with you this, and for the specific current situation in Mali. Thanks
Deepika
A longitudnal survey is very much needed in post conflict situation, as we all know that relief phase is able to attract funds from internal and extrnal agencies and lot of programmes are conducted but are these programmes able to make sustainable change even during and after rehabilitation phase must be studied
Patricia Justino
Rachel Very interesting post. I would like to note that longitudinal surveys of conflict countries already exist. Some were collected as part of the MICROCON programme: www.microconflict.eu. Others have been collected by affiliates of the Households in Conflict Network: www.hicn.org. Fafo in Norway has also collected a couple of those. For issues around methodology, analysis and implementation I would like to bring to your attention this paper: http://www.hicn.org/wordpress/wp-content/uploads/2012/06/wp79.pdf Best, Patricia Justino Senior Fellow, Vulnerability and Poverty Reduction Team Head, Conflict, Violence and Development Cluster Institute of Development Studies, UK Director, MICROCON (www.microconflict.eu) Co-director, Households in Conflict Network (www.hicn.org)
Annoymous
Thank you for this. I was previously working at a research firm in Afghanistan which focused on "perception based" qualitative research. In reality, the line that quantiative research cannot be done, and if done would be of poor quality or limited usefulness was in reality a line to convince donors that our work, despite being methodologically weak, was still worth funding. The greatest shame has been that rather than respond to the challenge of doing quantiatie reseach in fragile states by developing new methodologies and innovative ways of doing research, researchers have instead given up and argued that such work isn't possible. The question I'm curious though: how is secure livelihoods going to accomplish this project in areas where governments are extemely suspicious of outside researcher, i.e. Pakistan?
fadia
I am not surprise when you said : "When I tell people that we are doing a quantitative panel survey, I get three reactions from people. The first is a frown. It may be accompanied by the second reaction – (often fierce) shaking of the head. The third (definitely the worst!) is a reaction which feels like I'm being patted on the head and told ‘never mind, my dear, you’ll soon learn’ Because you are talking about the survey like a trip you have done with a team of enumerators without talking about the life hood &daily surviving conditions of individuals within a conflict situation. Such situation should be very very hard for the concerned population and all what they need is getting toward peace and justice in the existence of an accurate governance. The recovery from this cavernous situation will take time to get a place and any survey will not respond for its goal under any conflict time pressure .
Pramod Sharma India.
Thanks for the discussions. I am always in favor of complete research (which includes both qualitative research and quantitative research). Some times, researchers prefer to initiate research with quantitative data collection and then do the qualitative research. In second approach researchers initiate with qualitative data and then quantify the data. For example, If a researcher found that Land slide is a problem in the mountain area through qualitative data (Observations, key informant interviews etc). Then he/she quantify the wideness, extant, intensity of land slide. And in the first approach, Researcher tried to find the common issues and try to understand its depth through qualitative data and it can also use at the time of some mixed or confused quantitative findings and assess the future trends which we can not assess only on the basis of quantitative data. For example, if we found 75% HH are indebted to private money lenders; then we try to understand the process of money lending, good and bad practices, good and bad impacts on people and many such questions. So in my view, we need to consider both the approaches and also try to identify which is the best practice (First qualitative then quantitative) or vice versa..

Add a comment

Name:
Comment:
 
 

Latest SLRC blogs

Here you will find the latest SLRC blogs.

Our bloggers

Archives

Welcome to SLRC's blog.

This blog will feature reflections from our team of researchers on the practicalities of actually conducting research in conflict-affected situations. We will also be posting guest blogs written by key researchers and practioners working on livelihoods, basic services and social protection in conflict-affected situations.