Filling the data gaps: community-led research

Not just no data, but also the wrong type of data

Readers who have had anything to do with the Global Fund to fight AIDS, Tuberculosis and Malaria will know that there is about to be a huge flurry of activity, as the Fund prepares to launch its new funding model after a hiatus of a couple of years.

Among many, many other things the new model is emphasizing the need to make sure that programmes are based on sound epidemiological data.  In principle this is a good thing: it is a way of making sure that programmes are designed to reach the right people. The people who, particularly in the context of HIV and AIDS, have been overwhelmingly neglected and marginalised.

But, as Stef Baral wrote on this site a few posts back, the same systems and prejudices that lead to the neglect of groups such as sex workers, men who have sex with men, transgender people, and people who use drugs, also operate to ensure that we don’t really have good data.  Perversely, we end up in a situation where no data = no programmes.

That’s not the only problem, however.  The types of data that are generally considered “robust” and valid for strategic and programming purposes tend to be fairly limited in terms of the insights they provide: we’re talking about estimations of the sizes of different population (or “at risk”) groups, and estimations of HIV prevalence rates among these groups.  While this information, if available, can help to indicate where the majority of new infections are occurring in a given country or area, it tells us little or nothing about the reasons why, or the challenges that highly affected groups are facing.  It limits them to epidemiological risk-factors.

Community-led research an help bridge some of this gap.  It may not provide the hard numbers that decision makers want as a basis for funding allocations, but it is an excellent way of reframing priorities that can help improve how services and programmes are designed.

Sex worker led research in Namibia

A few years ago, because there has been very little research on sex work in Namibia, and because most of the programmes designed to support sex workers are framed around a very narrow HIV focus (information, condoms, cajoling or even coercing people to get tested and have STI check ups; and no attention to issues like violence, discrimination and insecurity), UNFPA and UNAIDS wanted to do a bit of qualitative research to look in more detail at what was going on.  

Although I’m a big user of epidemiological research (quantitative and qualitative), in this context it wasn’t particularly feasible (given the resources available) or appropriate to see this as a classic research project, with publication in a peer-reviewed journal or changing national policies as the ultimate goal.  What seemed more important, given that a major new HIV programme aimed at sex workers was about to be launched, was to document some of the specific situations in the towns that the programme was going to target, to help influence the sorts of things that get addressed, and to identify and point out any gaps in the programme.  Moreover, there were quite a few sex workers in Namibia who are very involved in community work, whether in relation to HIV or more broadly, and we wanted to help them get even more involved.  

So we decided to provide some introductory training on one qualitative method – focus group discussions – and got them to think through what sorts of issues their colleagues might want to discuss.  We used those suggestions to develop a guide, and sent them out to conduct their own research.  Unsurprisingly when they developed the guide, they did not start with a list of questions about “condom negotiation” or “access to HIV testing”.  They started with questions about violence, abuse by the police, and discrimination in health services.  This is important is because the whole idea behind the work we did in Namibia was to move away from the standard survey approaches which ask sex workers the same standard questions about condom use and access to services, and to give sex workers space to talk, among other things, about the issues that HIV programmes aren’t helping them with and maybe even won’t help them with.

The report describes the results in detail.  It also describes the limitations, of which there are many.  Although I remain adamant that the purpose of this activity was never to extract data that will tell the whole story and represent the realities of sex workers throughout Namibia, some common themes come out of each of the five towns.  But there are also differences.  It’s the differences that interest me.  I wanted to give people an opportunity to discuss and think about what was going on in their own towns, and what, practically, immediately, might be done to fix some of the problems in each town.  And to an extent, I think that’s what we got.  It’s not generalisable; in fact the results from each town are probably very biased.  We know, for instance, that in most of the towns, we failed to talk to any male or transgender sex workers.  But if the biases and their relevance to each town are recognised, and used to get positive change in each town, then that’s OK.

We also got a team with a new set of skills, who could do the same thing again, or can replicate it in other towns, or – why not – help other marginalised groups like men who have sex with men, migrants, or people living in slums do the same thing.  The team has, on a number of occasions, used the findings to frame their input into national discussions about HIV programming – so this research, in a very real way, helped to plug the data gap.

Maybe “research” is the wrong term to describe using research techniques in creative ways.  This participatory approach isn’t new to community development work: far from it.  It isn’t new to public health researchers either.  Practitioners have been advocating it for decades.  But it remains a marginal rather than a mainstream practice.  When there’s no data, however, this sort of research is an excellent starting point for filling the gap.


Eight lessons on finding the money

Post by Owen Ryan


Where there is too much data… (Image courtesy of Salvatore Vuono/

Much of what we have discussed so far on Where There Is No Data is about collecting information where none or very little exists. But I focus on a type of research that is characterized by a slightly different obstacle: too much data. Anyone who has conducted any type of budget monitoring or program implementation analysis will be able to relate closely to this issue.

Collecting epidemiological information on marginalized groups can often feel like walking into a dark cave with a very small torch, but collecting budget and policy data on the same populations more closely resembles searching for a grain of salt on a beach of sand.  You know the real data are out there but they are surrounded by – and obscured by – a lot of what you don’t need.

A much better writer than me referred to this as “the signal and the noise” – the signal is what you’re looking for and the noise is everything that distracts you from it. Donors and national governments publish thousands of pages of documents on national AIDS programs but each of these varies tremendously in terms of quality and reliability (for an assessment of donor reporting see here). Finding the signal buried inside reams of noise can be almost impossible, but that information is a crucial advocacy tool and is worth searching for.

Budget and program data when combined with up to date epidemiological data can form a strong basis for effective advocacy with donors and national governments.   Recent examples show how efforts to describe the needs of marginalized groups combined with budget information can result in increased funding and services for these populations.

In recent years, budget monitoring and accountability have become much more prevalent. There are fantastic civil society organizations all over the world conducting this work (find them here). The International Budget Project has a very comprehensive guide and other resources that can help civil society advocates who are new to this work as well as those who are more experienced.

Researching how funding for HIV is allocated or what types of programs are being implemented is not a perfect science. My team at amfAR (the Foundation for AIDS Research) and I have spent years searching through donor reporting and budget spreadsheets to describe how national AIDS programs respond to the needs of sex workers, people who inject drugs, men who have sex with men and transgender individuals. Here are eight lessons we learned from our own mistakes:

  1. Start with the end in mind. If you are planning on writing a short paper on your findings, you will need to limit how much information you are looking for. If you are looking to develop a database on national AIDS spending, you may want to spend a lot more time and effort on your data collection. Take the time to define what you want your finished product to look like and let that guide your research.
  2. Define what you’re looking for. Do you want to know more about how much money the government spends preventing HIV among sex workers? Are you interested in the availability of condom compatible lubricant for MSM? Be clear in the beginning what you are trying to track and write specific questions for each. This will help guide and focus your research.
  3. Don’t reinvent the wheel. Take time to understand what others have written about your subject so you can build off of that knowledge without repeating the same research.
  4. Start with official reporting then quickly move on. Government and donor reporting is a good place to start learning more about a country’s national AIDS response, but it can also take too much time (e.g. PEPFAR’s 2012 plan for South Africa is 663 pages; the current National Strategic Plan is 83 pages). Read the pieces you need to understand, but quickly move on to other sources.
  5. Sometimes you can get a lot more out of a phone call or meeting than you can out of an official report. Many donor agencies and national governments have a contact person who is available to talk to civil society organizations interested in learning more about their national AIDS response. Getting a meeting with them can take a lot of patience but their information is often more accurate and reliable than official reporting.
  6. Be rigorous with your data collection. You will quickly find yourself knee-deep in a lot of data. Use a system to collect this information and keep track of your sources. Stick to that system. That way, you can accurately describe what you found when you’re finished. If people raise questions about your findings, you can quickly point to your data.
  7. Keep talking to your sources. You will find people in government and at donor agencies that are willing to help you with your research. Keep talking to them and make sure when you find information that they know something about, you check with them to get their opinion on it.
  8. Put the same level of effort into communication that you did for data collection. Your findings are no good if they sit on a shelf. Talk to others about the best (and, depending on the context, safest) ways to make sure the right stakeholders hear what you have to say. This is never accomplished with one meeting or one presentation. Keep talking about it even months after your work is finished.

It is worth repeating – this is not a perfect science, but it is an essential element in describing the needs of marginalized groups.

Owen Ryan is Deputy Director for Public Policy at amfAR, the Foundation for AIDS Research where he focuses on key populations, budget monitoring for HIV, and new prevention technologies. Owen is also an Alternate Board Member to the Global Fund to Fight AIDS, Tuberculosis and Malaria.

What is data for health?

The first few posts on Where There Is No Data have set the scene by discussing the challenges faced by marginalised populations in HIV programmes.  We’ve also talked about the fact that having good data isn’t always enough – policy decisions tend to be influenced as much by politics and prejudices as they are by evidence.  Nonetheless, while data isn’t everything, getting better data is important.  It can begin to shine a light on problems that have been ignored; and when there is political commitment to tackling these problems, it can help make sure that health programmes are properly designed.

Policy makers, funding agencies and NGOs often talk about different types of data and research, but not everyone is familiar with these.  When we talk about data for health programmes what do we mean?

  • Data that describes the burden of health problems including the prevalence and incidence of these conditions, and how they are distributed among different sections of the population.  In most countries, data on HIV prevalence (i.e., the percentage of the population that is infected with HIV) is derived from a range of sources: surveys of a representative sample of households during which respondents are asked to participate in anonymous HIV testing; surveys of specific population groups such as pregnant women, men who have sex with men, sex workers (these are often called “sentinel surveillance” surveys); and routine data – for instance, the proportion of people volunteering for HIV testing who are found to be HIV positive.  None of these methods provides a complete picture of HIV prevalence but, if all of them are used fairly regularly, they can help give a good overview of what is happening.  In some countries, special studies are carried out to measure HIV incidence (i.e. the rate of growth of the epidemic in a given period of time such as a year), but this type of study is rarely carried out at national level.
  • Prevalence and incidence studies can also help to identify how a health problem is distributed in the population – whether it affects men or women more, whether some age groups are disproportionately affected, and which behaviours or sub-populations are most affected.  A lot of this information can be gained from household surveys and routine data.  However, because in many societies people are reluctant to talk openly about sex and sexuality, and in particular because of the stigma against behaviours such as sex work, they are often under-represented.  Moreover, knowing how a health problem is distributed in the population is not just a matter of knowing the prevalence of the problem in each sub-population group.  It also means knowing the size of each these sub-population groups is.  This is useful for planning programmes and ensuring that resources are allocated in the right places.  Once again though there are particular challenges in estimating numbers of marginalised populations.  Sex between men, sex work, and drug use are behaviours not identities, and people with these behaviours often have good reasons to avoid being counted or included in surveys.  In many locations there has been little or no research on these groups and informal, community level research is needed before any formal surveys can be carried out.
  • Information about what makes people vulnerable or at risk.  The same surveys described above can help provide an indication of why some people are more vulnerable or at risk than others.  In the context of HIV, researchers often ask respondents about their sexual behaviour, condom use and so on – although once again, peoples’ responses to these questions are not always reliable.  Moreover it is not enough to know whether people have good enough knowledge about HIV or whether they use condoms or whether they have access to health care.  It is also important to know why these things happen.  Is it because programmes are not reaching them? What role does stigma and discrimination play?  Getting the answers to these questions often requires a different approach: one that engages much more with the people concerned, and ideally one that is led by them.  These factors also often vary from place to place and can change over time, so it is important to have mechanisms that enable communities to collect, understand and act on local data in a regular way.
  • Data on what types of programme or “intervention” are effective.  Many policy makers rely on experimental research methods to provide evidence on the effectiveness of programmes or interventions for preventing or treating health problems.  Although they are expensive to conduct these studies help provide an estimate of how effective different approaches are.  There is considerable debate surrounding the reliability of this sort of study for evaluating social change programmes, although their use in this area is growing.  In any case, because they are experiments, generally conducted in controlled conditions, they will not necessarily be as effective or work in the same way when implemented at scale.  For this reason programmes have to find ways of continually monitoring their impact and of identifying any unintended consequences.  Once again a combination of large scale survey and routine data and more qualitative, community based approaches is needed.
  • There is increasing interest in good quality programme related data – for instance, data on the coverage of programmes (how many people they reach and who these people are); data on how programmes are funded (where does the money come from? Communities? The government? Donors?); and data on what it costs to implement different types of programme.  Good data on costs, combined with good data on effectiveness can be used to make sure the most cost-effective programmes get funded – in other words that available resources have the biggest possible impact.

Readers working on HIV and AIDS may be familiar with concepts such as “know your epidemic”, “know your response” and “strategic investment”.  The description of the different types of data described above is somewhat simplistic; however they are the basic building blocks behind these concepts.  It is particularly useful for community actors, key or marginalised populations to know about these types of data and concepts so that they can clearly articulate the gaps when speaking to policy makers.

We will explain all of these types of data in more detail on this site – including how reliable are, what sorts of skills are required to collect them, and the methods and the costs of collecting them.  There are a number of free online courses and resources where you can learn more about different research techniques and we will post links to these too.  Finally, we will talk about the importance of data generated by communities and the challenge of trying to get this sort of data taken seriously.

The “data paradox”

In this short post we discuss further some of the challenges faced in planning HIV programmes for key populations.  The post is based on discussions that were originally posted on our Facebook page.


A write-up of a discussion among sex workers about where they are most likely to face violence in their community, and what they can do about it. By FIMIZORE, Madagascar network of sex workers (c).

Stef Baral

So why are we in a situation where we have no data on some of the groups most affected by HIV? To characterize the widespread and generalized HIV epidemics found in mostly southern and eastern Africa, we have relied on surveillance systems that do not meaningfully assess the very populations known to be most affected by HIV in most parts of the world including sex workers, gay men and other men who have sex with men, people who use drugs, and trans populations. I understand the design and use of these systems–and there is no doubt that having a sampling frame made up of households is the most effective tool for population-based sampling to retrieve population-based estimates of HIV.

However, I also understand that for there to be household based transmission of HIV, there has to be sero-discordant relationship. And that tells me that there has to have been a primary infection event outside of the household. It is those risk factors for the extra-household primary infection events that we still seem to know very little about. And it seems to me that we may not have invested in this since maybe we don’t want to hear the answer.

Maybe it is because all around the world there are people that sell sex–as there always has been and always will be. All around the world, there are people who do not fit into heteronormative social expectations for sexuality and simple binary expressions of gender. And all around the world there are people who use drugs–some for fun and some because of untreated or undertreated mental health. And each of these people have specific needs for the prevention and treatment of the acquisition and transmission of HIV. But how can we fully understand these needs, if we don’t want to admit that populations exist everywhere. So we still, 30 some odd years later, find ourselves in a data paradox. A paradox where we know less about the needs of diverse populations in settings with the most stigma.

Does that mean we should not respond until we have “the data”? It took me a while to get how brilliant of a strategy this is–argue that the lack of data is a reason to not launch any programs or funding that may result in data. Ie, the lack of data feeds itself.

Matt Greenall

So the “data paradox” is this: decision-makers deny that most affected populations exist, or that they are relevant to the epidemic; so no research gets done on these populations; the lack of data feeds the denial; and so on.

But the other challenge we face is that decision-makers don’t always put resources towards programmes with key populations even when good data is available. Experience tells us that politicians, in particular, are very capable of ignoring data when it suits them to do so – and there are many countries where the quality of the data is reasonable enough but the investments in programming with the most affected groups are still all wrong. So while it is important to encourage better national level research and data that can eventually be plugged in to modelling and strategic planning exercises, we should also be aware that they won’t necessarily resolve everything.

Although there is no doubt that having the right laws and supportive leadership from the government or Ministry of Health makes a huge difference to how different problems get addressed, not all change comes from the top down. Similarly, the stigma and marginalisation that key populations face does not only come from the national level – it also has a lot to do with attitudes and behaviours of health care workers, law enforcement officers, and community members at the local level. These need to be addressed to. This discussion at the recent International AIDS Conference in Kuala Lumpur is an example of how research and programmes can be initiated at local level even without strong support from policy makers. I’ll also post something in a few days, and post a link here, about local, community-led research and how it can help groups get organised at local level.