Post by Owen Ryan
Much of what we have discussed so far on Where There Is No Data is about collecting information where none or very little exists. But I focus on a type of research that is characterized by a slightly different obstacle: too much data. Anyone who has conducted any type of budget monitoring or program implementation analysis will be able to relate closely to this issue.
Collecting epidemiological information on marginalized groups can often feel like walking into a dark cave with a very small torch, but collecting budget and policy data on the same populations more closely resembles searching for a grain of salt on a beach of sand. You know the real data are out there but they are surrounded by – and obscured by – a lot of what you don’t need.
A much better writer than me referred to this as “the signal and the noise” – the signal is what you’re looking for and the noise is everything that distracts you from it. Donors and national governments publish thousands of pages of documents on national AIDS programs but each of these varies tremendously in terms of quality and reliability (for an assessment of donor reporting see here). Finding the signal buried inside reams of noise can be almost impossible, but that information is a crucial advocacy tool and is worth searching for.
Budget and program data when combined with up to date epidemiological data can form a strong basis for effective advocacy with donors and national governments. Recent examples show how efforts to describe the needs of marginalized groups combined with budget information can result in increased funding and services for these populations.
In recent years, budget monitoring and accountability have become much more prevalent. There are fantastic civil society organizations all over the world conducting this work (find them here). The International Budget Project has a very comprehensive guide and other resources that can help civil society advocates who are new to this work as well as those who are more experienced.
Researching how funding for HIV is allocated or what types of programs are being implemented is not a perfect science. My team at amfAR (the Foundation for AIDS Research) and I have spent years searching through donor reporting and budget spreadsheets to describe how national AIDS programs respond to the needs of sex workers, people who inject drugs, men who have sex with men and transgender individuals. Here are eight lessons we learned from our own mistakes:
- Start with the end in mind. If you are planning on writing a short paper on your findings, you will need to limit how much information you are looking for. If you are looking to develop a database on national AIDS spending, you may want to spend a lot more time and effort on your data collection. Take the time to define what you want your finished product to look like and let that guide your research.
- Define what you’re looking for. Do you want to know more about how much money the government spends preventing HIV among sex workers? Are you interested in the availability of condom compatible lubricant for MSM? Be clear in the beginning what you are trying to track and write specific questions for each. This will help guide and focus your research.
- Don’t reinvent the wheel. Take time to understand what others have written about your subject so you can build off of that knowledge without repeating the same research.
- Start with official reporting then quickly move on. Government and donor reporting is a good place to start learning more about a country’s national AIDS response, but it can also take too much time (e.g. PEPFAR’s 2012 plan for South Africa is 663 pages; the current National Strategic Plan is 83 pages). Read the pieces you need to understand, but quickly move on to other sources.
- Sometimes you can get a lot more out of a phone call or meeting than you can out of an official report. Many donor agencies and national governments have a contact person who is available to talk to civil society organizations interested in learning more about their national AIDS response. Getting a meeting with them can take a lot of patience but their information is often more accurate and reliable than official reporting.
- Be rigorous with your data collection. You will quickly find yourself knee-deep in a lot of data. Use a system to collect this information and keep track of your sources. Stick to that system. That way, you can accurately describe what you found when you’re finished. If people raise questions about your findings, you can quickly point to your data.
- Keep talking to your sources. You will find people in government and at donor agencies that are willing to help you with your research. Keep talking to them and make sure when you find information that they know something about, you check with them to get their opinion on it.
- Put the same level of effort into communication that you did for data collection. Your findings are no good if they sit on a shelf. Talk to others about the best (and, depending on the context, safest) ways to make sure the right stakeholders hear what you have to say. This is never accomplished with one meeting or one presentation. Keep talking about it even months after your work is finished.
It is worth repeating – this is not a perfect science, but it is an essential element in describing the needs of marginalized groups.
Owen Ryan is Deputy Director for Public Policy at amfAR, the Foundation for AIDS Research where he focuses on key populations, budget monitoring for HIV, and new prevention technologies. Owen is also an Alternate Board Member to the Global Fund to Fight AIDS, Tuberculosis and Malaria.