Monday, June 27, 2011

Addressing the lack of humanitarian information standards

It was fourteen years ago that a group of humanitarian NGOs and the Red Cross/Red Crescent movement came together and created the Sphere Project which defined minimum standards for disaster response. The cornerstone of the Sphere Handbook was the Humanitarian Charter which describes the core principles that govern humanitarian action. On top of this there are minimum standards and indicators defined that currently are utilized as a reference all disaster response.

Some of the Sphere core standard do address information sharing, like the following excerpt show:


Core Standard 1 – People Centered Humanitarian Response

People have a right to accurate and updated information about actions taken on their behalf. Information can reduce anxiety and is an essential foundation of community responsibility and ownership. At a minimum, agencies should provide a description of the agency’s mandate and project(s), the population’s entitlements and rights, and when and where to access assistance (see HAP’s ‘sharing information’ benchmark). Common ways of sharing information include noticeboards, public meetings, schools, newspapers and radio broadcasts. The information should demonstrate considered understanding of people’s situations and be conveyed in local language(s), using a variety of adapted media so that it is accessible to all those concerned. For example, use spoken communications or pictures for children and adults who cannot read, use uncomplicated language (i.e. understandable to local 12-year-olds) and employ a large typeface when printing information for people with visual impairments. Manage meetings so that older people or those with hearing difficulties can hear.

Core Standard 2 – Coordination and Collaboration

  • Be informed of the responsibilities, objectives and coordination role of the  state and other coordination groups where present.
  • Provide coordination groups with information about the agency’s mandate,  objectives and programme.
  • Share assessment information with the relevant coordination groups in a  timely manner and in a format that can be readily used by other humanitarian agencies
  • Use programme information from other humanitarian agencies to inform  analysis, selection of geographical area and response plans.
  • Regularly update coordination groups on progress, reporting any major delays, agency shortages or spare capacity.

Efficient data-sharing will be enhanced if the information is easy to use (clear, relevant, brief) and follows global humanitarian protocols which are technically compatible with other agencies’ data. The exact frequency of data-sharing is agency- and context-specific but should be prompt to remain relevant. Sensitive information should remain confidential

Core Standard 3 – Assessment

Pre-disaster information: A collaborative pooling of existing information is invaluable for initial and rapid assessments. A considerable amount of information is almost always available about the context (e.g. political, social, economic, security, conflict and natural environment) and the people (such as their sex, age, health, culture, spirituality and education). Sources of this information include the relevant state ministries (e.g. health and census data), academic or research institutions, community-based organisations and local and international humanitarian agencies present before the disaster. Disaster preparedness and early warning initiatives, new developments in shared web-based mapping, crowd-sourcing and mobile phone platforms (such as Ushahidi) have also generated databases of relevant information.

Initial assessments, typically carried out in the first hours following a disaster, may be based almost entirely on second-hand information and pre-existing data. They are essential to inform immediate relief needs and should be carried out and shared immediately.

Data disaggregation: Detailed disaggregation is rarely possible initially but is of critical importance to identify the different needs and rights of children and adults of all ages. At the earliest opportunity, further disaggregate by sex and age for children 0–5 male/female, 6–12 male/female and 13–17 male/female, and then in 10-year age brackets, e.g. 50–59, male/female; 60–69, male/female; 70–79, male/female; 80+, male/female.

Sharing assessments: Assessment reports provide invaluable information to other humanitarian agencies, create baseline data and increase the transparency of response decisions. Regardless of variations in individual agency design, assessment reports should be clear and concise, enable users to identify priorities for action and describe their methodology to demonstrate the reliability of data and enable a comparative analysis if required.


One of the key issues that is hindering effective humanitarian coordination is that information is not being shared effectively between the various response organizations. Many of them don’t see value in sharing information and often feel that sharing information with others will hurt their own ability to gather funds and drive their own programs forward.

What we need is a Humanitarian Information Charter that describes the core principles that govern humanitarian information sharing and management. These should define why organizations should share and as organizations endorse this charter they commit to sharing information with each other.

It is however not enough to tell organizations to share. Information needs to be shared in such a manner that it can also be compared to other information and analyzed for trends. However during almost every recent emergency data being shared has not been compatible with data coming from other organizations. Lot of effort has been needed to convert the data into compatible formats and often the analysis is delayed so long that the data becomes irrelevant by the time it becomes available. This in return leads to organizations not seeing any value in sharing information.

It is amazing that we have had organizations like UNGIWG active for over 10 years and we have had the global clusters for over 5 years now and the IASC Task Force on Information Management active for over two years now  and yet none of these have managed to agree upon standards for representing the information required to effectively coordinate disasters.

Those of us sitting in some of these bodies and having representatives in them must take the blame for not putting focus on the right things in our efforts there. If we want information sharing then we must ensure information interoperability. We ensure information interoperability by defining the data standards for how to share each type of information.

We have 20-30 types of spreadsheets and databases for each dataset that we want to capture. Now that we have finally agreed upon what the common and fundamental datasets are, then we must agree upon the format for sharing them. Once we have defined that standard, then we must actually agree to use it and nothing else.

We must then go through each cluster and ensure we define the core standards for each dataset that needs to be captured and shared to ensure effective coordination in the cluster.

Once we have the standards defined, we can actually start sharing templates and databases for collecting this data. Then we can even move forward and start sharing data capture applications and analysis modules. Then we can actually start comparing data from different organizations.

It is important for all of us to stop arguing about politics for a while and start addressing this core issue. We must understand that no data standard will be perfect and we must move towards minimum data standards and not perfect data standards.

I hereby challenge all the global cluster members as well as all the workgroup and task force members to give themselves 6 months to agree upon these standards. What we have at the end of six months will what we will use as the version 1 of the Humanitarian Information Standards. Aim for simplicity and interoperability instead of perfection and silos of data.

I am ready to work on a Humanitarian Information Charter and put together the minimum standards for humanitarian information sharing – are you?

8 comments:

  1. Such an important issue! I hope the right people will read this!

    ReplyDelete
  2. Honourable cause and good ideas.
    However since the 90's in this business I have always believed that there is some intrinsic incompatibility between the mindset of those who work in emergency response and those who accept standards. Many standards have been proposed, but I have rarely heard of proposals to accept standards put in use by others.

    When disaster strikes its a wonderful opportunity to re-invent the wheel again.
    What fun!

    ReplyDelete
  3. George - one of the key reasons for that is those few standards that exist have taken years to be created and have aimed for perfection in representation of all possible data they were supposed to cover. This means that they become too complex to use in the field. That is why we must have simple, usable standards that are easy to use in the field, within spreadsheets, databases, mobile forms, etc.

    ReplyDelete
  4. Hi Gisli

    Thought I'd reply to your tweet in more than 140 chars. I agree that we need standards, but I'm not sure that 6 months is a reasonable deadline or that the Clusters are the right organizations to solve the problem. Creating standards is *tough*, and the simplest ones are often the hardest to get right. For example, look at the very long discussions for the last 12 months just on missing persons standards at:
    http://groups.google.com/group/personfinder
    One of the smartest engineers I know, Ka-Ping Yee, is heading up the creation of the standard (person finder interchange format) but even this small piece of humanitarian information standards will take a couple of years to get right.

    The Sphere Project did not create standards (even if that's what they called them). The project created a well-thought-out set of principles and protocols, but that's it. It's not the same thing as standards and it doesn't aid the actual implementation of sharing. Take these two sentences:
    "The exact frequency of data-sharing is agency- and context-specific but should be prompt to remain relevant. Sensitive information should remain confidential"
    What are 'prompt' and 'relevant'? Who defines them in a given context: is it the agency receiving the information or passing the information? How does an agency convey when information is no longer relevant? Who decides what is 'confidential'? Can the existence of 'confidential' information be known without exposing the actual details? More importantly, what are the protocols for telling partner organizations to delete data that has become sensitive and ensuring that this takes place? How can organizations that share data ensure that partner organizations correctly report the actual source when republishing? The answers to these questions, and how to formalize them, are how to formalize them.

    Even if the precise categories of information are themselves very simple, all these factors still need to be carefully worked through. Of course, these complications are exactly *why* we need formal standards for information sharing. Creating standards is a specialized skill and most engineers in humanitarian work don't have that skill as they are application developers, not analysts. This is the real reason that we don't yet have good sharing standards: the people previously tasked with creating them are either not technical or the wrong kinds of engineers.

    I agree we need to create these standards, but we need to first identify the right people to create them.

    ReplyDelete
  5. Hi Robert,

    I agree that creating standards is a tough job that takes a long time to do perfectly. However I feel that we can get simple standards for the core data we need to share quickly out the door. We can then take years to make them more robust and perfect.

    I also think you need a combination of people to create the standards. If you only use application developers then you end up with something not usable by the humanitarians. If you use humanitarians only you end up with something not usable by the application developers. So the right mix i what is needed.

    We then have a bunch of stuff to use as guidance. Templates from various organizations, existing applications (for example Sahana) but all we need is the willingness to work together to solve this project. That lack of willingness has been missing for years. We need to make this one of our core priorities within the clusters and the IASC and ensure we have something ready for initial use in 6 months, not 12, not 18, not 24 (I guess that means teaching the UN to work at light speed).

    ReplyDelete
  6. Hi Gisli,

    you cannot even imagine how long I have been waiting for someone concretely addressing these problems of humanitarian information data standards...

    I fully agree on all your analysis and yes, I'm ready to work with you (if you want) on a Humanitarian Information Charter putting together the minimum standards for humanitarian information sharing as well as on the core sectoral datasets to be shared by all actors, hopefully into a unified geodatabase.

    On my opinion the UN clusters won't be able to define the sectoral common fundamental datasets in a reasonable time and we should think about the opportunity to define them as well... the ideas that I shared during my ignite Talk at the UN-SPIDER expert meeting in Vienna were going exactly in this direction…

    I hope we will find ways to work together on it,

    @srgdrz

    ReplyDelete
  7. You're right that application developers aren't going to accomplish much by themselves. To be honest, application developers wouldn't be much help at all.

    Creating formal standards is simply a different skill set. As an analogy, a doctor could spend a whole career treating genetic illnesses and it would not make them more qualified to sequence a genome. Similarly, you could spend a whole career as an application developer and not be any wiser about how to correctly design the standards for data interchange. Technology people in the humanitarian world are almost all application developers. I think it's a great idea, but someone is going to need to front the money/resources necessary to undertake the systems analysis and design for data interchange standards and make sure that the right people are creating them.

    I'd suggest changing the challenge a little. Rather than standards for sharing between organizations, what about a mandate for making data public? Sharing information between aid orgs might just be extending the size of the information silo. A combination of age and leapfrog technologies means that the average aid worker is probably *worse* at distributed information/communication management than the average global citizen. Until that changes, I think you could make a good argument that making data public should be prioritized over sharing between professional aid organizations. At the very least, we could reach a point where organizations were simply open about what data/information they possessed.

    Just a suggestion, of course. Would love to see open data sharing as much as anyone!

    ReplyDelete
  8. And grey is all theory...

    In practice I am pretty confident that existing solutions will sooner or later materialize as de-facto standards for notification services. A common set of data interchange formats (like CAP is, taking it to a very simple level) and API sets will pave the way. Also regulations eg. within the US and EU - currently being implemented along the different projects - will deliver best practice to other markets and regions.

    I do agree - it is time to act and get things going. Being myself relatively new to the subject though it could be of benefit to be one of the prime drivers for this. I am sure that soon these de-facto implementations will be of benefit for all others who are ready to deploy such solutions.

    There is some willingness needed though from the bodies to support trials and first market implementations...!

    ReplyDelete