This article has been updated to reflect the 2019 data.

Thank you for your feedback!

Thank you very much to everyone who has taken the time to tell us what you think about our human rights data. We’re constantly working on making our data better, so your feedback is always welcome and very useful. We’ve received a fair number of comments on our economic and social rights metrics (the green petals on our radar charts). While much of that feedback has been positive, many of you have raised questions. In particular, some of you have felt that the scores for countries you are familiar with are too high. For example, we heard from one group of advocates focusing on Zimbabwe, Zambia, and Malawi that the scores on the rights to food, health, and work seemed too high in comparison with how they saw the situation on the ground.

A summary picture of Zimbabwe’s human rights performance

Note: Economic and social rights scores are calculated using the core assessment standard. Civil and political rights scores have not yet been calculated for Zimbabwe. On the face of it, we understand what you are saying. Looking at the graphs above, it might look like we’re saying that that 80.2% of people in Zimbabwe have enough food. In fact, our metrics are a bit more complicated than that. What we are actually measuring is how well Zimbabwe is doing on fulfilling its people’s human rights relative to how well it ought to be able to do so, given the level of resources it has.

So what exactly do HRMI’s economic and social rights metrics measure?

A good way of thinking about HRMI’s metrics is that they are a way of assessing how well each country is using its available resources to achieve respect for each right. By contrast most other indicators (eg the United Nations’ Sustainable Development Goals) look strictly at the level of rights enjoyment in the country; that is, they might track what proportion of people in that country are getting adequate nutrition (the right to food), but without taking into account the income level of the country. Let’s consider an example. A good indicator of adequate nutrition is the child stunting rate. In 2014, Zimbabwe’s child stunting rate was 28%.  That looks like a lot of children who are malnourished! But the question then arises, “Could Zimbabwe do better?”  After all, Zimbabwe has a GDP per capita level of only $2000 (measured in a way that equates purchasing power between countries and adjusts for inflation) and perhaps cannot be expected to ensure the right to food for everyone. To what extent should it be ensuring that people have access to adequate nutrition?  Is it meeting its obligations under the International Covenant for Economic, Social, and Cultural Rights with regard to the right to food?

Traditional indicators can’t answer those questions. HRMI metrics can. To put it another way, HRMI’s right to food score for Zimbabwe is not evaluating how well-nourished the population is, but rather, the extent to which Zimbabwe is doing as much as it can to fulfil its people’s right to food. Looking at the chart above you can see that Zimbabwe gets an 80.2% score on the right to food. This tells us is that Zimbabwe is only doing 80.2% as well as it should be able to do. Indeed, digging into the numbers in more detail we learn that no country as poor as Zimbabwe has been able to fulfil the right to food for all its people. But based on what other poor countries have achieved we know that Zimbabwe should be able to reduce its stunting rate to around 19.8%.

Why does HRMI use this way of measuring?

The short answer is that this is what most countries have signed up to in international law. Article 2 of the International Covenant for Economic, Social, and Cultural Rights (which has been ratified by almost 170 countries) recognizes that resource constraints will prevent poorer countries from immediately fulfilling economic and social rights.  But it obligates them “to take steps… to the maximum of [their] available resources…” to progressively fulfil economic and social rights. The percentage score on our HRMI metrics tells you the percentage level of enjoyment achieved on a given right relative to what should be feasible for a country with that per-capita income level. More generally, we are evaluating how effectively a country translates its resources into the enjoyment of the different economic and social rights. It’s not only because it’s in international law that we do it this way. It also makes sense. Many of you tell us that you love this approach because it puts the onus of responsibility on all states to do their best, regardless of their level of income. Experts have also given this methodology a big tick, for example by awarding the book I wrote on this methodology (with my co-authors Sakiko Fukuda-Parr and Terra Lawson-Remer) a best human rights book prize in 2016.

My country is not poor but its scores also look too high!

Many of you from richer countries have also told us that the scores for your countries look too high. In this case, the explanation is usually different from the Zimbabwe example above, and it’s typically related to inequality and discrimination. You are probably well aware of the disparity in rights enjoyment between the general population and vulnerable sub-populations (for example, refugees, particular ethnic groups, etc). So you may find a relatively high score on one of our HRMI metrics inconsistent with what you see on the ground. Even if a minority group’s enjoyment level of a particular right is dramatically lower than the general population’s, if that minority group comprises a small proportion of the population, the country’s score on our metric can be quite high. We can, however, show how serious the problem of discrimination is, by disaggregating our economic and social rights metrics, splitting them up by population subgroup. A good example is the education quality component of Australia’s score on our right to education metric. Australia’s aggregate score in 2015 was a troubling 74%: that is, Australia was only doing 74% of what it ought to be able to do to fulfil its people’s right to education, given how rich Australia is. To see whether Australia was violating its legal obligation not to discriminate, we calculated this score separately for non-indigenous and indigenous students. We found the score for non-indigenous students was 75%, barely better than the aggregate score. However, the score for indigenous students was a deeply concerning 47%, reflecting very serious levels of historical or current discrimination. This example also illustrates how if a population subgroup is small relative to the total population, even when that subgroup is subject to serious discrimination, it will not be reflected in the country score on our aggregate metrics.

In future we hope to address this concern by calculating our metrics separately for different population subgroups. This will provide a much richer picture of a country’s performance in meeting its economic and social rights obligations. In addition to undertaking some disaggregated studies ourselves, we would be happy to provide support for you to learn how to disaggregate the scores for your country. Please feel free to reach out to us.

A Beginner’s Guide to Interpreting these Metrics

Our focus on the extent to which countries meet their obligations to fulfill economic and social rights (as opposed to the level of rights enjoyment) leads to some results that, without reflecting more directly on what it is our metrics capture, can seem counter-intuitive. In particular:

• A poor country can be completely meeting its obligation to fulfil economic and social rights – that is, achieving a score of 100% on our metrics – even when a large proportion of the population fails to enjoy the right.
• A wealthy country can have very high rights enjoyment levels, yet still receive a score well below 100% on our metrics.
• The scores on our rights metrics shouldn’t be interpreted in the same way as grades on a school exam. A score of 90% is not passing. Any score less than 100% is a failing score – the country has failed to meet its obligations under the International Covenant on Economic, Social and Cultural Rights (ICESCR) to fulfill economic and social rights to the best of its ability.
• Even if a country scores 100% on our metrics, it could be in violation of the non-discrimination and other obligations under the ICESCR.
• Two countries with the same scores on HRMI’s economic and social rights metrics but different per-capita incomes, will not have the same rights enjoyment level.

So, what is next?

We hear and understand your concerns about some of the ways our metrics can be misinterpreted. What we’ve done in this blog post is help to explain why this problem is happening. How else do we plan to get our information out more successfully? Here’s what we have in mind:

• As mentioned above, we need to do much more disaggregation of our metrics by gender, race, ethnic group etc, and present this data alongside the aggregate scores.
• We have ideas for providing more detail and better explanations on our Rights Tracker. For example, if we compare Zimbabwe’s stunting rate of 28% with the much better 18% stunting rate that should be achievable, would that help make the scores seem more intuitive?
• Short videos can make information a lot more accessible than long blog posts, so we have produced some. Please follow us on Twitter and Facebook and share our videos when we release them.
• We tested a lot of these ideas, and co-designing new ideas, in the workshop we ran (with Amnesty International) in Johannesburg in September 2018. We look forward to seeing what solutions emerge!

If you would like to help fund this work, please get in touch.