Today we feature a guest post from Professor Morten Jerven (Professor of Development Studies, International Environment and Development Studies (Noragric) Norwegian University of Life Sciences), based on his presentation at Global Health Histories Seminar 101. You can access a recording of the entire event at www.youtube.com/CGHHYork
The most challenging notion to take on board in the governance of today’s world is that not all that counts can be counted. We increasingly rely on numbers as shortcuts to information about the world that we do not have time to digest.
The name of the game is governance “as if” the world counts. It might be a smart shortcut sometimes, but we are in deep trouble if we forget that we are doing it “as if” the world counts. Leadership should take making good decisions seriously. If the method by which we get knowledge and the method by which we make decisions is limited to what can be numbered, we are setting up a system of governance that’s systematically getting stuff that actually counts wrong.
Unfortunately, we are being led down the wrong path by the United Nations and its experts. In 2014, the U.N. High-Level Panel delivered its report with recommendations for the Sustainable Development Goals, subsequently to be adopted by the U.N. General Assembly in 2015. One small aspect of the report very soon caught everyone’s attention. Buried on page 8 was a call for a “data revolution” in development. It generated a frenzy of enthusiasm among the international development community.
Later the same year the secretary-general’s Independent Expert Advisory Group on a Data Revolution for Sustainable Development put forward its recommendation, titled “A world that counts.” The report laid out a grand ambition: It recognized that currently “whole groups of people are not being counted and important aspects of people’s lives and environmental conditions are still not measured.”
From that acknowledgement it took a surprising next step. From now onwards, the report declared, “Never again should it be possible to say, ‘We didn’t know.’ No one should be invisible. This is the world we want — a world that counts.”
It is not the world I want. The most important things in this world are the things that we cannot count. The most marginalized issues are those issues that, willfully or not, remain and will remain uncounted. That should be the first principle when it comes to making plans for global governance. Yet, on expert advice, the U.N. did exactly the opposite. In the year 2000 the world adopted the eight Millennium Development Goals, 18 targets and 60 indicators, and this year, intoxicated on what the establishment perceived as a ringing success, laid down the path for the next 15 years with 17 goals, 169 targets and so far over 1,000 suggested indicators.
I bet the majority actually believed the U.N. when we were told that world poverty was halved by the MDGs. It was an elaborate hoax on many levels: To begin with it was a basic lesson in “how to lie with statistics.” World poverty has been on the increase until very recently. Last time we had some data, the total number of poor was still on the increase in sub-Saharan Africa. It would have been most accurate to say, “We don’t know.” Failing that, one could have said that probably there have never been more poor people on the planet than right now. But the story they were selling was that the ratio of extremely poor people to not so poor people in poor countries was halved.
Except, we don’t know that. We actually do not have a good grasp of the total number of poor people today or in 1990. The freshly minted Nobel Prize laureate Sir Angus Deaton, memorably called measuring poverty and agreeing upon who is below and above the magic one dollar and something cents day line “a statistical problem from hell.”
According to a study using World Bank data, for the 150 countries it monitored poverty in between 1990 and 1999 only in 41 countries, not even a third, was there “satisfactory” poverty data. That meant two or more points of observation. First, famously you need two points to draw a line; the majority of countries had less. Second, for 50 of the 150 countries there was no poverty data. At all.
U.N. Secretary-General Ban Ki-moon should not step up to the podium to declare anything about halving poverty. Not yet anyhow. The 2015 data will not be ready until sometime in 2018. So any pronouncements about what has happened or hasn’t happened by 2015 is not telling us anything about poverty in the world, but is rather giving us some strong indications about the relative importance of public relations versus real knowledge when it comes to counting poverty.
Even when this 2015 number is ready, it is still largely guesswork how it relates to actual poverty in the majority of the countries. According to the World Bank study, for the last decade for which we actually have some data (2002-2011) there was satisfactory data for 63 out of 155 countries, whereas 29 countries are just borrowing poverty data from other countries because they have none themselves.
The number which we spend so much time and energy on trying to understand and to collect data for is largely irrelevant for actually doing something about poverty. It is a classic case of aggregating something upwards to provide an indicator, but not being able to disaggregate down again to find a poor person. Because it is an indicator. It is like the weather. If someone tells you it is 17 degrees Celsius it won’t give you a grip on whether it is warmer or colder than it should be. Most fundamentally, it contains no information as to what causes temperatures to go up or down. So it may be counting, but it fails the basic test of accounting. That is, being able to tell you what caused stuff to go up or down.
More fundamentally, it is also fails the accountability test with flying colors. Let us recount our steps. The step from just counting stuff, to actually accounting for stuff is to say “Hey, I know why the numbers stack up like this,” instead of just grunting, “It’s 324.87.”
Accountability is the crucial step. It is about who is responsible for what, and who owes what to whom. That’s why adding up assets and liabilities matters. Someone’s ass is on the line.
If every Malawian under the dollar per day line had a dollar each time some U.N. or nongovernmental organization person having some claim to work under the MDGs umbrella implied that they had something do with “halving world poverty” between 1990 and 2015, we would have gone some way to relieving poverty in Malawi. Frequently heard is the phrase that the MDGs “contributed to” halving poverty. It does not pass the accountability test. The U.N. MDGs taking credit for halving poverty is as credible as it would be to blame the U.N. for all the people who died in civil war during the same period, because civil war deaths were not counted as part of the MDG exercise.
Our knowledge problem in development statistics is double biased. We know less about poor countries, and less about the poor people in those poorer countries. This knowledge problem is replicated in big data and may be worse. A traditional survey misses, by design, criminals, the homeless, refugees, nomads and the sick — if you do a survey on mobile phones, or capture passively exhausted data from smartphones, you miss most of the population, and you lose the poorest part (most of these people live in rural areas).
If high-quality data are scarce in supply, we must be very conscious about our demands on data. The deeper point for data users in the development community here is that numbers need to be interrogated meticulously. Confronted with secondary data in international databases, users need to conduct basic source criticism and ask “Who made this observation?” “Under what conditions was this observation made?” and “Is there any reason to think that the observation is biased?” Failure to do so increases the distance between the observer and the observed and may lead to a disconnect between reality and the numbers.
Even if there was a world where “superbureaucrats” counted everything correctly, we must not forget that all the time we have pretended to approach poverty “as if” it can be counted. Decades of research has told us that poverty is multidimensional, contextual and cannot be reduced to dollars and cents. Let us keep that in mind for a while before we move ahead and continue governing the world “as if” it counts.
According to a study using World Bank data, for the 150 countries it monitored poverty in between 1990 and 1999 only in 41 countries, not even a third, was there “satisfactory” poverty data. That meant two or more points of observation. First, famously you need two points to draw a line; the majority of countries had less. Second, for 50 of the 150 countries there was no poverty data. At all.
U.N. Secretary-General Ban Ki-moon should not step up to the podium to declare anything about halving poverty. Not yet anyhow. The 2015 data will not be ready until sometime in 2018. So any pronouncements about what has happened or hasn’t happened by 2015 is not telling us anything about poverty in the world, but is rather giving us some strong indications about the relative importance of public relations versus real knowledge when it comes to counting poverty.
Even when this 2015 number is ready, it is still largely guesswork how it relates to actual poverty in the majority of the countries. According to the World Bank study, for the last decade for which we actually have some data (2002-2011) there was satisfactory data for 63 out of 155 countries, whereas 29 countries are just borrowing poverty data from other countries because they have none themselves.
The number which we spend so much time and energy on trying to understand and to collect data for is largely irrelevant for actually doing something about poverty. It is a classic case of aggregating something upwards to provide an indicator, but not being able to disaggregate down again to find a poor person. Because it is an indicator. It is like the weather. If someone tells you it is 17 degrees Celsius it won’t give you a grip on whether it is warmer or colder than it should be. Most fundamentally, it contains no information as to what causes temperatures to go up or down. So it may be counting, but it fails the basic test of accounting. That is, being able to tell you what caused stuff to go up or down.
More fundamentally, it is also fails the accountability test with flying colors. Let us recount our steps. The step from just counting stuff, to actually accounting for stuff is to say “Hey, I know why the numbers stack up like this,” instead of just grunting, “It’s 324.87.”
Accountability is the crucial step. It is about who is responsible for what, and who owes what to whom. That’s why adding up assets and liabilities matters. Someone’s ass is on the line.
If every Malawian under the dollar per day line had a dollar each time some U.N. or nongovernmental organization person having some claim to work under the MDGs umbrella implied that they had something do with “halving world poverty” between 1990 and 2015, we would have gone some way to relieving poverty in Malawi. Frequently heard is the phrase that the MDGs “contributed to” halving poverty. It does not pass the accountability test. The U.N. MDGs taking credit for halving poverty is as credible as it would be to blame the U.N. for all the people who died in civil war during the same period, because civil war deaths were not counted as part of the MDG exercise.
Our knowledge problem in development statistics is double biased. We know less about poor countries, and less about the poor people in those poorer countries. This knowledge problem is replicated in big data and may be worse. A traditional survey misses, by design, criminals, the homeless, refugees, nomads and the sick — if you do a survey on mobile phones, or capture passively exhausted data from smartphones, you miss most of the population, and you lose the poorest part (most of these people live in rural areas).
If high-quality data are scarce in supply, we must be very conscious about our demands on data. The deeper point for data users in the development community here is that numbers need to be interrogated meticulously. Confronted with secondary data in international databases, users need to conduct basic source criticism and ask “Who made this observation?” “Under what conditions was this observation made?” and “Is there any reason to think that the observation is biased?” Failure to do so increases the distance between the observer and the observed and may lead to a disconnect between reality and the numbers.
Even if there was a world where “superbureaucrats” counted everything correctly, we must not forget that all the time we have pretended to approach poverty “as if” it can be counted. Decades of research has told us that poverty is multidimensional, contextual and cannot be reduced to dollars and cents. Let us keep that in mind for a while before we move ahead and continue governing the world “as if” it counts.
Morten Jerven
