Big data and global development

A primer on using online and mobile data to make the world a better place

The data revolution is not restricted to the industrialized world. The spread of mobile phone technology into the hands of billions of individuals may be the single most significant innovation that has affected developing countries in the past decade.

Across the developing world, mobile phones are used daily to transfer money, buy and sell goods, and communicate information including test results, stock levels and prices of commodities. Mobile technology is used as a substitute for weak telecommunications and transport infrastructures as well as underdeveloped financial and banking systems.

The numbers of real-time information streams and people using social media are growing rapidly in developing countries as well. Tracking trends in online news or social media can provide insights on emerging concerns that can be highly relevant to global development.

The recent waves of global shocks – food, fuel and financial – have led to greater volatility, and policymakers are increasingly aware of the social and financial impact. Despite greater interconnectivity, local impacts of shocks like food crises or natural disasters may not be immediately visible and trackable.

These are important issues that often unfold beneath the radar of traditional monitoring systems, and by the time hard evidence finds its way to the front pages of newspapers and desks of decision makers, it’s often too late and more expensive to respond. While early-warning systems and data collected through “traditional” methods (surveys and statistics) continue to generate relevant information, the digital revolution presents a tremendous opportunity to gain richer insight into the human experience, and big data can complement the existing indicators.

What is big data for development?

“Big data for development” is a concept that refers to the identification of sources of big data relevant to the policies and planning for development programs. It differs from both “traditional” development data and what the private sector and mainstream media call “big data.”

In general, sources of big data for development are those which can be analyzed to gain insight into human well-being and development, and generally share some or all of the following features:

1. Digitally generated: Data is created digitally, not digitized manually, and can be manipulated by computers.
   
2. Passively produced: Data is a by-product of interactions with digital services.
   
3. Automatically collected: A system is in place that automatically extracts and stores the relevant data that is generated.
4. Geographically or temporarily trackable: For instance, this is the case in mobile phone location data or call duration time.
   
5. Continuously analyzed: Information is relevant to human well-being and development, and can be analyzed in real time.

Big data for development is constantly evolving. However, a preliminary categorization of sources may reflect:

• What people say (online content): International and local online news sources, publicly accessible blogs, forum posts, comments and public social media content, online advertising, e-commerce sites and websites created by local retailers that list prices and inventory.
• What people do (data exhaust): Passively collected transactional data from the use of digital services such as financial services (including purchase, money transfers, savings and loan repayments), communications services (such as anonymized records of mobile phone usage patterns) or information services (such as anonymized records of search queries).

Before it can be used effectively, big data needs to be managed and filtered through data analytics - tools and methodologies that can transform massive quantities of raw data into “data about the data” for analytical purposes. Only then it is possible to detect changes in how communities access services that may be useful proxy indicators of human well-being.

If properly mined and analyzed, big data can improve the understanding of human behavior and offer policymaking support for global development in three main ways:

1. Early warning: Early detection of anomalies can enable faster responses to population in times of crisis.
2. Real-time awareness: Fine-grained representation of reality through big data can inform the design and targeting of programs and policies.
3. Real-time feedback: Adjustments can be made possible by real-time monitoring of the impact of policies and programs.
   

Global Pulse is a United Nations innovation initiative of the Secretary-General, exploring how big data can help policymakers gain a better understanding of changes in human well-being. Through strategic public-private partnerships and R&D carried out across its network of Pulse Labs in New York, Jakarta and Kampala, Global Pulse functions as a hub for applying innovations in data science and analytics to global development and humanitarian challenges.

Big data analytics is not a panacea for age-old development challenges, and real-time information does not replace the quantitative statistical evidence that governments traditionally use for decision making. However, it does have the potential to inform whether further targeted investigation is necessary, or prompt immediate response.

Big data and global development examples

To evaluate the effectiveness of harnessing big data for development, UN Global Pulse has worked on several research projects in collaboration with public and private partners. Such proof-of-concept projects and prototypes demonstrate how big data analysis can be beneficial to the work of policymakers in different contexts – from monitoring early indicators of unemployment hikes to tracking fluctuations of commodity prices before they are recorded in official statistics.