Many around the world celebrated the agreement of the Sustainable Development Goals (SDGs) and a new agenda for transformative development by 2030. But, practitioners and policy makers were left scratching their heads as to how they were going to monitor the detailed 169 targets and ever more numerous indicators, never mind understanding and achieving these goals.
It is in this context that we’re seeing a growth of interest in using data to help solve development problems. Indeed, we can say that the infrastructures now being built to support data are likely to become central to how we make development decisions in the future.
How will such data infrastructures shape our thinking about development over the next decade? What types of limitations and biases might they embed? How should they best be designed and implemented? It is these questions that we looked to explore in a recent paper  analysing big data use for development in India.
In this paper we dug into two cases where big data was being used to support wider development over commercial goals – the Bengaluru Metropolitan Transport Corporation (BMTC) and big data transport upgrades in Bengaluru, India; and Stelcorp (name changed), a state initiative using big data for improving electricity systems.
Digging into big data
Digging into these cases, we found that both of these initiatives were connected into longer, often decades-old histories of data collection and decision making. This meant that new data innovations were being introduced in an attempt to understand long running development problems. Thus, the main focus of BMTC was on using vehicle tracking and big data innovations to improve the notoriously unreliable city bus services.
We found that big data innovation allowed improved integration of rich information flows, and led to centralisation of decision making. In StelCorp, previously manually-collected meter data was now digitally-collected and aggregated (see images below). The supporting infrastructure allowed a near real-time analysis of the status of the electricity network, and was more effective at monitoring around failures and blackouts. A new central data centre played a growing role in processing and analysing this data. In BTMC, new bus transportation data was aggregated and fed in real-time to large screens in a “control centre” where activity was monitored by administrators.
Beyond day-to-day monitoring, we also saw signs that the new data was feeding into more strategic decisions. In the electricity sector, for example, upgrades have been plagued by poor and politicised decision making, but the state-wide data from Stelcorp is now being used in upgrading decisions.
More conceptually, there is evidence that these initiatives are playing a role in supporting new forms of state commitments, or citizen interaction. BTMC has been associated with a ‘Smart City’ initiative and citizens interacting with a set of efficient urban services. Indeed, BTMC introduced a citizen mobile app for tracking bus routes which has had over 50,000 downloads. In the Stelcorp initiative, state political visions about “24/7 electricity” have in part emerged from the better data that allows improved management of the electricity system.
Whilst big data has led to these operational, strategic and visionary advances, there were a number of concerns in these projects. One key concern raised was the quality of data being used in these projects, which was often incomplete, short-term, or skewed.
Most problematic was that data from marginal groups was difficult to obtain, so in Stelcorp, automated electricity data was mainly coming from cities, where rural data was still manually collected, and in both cases there was often the need for “data wrangling” before the data had value.
These data limitations pose questions of how representative the data being used is of the population. If certain measures are skewed towards those more affluent, data coming from those more marginal might then be seen as “nonconforming” or even deviant. Moreover, the way that the data is selected, measured and transformed in such systems will be important in determining what processes are made visible by data and what might remain in the shadows.
There were also more general questions about the focus of big data projects. These projects were marketed and discussed under lofty development goals, but in implementation they were often quite narrow projects. BTMC, for all its discussion of smart cities and citizens, was far more focussed on stamping out corruption among bus employees than making the city’s public transport smart.
Further, in all these projects there is scant sharing of the new data produced. These projects have not been about the public shining a light on opaque mechanisms of decision making. In fact, with a growing number of public and private actors involved, mechanisms of decision making are becoming even less transparent.
Big data for development
Big data projects are in their infancy in countries like India, but as these cases show they are becoming important to support decision making on key development issues, not only at an operational level, but in strategic decision making and in supporting new visions of developmental partnerships between citizens, private sector and the state.
However, these initiatives rarely follow the vision of big data driving transformative changes. They so-far tend to use problematic data to enhance decision making. They also tend to focus on quite narrow aspects of problems in implementation over the bigger development problems that might be more impactful.
We also need to make sure that big data does not solely lead to technocratic solutions, or underplay the importance of integrating with a wider set of social and political activities for development – data showing electricity pilferage will have limited impact without solving the complexities of local politics of electricity in rural and slum areas, and data on public vehicle movements cannot replace the underfunding of urban transport.
 Heeks, R., Rakesh, V., Sengupta, R., Chattapadhyay, S. & Foster, C. (In press) Datafication, Value and Power in Developing Countries: Big Data in Two Indian Public Service Organisations. Development Policy Review.
This is an adapted version of a blog originally posted on the Sheffield Institute of International Development (SIID) blog.
With thanks to Vanya Rakesh & Ritam Sengupta for their research in India and SIID and the University of Manchester for the small grant support for this work.Follow @CDDManchester