Archive

Posts Tagged ‘Researching ICT4D’

A Research Agenda for Data-Intensive Development

18 July 2016 1 comment

In practice, there is a growing role for data within international development: what we can call “data-intensive development”.  But what should be the research agenda for this emerging phenomenon?

On 12th July 2016, a group of 40 researchers and practitioners gathered in Manchester at the workshop on “Big and Open Data for Development”, organised by the Centre for Development Informatics.  Identifying a research agenda was a main purpose for the workshop; particularly looking for commonalities that avoid fractionating our field by data type: big data vs. open data vs. real-time data vs. geo-located data, etc; each in its own little silo.

IMG_0828

A key challenge for data-intensive development research is locating the “window of relevance”.  Focus too far back on the curve of technical change – largely determined in the Western private sector – and you may fail to gain attention and interest in your research.  Focus too far forward and you may find there no actual examples in developing countries that you can research.

In 2014 and 2015, we had two failed attempts to organise conference tracks on data-and-development; each generating just a couple of papers.  By contrast, the 2016 workshop received two dozen submissions; too many to accommodate but suggesting a critical mass of research is finally starting to appear.

It is still early days – the reports from practice still give a strong sense of data struggling to find development purposes; development purposes struggling to find data.  But the workshop provided enough foundational ideas, emergent issues, and reports-back from pilot initiatives to show we are putting the basic building blocks of a research domain in place.

But where next?  Through a mix of day-long placing of Post-It notes on walls, presentation responses, and a set of group then plenary discussions[1], we identified a set of future research priorities, as shown below and also here as PDF.

DID Research Agenda

 

 

The agenda divided into four sub-domains:

  • Describing/Defining: working out the basic boundaries, contours and contents of the data-intensive development domain.
  • Practising: measuring and learning from the practice of data-intensive development.
  • Analysing: evaluating the impact of data-intensive development through various analytical lenses.
  • Resisting: guiding practical actions to challenge potential state and corporate data hegemony in developing countries.

Given the size and eclectic mix of the group, many different research interests were expressed.  But two came up much more than others.

First, power, politics and data-intensive development: analysing the power structures that shape DID initiatives, and that are inscribed into data systems; analysing the way in which DID produces and reproduces power; analysing what resistance to data hegemony would mean.

Second, justice, ethics, rights and data-intensive development: determining what a social justice perspective on DID would mean; analysing what DID can contribute to rights-based development; understanding how ethical principles would guide civil society interventions for better DID.

We hope, as a research community, to take these and other agenda items forward.  If you would like to join us, please sign up with the LinkedIn group on “Data-Intensive Development”.

 

[1] My thanks to Jaco Renken for collating these.

Advertisements

Discussing ICTs and the SDGs

5 January 2016 1 comment

Now the Sustainable Development Goals are with us, what are the implications for ICT4D?  A recent discussion held by members of the Centre for Development Informatics gave some pointers.

The MDGs have run their course, achieving a mixed bag of success. The post-2015 Sustainable Development Goals (SDGs) – an ambitious set of 17 goals and 169 targets – take over the proverbial baton in the global race towards achieving, what has been described as “the world we want”. There are criticisms of the efficacy of these types of goals and the processes by which they are derived.  But they provide a starting point and framework around which actors with varied mandates can gather. Indeed, the SDGs have already begun to shape the development discourse, development models and development funding mechanisms.

The discussion was initially motivated by a blog post from Tim Unwin where he critiques the limited role of ICTs within the SDGs.  While several discussants sympathised with many of the points raised in Unwin’s article, others took an alternate view. Too great a presence for ICTs could risk re-kindling the ICT4D hype-cycles that generated unrealistic expectations in the 1990s and early 2000s. If the telecentre age taught us anything, it is that overemphasising the ability of ICTs to generate development outcomes is counterproductive for developing communities, as well as for donor and ICT communities.

Others argued that the low profile for ICTs was encouraging because it reflected the times in which the SDGs were written: a recognition of the embeddedness and pervasiveness of ICTs within a progressively digital society. Consequently, not only are ICTs now seen as instrumental, they have become a platform through which development activities are increasingly mediated. For instance, even if not explicitly mentioned, it is impossible to conceive effective environmental monitoring that does not involve sensors, satellite imaging, and a solid infrastructure to handle the data generated. Additionally, ICTs are now raising development issues of their very own: digital identities, digital exclusion, privacy and security come to mind.

Another theme we tackled was the relationship between the SDGs and ICT4D research. The questions considered included: “Do we obtain our research agenda from the SDGs or from what we see happening in the world of ICTs? Should the engagement of the ICT4D academic community with our peers in policy and practice be informed by the SDGs?”.

There was consensus that, while the SDGs might not necessarily drive ICT4D research agendas, they can provide a vehicle and language through which we can make more explicit linkages between our research and the development issues of our day. Developmental progress is often seen to result from changes in behaviour. Identifying and fostering the factors that cause or inhibit behavioural change are, therefore, integral to development planning and policy-making. ICT4D researchers can improve the support we offer to policy, practitioner and entrepreneurial colleagues by providing better evidence of how ICTs impact behavioural changes that are aligned with the realisation of the SDGs. Therefore, we discussed the need for ICT4D researchers to become more adept at discerning issues of causality around human behaviour and ICTs.

As researchers motivated by global inequality and pressing social concerns, we felt our work should not just focus on addressing knowledge gaps but development gaps. Here, the SDGs provide guidance. Case in point, Goal 13 calls for urgent action against climate change and its impacts and a recent survey of ICT4D research identified significant gaps in our knowledge about ICTs, the environment and climate change. So, if you have a particular concern for the environment (perhaps we all should?) and are keen on starting a PhD, this might be an area on which to focus.

The example above highlights bigger questions about the relationship between knowledge gaps and development priorities and how knowledge gaps around particular development priorities, such as climate change, have remained scarcely addressed within our field. On this theme, we focused on how the SDGs can be used to bridge these gaps and priorities. One practical approach for academics and anyone interested in addressing development priorities within the ICT4D space – practitioner, policy maker, entrepreneur or combination – is to use the SDGs as a stepping stone to find that unique point where the wider social concerns of development, our desire to make a difference (personal actualisation), and sustainable mechanisms (through business, NGO, public agency, etc) intersect.

ICT4D Brown Bag Priorities

On Addressing Development Priorities through­ ICT4D

These are just a few ideas. We are curious to hear what others have to say and welcome your thoughts in the comments section below.

Written by Ritse Erumi, Juan Gomez and Ryo Seo-Zindy (CDI PhD Researchers)

Actor-Network Theory, Technology and Development

What can actor-network theory offer to our understanding of technology and development?

This blog entry summarises the answer from an open access paper in the journal Development Studies Research: “Technological Change in Developing Countries: Opening the Black Box of Process Using Actor–Network Theory”, and it builds on an earlier entry on ANT and development.

Technology rather dropped from the development agenda during the 1980s and 1990s, but has re-emerged strongly in the 21st century; not least due to the spectacular diffusion of ICTs.

Yet, to date, conceptualisation of technological change in developing countries has had three problematic gaps:

  • It has been de-humanised: organisations are recognised as actors but people – as identifiable individuals with agency – rarely appear in the technology and development literature.
  • Technology may be understood as a physical artefact, as a system of elements, as the embodiment of knowledge. But it is not seen as playing any active role: technology is acted-upon but is not itself acting.
  • Research has tended to study factors or social structures affecting processes of technological change. But it does not describe those processes in detail: actual practices of change tend to be black-boxed.

In sum, research to date has typically stood outside the technology processes it seeks to investigate; freezing them in time and concealing their main actors.

As luck would have it, these are just the kind of lacunae that actor-network theory was intended to address.  Yet application of ANT to cases of technological change in developing countries has been rare; and within development studies literature, almost non-existent.  So new ANT-based case studies of technology and development are required to assess what insights actor-network theory can offer.

One such case study – applying Callon’s “moments of translation” to a digital information system in the Sri Lankan public sector – is presented in the Development Studies Research paper (which should be accessed for full details).  It finds that an initial network supporting technological change fell apart in mid-project, and had to be reconstructed around a new technology design and a new vision for future change.

Three challenges emerged in applying ANT:

  • Methodological: admission of subjectivity in framing an ANT-based case, and problems of thinning out detail to fit a journal-length account.
  • Analytical: that ANT can provide a rich description of how things happen, but stutters in seeking to analyse why.
  • Instrumental: the difficulty of extracting practical guidance from ANT other than rather “Machiavellian” prescriptions.

On the other hand, the case analysis shows that ANT can open the black box of technological change processes and offer new insights:

  • Networks: explaining the networks of relations that both support and oppose technological change, and also the detailed process by which they come to be formed, dissolved, etc.
  • Technology: exposing the active role that technology plays in international development – shaping, enabling, co-operating, resisting, etc.
  • Human practices: providing a detailed account of the role played by individuals and groups in technological change; particularly the way in which lead actors modify the perceived interests and even identities of others involved.

ANT therefore shows us not just that human interests, identities and relations change in a technology-and-development project; it also explains in what way they change, how it is that those changes come about, and how they relate to the project’s trajectory.

The case analysis shows that ANT will not help answer questions about the impact of context on technological process, or about the developmental impact (in the traditional sense) of technology. However, it may help to answer questions such as:

  • How do we explain the trajectory of a technology and development project?
  • How does a particular innovation in a developing country diffuse, scale up or sink without trace?
  • What role does technology play in processes of technological change?
  • How does power manifest itself in such processes? How are apparently relatively powerless actors sometimes able to influence the direction of technological change? How are apparently relatively powerful actors sometimes not able to get their way on a technology project?

As the technology used in development becomes more complex, more interconnected, more intertwined into the lives and livelihoods of developing communities, and changing at an ever-faster pace; then ANT will likely become more relevant and more useful as a conceptual frame.

 

A Development 2.0 Research Agenda

25 April 2014 4 comments

A key theme in the post-2015 development agenda is transformation: a belief that the incremental developmental changes achieved to date will no longer be sufficient in the remainder of the 21st century; and an aspiration for a step-change in approach.

Analysis reported earlier argues development informatics research – studying ICT4D policy and practice – should give a higher priority to researching the relation between ICTs and the transformation of development.  Such research already has a terminology – Development 2.0; understood as the ICT-enabled transformation of development.

But what would the Development 2.0 research agenda consist of?

Defining that research agenda has been difficult because defining Development 2.0 has been difficult.  And defining Development 2.0 has been difficult because defining “transformation of development” has been difficult.

First, there is the threshold problem – when is a change sufficiently large to be classified as “transformative” as opposed to just “incremental”?  Second, there is the direction problem – transformation of what?  Of context (e.g. structures)? Of inputs (e.g. goals, visions, aspirations)?  Of processes (e.g. business models, partnerships)?  Of outputs (e.g. inclusion, sustainability)?

But uncertainty of this type can provide the basis for research.  We can use this, plus a few sources that do engage with Development 2.0 as the intersection of ICTs, transformation and development (Thompson 2008[1], Heeks 2010[2], Hanna 2011[3], Thompson 2013[4], Hanna 2014[5]), to give some outline shape to a Development 2.0 research agenda:

1. Definition: what does Development 2.0 mean?  This could start with content analysis of what little has been said and written about Development 2.0; looking for definition in terms of the extent and content of transformation of development.  Interpretive work on a broader range of stakeholder views could also be provided.

2. Conceptualisation: how should we understand Development 2.0?  Related to definition, this might attack the issue in a more deductive manner by seeking to conceptualise Development 2.0 through particular theoretical lenses drawn from development or informatics studies or other disciplines.

3. Political Economy: who drives Development 2.0?  Who are the main stakeholders arguing for ICT-based transformation of development?  Why are they putting forward these arguments?  Who benefits from this discourse?

4. Ecosystem: who and what makes up a Development 2.0 ecosystem?  A Development 2.0 ecosystem is that combination of organisations (government, private sector, NGO/community, etc); institutions (policies, culture, etc), technologies (standards, infrastructure, architecture, applications, etc), and other resources (money, skills, etc) which allows ICTs to have a transformational effect at anything from district to regional to national to international level.

5. Business Model: what are the new ICT-based business models that provide for a transformative developmental impact?  In many ways, the Development 2.0 business model is the organisational equivalent of the higher-level ecosystem; covering organisational strategy, structure, process and value chain from suppliers to clients.  Despite the ‘business’ language, Development 2.0 models can be identified in public, private and NGO sectors (Heeks 2010).

6. Facilitation: what processes and capacities are needed to facilitate emergence and successful implementation of Development 2.0?  This can be answered for both broader ecosystems and narrower business models.  It can encompass a focus on structures, on processes, and on the agency of individuals or groups.

7. Impact: what impact does Development 2.0 have?  This could be answered in terms of any economic, social, political or environmental understanding of development.  So, for example, using lenses of growth, capabilities, inclusion, or sustainability.

The agenda here is still quite general – feel free to suggest inclusions, exclusions, modifications, specifications – but at least it represents a starting point for us to follow.

 

 

[1] Thompson, M. (2008) ICT and development studies: towards development 2.0, Journal of International Development, 20(6), 821-835

[2] Heeks, R.B. (2010) Development 2.0: Transformative ICT-Enabled Development Models and Impacts, Development Informatics Short Paper no.11, Centre for Development Informatics, University of Manchester, UK

[3] Hanna, N. (2011) e-Transformation: Enabling New Development Strategies, Springer, New York

[4] Thompson, M. (2013) Development 2.0 and beyond, ICT4D Seminar Series, Oxford Internet Institute, 27 Feb

[5] Hanna, N. (2014) An E-Transformation Research Agenda, personal communication with author, 26 Mar

ICT4D Research Priorities from the Post-2015 Development Agenda

25 March 2014 5 comments

What should be the future priorities in researching ICT4D?

The post-2015 development agenda will be the single most-important force shaping the future of international development.  In planning our priorities for development informatics (DI) research – the academic study of ICT4D policy and practice – we should therefore pay close attention to the post-2015 agenda.

In previous blog entries, I have discussed: the process by which the post-2015 agenda is being created; its importance; its content; and the way in which it reflects changing trends and priorities in international development.

In this blog, I summarise the findings from a recent working paper: “Future Priorities for Development Informatics Research from the Post-2015 Development Agenda”.  This presents results from a content analysis exercise which compared the content of the post-2015 development agenda against the content of 116 recently-published papers researching ICT4D.

The basic comparison is shown in the figure below.  It provides a measure of research gap by plotting the extent of difference between the post-2015 text and the development informatics papers; aggregated into a set of development issues.  Issues above the line are more highly represented in DI documents than in the post-2015 agenda; issues below the line are less highly represented.  The larger the indicator the greater the over- or under-representation.

DI Research Gaps Chart

Figure 1: Measure of “Research Gap” Between Development Informatics Research and Post-2015 Agenda

 

This chart plus a whole set of other analytical data (see online paper for details) produce the development informatics research priority map shown below.  Laterally, it sorts research issues in terms of their relation to development.  Mainly by type of goals – environmental, economic, social, political, or cross-cutting – but also including mechanisms of development.

DI Research Priorities Map

Figure 2: Map of Post-2015 Development Informatics Research Priorities

 

Vertically, it sorts research issues in terms of gap.  The higher up the diagram a topic appears, the greater the gap between its presence on the post-2015 agenda and its presence in current DI research.  The larger the gap, the greater the need for additional development informatics research on this topic in future.  Put another way – if you are planning what ICT4D-related topic to research in future, there is a logic in starting your search at the top of the figure.

Further details about the topics identified in the research map can be found in the online paper.

Hot, Warm and Cooling Topics on the Post-2015 Development Agenda

12 February 2014 4 comments

The analysis presented in my previous blog entry helped understand the post-2015 development agenda.  But it was static, giving no sense of the dynamics and trends within that agenda.  Those dynamics are important to all development stakeholders: “hot” topics garner funding and attention and political support, and so can gather momentum and produce real-world impact.

So, using the MDGs as the comparison point, what topics are falling down, continuing on, and rising up the international development agenda?

A textual analysis – for details see “From the MDGs to the Post-2015 Agenda: Analysing Changing Development Priorities” – was undertaken comparing core MDG with core post-2015 documentation.  The figure below shows the results of that comparison for 25 development issues (each of which aggregates a number of separate terms).

PTDA Change

Figure 1: Averaged Issue Change in Frequency from MDG to Post-2015 Core Documentation

These can then be grouped into three types of issue and into four categories of change, as summarised in the table below.

MDG to PTDA Change Development Goals Development Mechanisms Development Perspectives
Diminution – MDG 8 with ICTs/Digital

– Manufacturing

– Insecurity

– Traditional Development Finance

– Development Strategy

Continuity – Wellbeing 

– Infrastructure

– Urban Development

– Institutional Development

– MDGs 1-6

– Informatics
Some Expansion – Rural/Agricultural Development

– Services

– Livelihoods

– Growth and Jobs

– Rights and Justice

– New Development Finance

– Technovation inc. Data and Mobile

– Complex Adaptive Systems
Significant Expansion – Open Development

– Inclusive Development

– Migration

– Environment and Sustainability

– Development Projects

– New Stakeholders

Table 1: Summarising Changes in Development Issues from MDGs to Post-2015 Agenda

A blog is not the place for lengthy explanations: if you’d like to understand what each of these issues represents, then refer to the working paper.

Instead, I’ll comment on the bigger picture of change.  The post-2015 agenda represent a richer, more multi-faceted view of development.  This reflects the breadth of consultation behind post-2015; criticisms of what the MDGs missed out; and the ongoing complexification of development.

Other context also matters.  In relative terms, the MDGs were written at a time of stable politics and growing economies.  The post-2015 agenda is being created within a world suffering an ongoing series of economic, environmental and socio-political shocks.

So some of the agenda dynamics reflects real-world change – aid is no longer as important as it was; there has been some decline in war and conflict; services have grown relative to manufacturing; migration and mobile use are rising; the private sector has an ever-larger role in developing countries.  Some of the agenda trajectory reflects a mix of real-world change and the moving political spotlight: growth, jobs, inclusion and inequality are rising because of new evidence and a new economic context, but also because political insecurities have made them more salient.  Climate change and sustainability also fall into this category, though the political impetus to address them remains distributed and volatile.

And some trends seem to fall more into the realm of fads and fashions.  There are long-burn issues that have taken a while to arrive at the centre of development debate: livelihoods, capabilities, rights, justice and systems are all candidates here.  Others are more cyclical – development projects and management, science and technology were central to development debate from the mid-20th century, then faded, and are only just returning.  Indeed, for these and other issues, we might invoke the Gartner hype cycle (see Figure 2).  ICTs, for instance, are much more important to life in 2014 than 1999 but are only just recovering from their over-hyped peak at the turn of the century.  Resilience and other recent arrivals on the development agenda may follow a similar path (see Dave Algoso’s analysis for more on this.)

Gartner Hype Cycle

Figure 2: The Hype Cycle

We can try to reach into the data to find the changing narratives of development.  One – which we can associate with the fastest-rising terms including sustainability, resilience and uncertainty – is that development in 2000 was about moving forwards.  Development in 2015 will be about that, but will also be about not slipping backwards.  With disability, inclusion/exclusion, partnership and stakeholders as other fastest-rising terms, we can also see a changing narrative from “development for many” to “development for all”.

In turn, the events and changing priorities of the 2000s could be seen as a(nother) challenge to the neo-liberal model that has been the dominant development paradigm.  Perhaps we have finally reached a point of inflection for that model in which the weight of its associated externalities give rise to some alternative.  Of course claims of such a point are arguably continuous from Marx onwards, and the MDGs themselves – while not really challenging the neo-liberal model – spoke as much from the human development paradigm as any other.

There is certainly an expressed desire to move from an incremental to a more transformative notion of development: that is a core leitmotif of the High-Level Panel report but it appears throughout the post-2015 discussions.  In practice, the aspiration for transformation sometimes means more of the same but if there is a paradigmatic transition, it is most likely to be to a sustainable development worldview.  How much political traction this will have with Western governments still likely to see themselves as fragile and emerging from recession during 2014 and 2015 remains to be seen.

There is additionally the sense that opposition to neo-liberalism is somewhat divided.  The post-2015 documents echo other development worldviews that could be transformational if they were the centrepiece for the future of development but which currently sit as one ingredient of the mix: inclusive development, rights-based development, perhaps even open development if it were able to deliver a well-grounded and broad narrative.

Returning to a main theme, above all, the post-2015 agenda – like the MDGs – reflects the world in which it is being created.  A world of growing climate change and growing inequality, of increasing global flows of capital and labour, of increasing complexity and connectivity in which a rising number of stakeholders want their voice to be heard and their views taken into account.  So alongside paradigms like sustainable, inclusive and open development will need to be a worldview that accepts development as a complex adaptive system, and seeks ways to manage that emerging reality.

Analysing the Post-2015 Development Agenda

30 January 2014 3 comments

In two earlier posts, I outlined the current process of creating the post-2015 development agenda, and analysed how important it will be to development practice and research.

But what will that agenda be?  The best guide at present appears to be four key documents that emerge from the totality of post-2015 activity as previously summarised:

  • The foundational “Realizing the Future We Want for All” document and its update “A Renewed Global Partnership for Development”: these are the products in 2012 and 2013 respectively of the UN System Task Team; the core of the post-2015 process.
  • As part of that process a High-Level Panel was set up based around the leaders of the UK, Indonesia and Liberia, which produced a report, “A New Global Partnership” in mid-2013.
  • The Open Working Group, and High-Level Political Forum, and Expert Committee associated with Rio+20 and the Sustainable Development Goals are all in mid-process, so the best guide as yet is the outcome of the Rio+20 conference; a UN General Assembly resolution of 2012 entitled, “The Future We Want”.

Textual analysis of these documents was undertaken.  A simple approach to this was the creation of tag clouds: the cloud for the combined post-2015 documentation is in the figure below.

PTDA TagCloud

Tag Cloud for Combined Core Post-2015 Documentation

 A more detailed analysis was then undertaken via word counts within the documentation.  In all, roughly 200 terms were analysed.  The term list was developed via:

a)   selection from the top 500 words counted in the document using Wordle, which also produced the tag cloud; eliminating all non-discriminatory terms (both simple terms like “and”, “the”, “of”, etc, but also those which relate to development but do not provide any particular guide to a development agenda such as “development”, “developing”, “countries”, etc), plus

b)   similar selection from the top 500 words within the MDG documentation (see future posts), and

c)   cross-checking with terms used in a set of other current development reports and journal paper titles.

The frequency of all terms was normed to a mean count per 10,000 words.

All meaningful terms which appeared more than 10 times per 10,000 words (i.e. with a frequency of more than 0.1% of the text) are shown in the table below.

Term

Freq. per

10,000 Words

Term

Freq. per

10,000 Words

Term

Freq. per

10,000 Words

Sustainab*

94.6

Women

19.9

Water

14.8

System*

38.2

Implementation

19.5

(In)equalit*

14.0

Partnership

36.1

Food

18.7

Security

14.0

Environment*

33.9

Education

18.6

Communit*

13.5

Social

31.5

Rights

18.2

Trade

13.3

Economic

31.2

Growth

17.6

Particip*

13.1

Finan*

29.3

Energy

17.5

Business

12.5

Poverty

28.8

State/States

17.5

Local

12.3

Policy/Policies

23.7

Public

17.1

Income

11.7

Health

21.3

Inclusi*

17.0

Transform*

11.7

Technol*

21.2

Accountab*

16.7

Job

11.5

Government

21.1

Process*

16.1

Information

11.0

Cooperation

15.7

Private

10.8

Child

15.5

Agric*

10.6

Institution

15.5

Impact

10.6

Most Frequent Development Terms in Post-2015 Documentation

 Detailed discussion of the dynamics of the post-2015 development agenda will be undertaken in a future post.  Here, I note the following ten conclusions:

  1. The importance of sustainable development as a core model, of course arising particularly because of the presence of the Rio+20 track within the post-2015 process; with some recognition of the role of inclusive development.
  2. Poverty and environment being the two most important individual development issues on the agenda.
  3. Perhaps, a reasonable parity between three of the main domains of development: environmental, social, and economic.  But a question mark over the place for political development: “politic*” scores just 8.3 and so does not appear; but “govern*” would score 31.2.
  4. A strong presence for items related to MDGs 1 to 6: e.g. poverty, health, women, food, education.
  5. A strong recognition of the importance of technology within development.
  6. A strong presence for what one might term the mechanisms or processes of development: the need for partnerships and cooperation and participation, the role of policies, but also of processes and implementation and impact.
  7. Despite moves towards a more multi-stakeholder perspective on development and the presence of business and communities; still a dominant role for the state in its various guises: state, government, public sector.
  8. Some sense of a systems perspective on development.
  9. Maslow’s shade – or at least the importance of basic needs – stands over the agenda given the presence of poverty, health, food, energy, water, security.
  10. The recognised importance of data (just outside the list at 9.2) and information as the foundation for decision-making and action in development.

Readers are encouraged to make their own analysis of the findings presented in the table, and to draw any other big picture conclusions.

%d bloggers like this: