Pragmatism and Development Studies

What role can pragmatism play in development studies research?

Pragmatism is a philosophy that emerged in the late 19^th century in the aftermath of, and as a response to, the destructiveness of the US Civil War.  It has subsequently fractured into a number of different shards but at root it “emphasizes practical consequences and real-world outcomes as the criteria for evaluating the truth or value of beliefs, theories, or actions”[i].

At present, it plays little overt role in development studies research, at least to judge from recent literature searches (using “pragmatism” and “pragmatist” as keywords):

• A title search in eight of the top development studies journals produced six papers discussing “what works” in policy or development approaches, and contrasting this with more dogmatic or ideological approaches.  While their worldview might have been consistent with philosophical pragmatism, these papers made no reference to it or its literature.
• A full-text search of World Development, Development and Change, and the Journal of Development Studies likewise threw up many papers but again referencing the everyday pragmatism of development actors or approaches but not the philosophy of pragmatism.
• A general literature search adding the keyword “development studies” produced hundreds of hits.  The vast majority of items reviewed were unrelated to international development or used the term pragmatism in the lay sense of a practical approach to politics, economics, healthcare, education, etc.  There were just three examples of explicit engagement between development studies and philosophical pragmatism[ii].

What these searches do highlight is the relevance of pragmatism to development studies because of the general orientation of the discipline towards practice.  The specific value of using pragmatism is that it provides a foundational rationale for the following in development studies research:

• Prioritising the practical outcomes of research so that the “development” in development studies does not get lost
• Being sensitive to the specificities of context, and steering away from deterministic and universalistic grand theorising
• Orienting towards participative methods that value and respect the knowledge and experience of local stakeholders
• Undertaking an iterative dialogue and learning process between theory and practice
• Flexibility in use of theory and methodologies and methods

Of course, one could just do all these things in one’s research, but pragmatism provides a coherent basis for them.

Pragmatism has, though, been criticised[iii].  That sensitivity to context means its findings may not readily be transferable between contexts, severely limiting generalisability of research.  Pragmatism has an agenticism that can mean it fails to recognise the deeper structures of power that need to be transformed if inequality, marginalisation and oppression are to be addressed.  And pragmatism tends towards an ethical relativism: its focus on “what works” provides no inherent judgement around “issues of whose problems are attended to, of who wins and who loses from practice”.

Given these three are all points that are addressed by critical realism, we are currently working to understand how pragmatism and critical realism might usefully be combined to inform development studies research.

Follow @CDDManchester

---

[i] ChatGPT (2023) What is pragmatism? Dec 15. https://chat.openai.com/

[ii] The three examples are:

– An excellent overview of the potential role of pragmatism in development studies: Farahani, A. F., & Esfahani, A. H. (2020). Exploring possibilities for a pragmatic orientation in development studies. In The Power of Pragmatism, J. Wills & R. Lake (eds) (pp. 244-264). Manchester University Press.

– A paper focusing on organisational research but which uses international and community development NGOs as its sample: Kelly, L. M., & Cordeiro, M. (2020). Three principles of pragmatism for research on organizational processes. Methodological Innovations, 13(2).

– A series of chapters and papers emerging from a research project on citizenship practices in East Africa, starting with: Holma, K., & Kontinen, T. (2012). Democratic knowledge production as a contribution to objectivity in the evaluation of development NGOs. Forum for Development Studies, 39(1), 83-103 and particularly in Holma, K. & Kontinen, T. (eds) (2020) Practices of Citizenship in East Africa: Perspectives from Philosophical Pragmatism, Taylor & Francis.

[iii] Heeks, R., Ospina, A. V., & Wall, P. J. (2019). Combining pragmatism and critical realism in ICT4D research: an e-resilience case example. In Information and Communication Technologies for Development. Part II, P. Nielsen & H.C. Kimaro (eds), (pp. 14-25). Springer International Publishing.

Image from: Free Stock photos by Vecteezy

Return of the “Third World”?

Are we seeing a return to the old notion of a “third world”?

Originating in the 1950s, the term “third world” was used to refer to those nations not aligned to either the bloc of Western democracies or the Eastern bloc of communist states.  Over time, and particularly since the end of the Warsaw Pact and dissolution of the Soviet Union, the term has fallen from use.

Recent events, though, may point to a revival in two senses.  First, politically.  Compare the two maps below: of first, second and third worlds in the 1970s[1]; and of reaction to Russia’s invasion of Ukraine in 2022[2].  Yes, there are plenty of differences but the neutral countries are almost all third world; and very few third world countries have taken a strongly-supporting or strongly-condemnatory stance.

Second, economically.  Resource procurement and global supply chains are being rethought.  Western democracies are seeking to delink from Russian energy[3], and Russia is turning East to find new sales outlets[4].  Russia and China are collaborating more closely on financial and other systems[5].  Western firms are considering moving supply chains closer to home, into domains that are both more secure and less abusive of human rights[6].  China’s “dual circulation” strategy presages less economic interaction with the West[7].  Overall, “democracies are banding together, as are autocracies”[8].

If there is some greater economic and political coalescence into a Western democratic bloc and an Eastern autocratic bloc, what are the implications for the “third world” of those countries outside those blocs?

Some will benefit as the blocs seek economic collaboration and political alliance.  Mexico, Vietnam, Indonesia and others are already benefitting, for example, from US firms’ search for non-Chinese production bases[9].  China’s Belt & Road Initiative and Western responses such as the US Build Back Better World initiative are competitively channelling infrastructure funding to lower-income countries[10].

Some states may be adept enough to play off the two blocs, squeezing concessions and enabling greater attention to local development goals and interests[11].  But many will come under pressure to pick a side, as seems particularly to be happening with Western pressure on states to turn away from Russia and China[12].  Third world history suggests, if they do this, then such states may then face attempts at destabilisation from the other bloc.

The world is different, more complex and more connected than it was during the era in which “Third World” arose as a concept.  The realities of first and second world bloc formation will likely be less than they might be.  Just as in the 1970s, and as the 2022 diagram above illustrates, many countries outside those blocs may be more aligned with one than the other.  But, nonetheless, echoes of the Third World are sufficiently strong to be taken seriously.

Follow @CDDManchester

[1] https://commons.wikimedia.org/wiki/File:World_map_worlds_first_second_third.gif https://rgs-ibg.onlinelibrary.wiley.com/doi/full/10.1111/tran.12480

[2] https://www.economist.com/graphic-detail/2022/04/04/who-are-russias-supporters

[3] https://www.theguardian.com/world/2022/apr/04/pressure-mounts-germany-embargo-russian-energy-imports

[4] https://www.energypolicy.columbia.edu/research/qa/qa-china-russia-energy-relations-will-new-oil-and-natural-gas-deals-help-russia-weather-economic

[5] https://www.economist.com/leaders/2022/03/12/will-china-offer-russia-financial-help

[6] https://www.economist.com/business/2022/04/02/is-cancel-culture-coming-to-free-trade; https://economistchina.com/wp-content/uploads/North-American-supply-chains-Will-reshoring-actually-happen.pdf

[7] https://www.economist.com/china/2021/03/11/a-confident-china-seeks-to-insulate-itself-from-the-world

[8] https://www.economist.com/finance-and-economics/2022/03/19/globalisation-and-autocracy-are-locked-together-for-how-much-longer

[9] https://www.economist.com/finance-and-economics/2022/01/01/new-research-counts-the-costs-of-the-sino-american-trade-war; https://economistchina.com/wp-content/uploads/North-American-supply-chains-Will-reshoring-actually-happen.pdf

[10] https://www.firstpost.com/world/explained-as-g7-plans-build-back-better-world-heres-how-much-china-has-spent-on-belt-and-road-initiative-9732641.html

[11] https://rgs-ibg.onlinelibrary.wiley.com/doi/full/10.1111/tran.12480

[12] https://www.economist.com/leaders/2021/07/17/bidens-new-china-doctrine; https://www.aspistrategist.org.au/pressure-building-on-india-to-condemn-russian-invasion-of-ukraine/; https://www.business-standard.com/article/international/pakistan-under-western-pressure-to-condemn-russia-s-invasion-in-ukraine-122030700563_1.html

Reviving Critical Modernism in Development Studies

Critical modernism forms a very small, rather dated trickle of ideas within development studies. How could it be updated to serve as a lens for current research?

Critical modernism can be understood as a wide sweep of ideas, particularly encompassing thinkers such as Habermas and Gramsci[1].  But it has only a small explicit footprint within development studies largely triggered by a chapter in Peet & Hartwick’s book Theories of Development, published in 1999[2].  Itself developed from earlier work, this was particularly a response to “post-development” ideas that arose in the 1980s.

Despite subsequent editions of Theories of Development, the core text on critical modernism by Peet & Hartwick remains unchanged, and the specific notion has gained little overt traction in development literature:

– A few works by Giles Mohan and collaborators in the mid-2000s[3].

– A recent paper taking a critical modernist perspective on rights-based development[4].

– Mark Thompson’s paper which included the question of how Development 2.0 would inform the “debate” on critical modernism within development studies[5].

Unfortunately there wasn’t really a debate, but we can revisit the question, to ask if critical modernism is worth rescuing from its development studies obscurity.

As a start, what is critical modernism?

As the name suggests, it is critical; meaning that – drawing from Marxist political economy – it focuses on the structures of power that shape the processes and outcomes of development.  Practically, it seeks to alter distributions of power in order to improve development outcomes.  Methodologically, it listens to subaltern voices: the voices of those who are excluded and marginalised; whose basic needs have yet to be met.  But it differs from structuralist critical theory through two additions.  First, an incorporation of post-structuralism that acknowledges – alongside the power of resources, institutions and structural relations – the power of discourse and ideas: the power of control over systems of knowledge (see diagram below).  Second, an incorporation of analytical lenses other than just class; for example a feminist lens that recognises patriarchal structures of power.

As the name also suggests, it is modernist: meaning that it accepts (albeit “critically”) and is optimistic about Enlightenment values.  Teleologically, this means critical modernism accepts the idea of development, with a purpose of progress and alleviation of material want[6].  Methodologically, this means an adherence to scientific method, to evidence-based conclusions, and to theorisation.  But it differs from simple modernism in two ways.  First, because it critiques modernism; not the substance of modernism but its current form as reflected in late-stage capitalism.  Second, because it recognise multiple modernities, as modernism interacts with multiple different localities and their contexts around the world.

From here, I suggest four developments of critical modernism, perhaps increasingly contentious:

a) Ontological development: it is an easy step to aver that critical modernism is commensurate with the research philosophy of critical realism. Hence that epistemological and methodological implications of critical realism apply when researching from a critical modernist perspective.

b) Conceptual development: listening to subaltern voices and incorporating the voluntarism of populist critiques of development means critical modernism recognises the agency of the marginalised – the ability of social movements to effect change, and the ability of the marginalised to use the tools (ideas, technologies, discourse) of the powerful to empower themselves. Hence a denial of structural determinism; instead arguing that structures of power shape but do not determine development outcomes. This requires a re-conceptualisation of power that incorporates both structural power (e.g. power over) and agentic power (e.g. power to); and which identifies power as deriving not from a monolithic structure but from multiple sources, both global and local.  Network theories of power may be especially relevant here; for example incorporating the connective power and agency that comes from membership of multiple and multi-scalar networks[7].  In practice, this means seeking universals and commonalities to link within a wider-scale network those local networks (movements and institutions) seeking to empower those at the margins.

c) Methodological development: “Critical modernism listens to what people have to say … Critical modernism finds worth in all experiences”[8]. If we are to take this seriously then it must include listening not just to the marginalised but also those within institutions of power. Critical researchers sometimes fail on this score; standing outside such an institution and painting a caricature that does not engage with, or listen to, its members.  This listening is itself universally critical: not unquestioningly believing all that is said by either the powerful or the powerless.

d) Critical development: as noted above, a central tenet of critical modernism is a “blame the player not the game” approach – “Critical modernism focuses on a critique of capitalism as the social form taken by the modern world rather than on modernism” – arguing that the problem is not modernism per se but capitalism as a particular form within modernism.

But the same logic must also be applied to capitalism.  Adding the requirements for rationality and evidence base, one can argue three things.  First, that capitalism – as well as being the driver for inequality and environmental unsustainability – has been the driver for many of the material gains experienced in the global South in the past two decades[9].  Second, that capitalism is not a form but forms[10].  And that the problems lie not with the substance of capitalism, but with particular forms that it has taken; notably the lightly-regulated forms of neoliberal capitalism and emergent digital capitalism.  Capitalism is not “a corrupt form of modernism”[11] but a corruptible form of modernism.  Third, that while socialism – even communism – may be highly effective in enabling the transformation from a largely agrarian society into early-stage industrialisation, alternatives to capitalism have largely failed to deliver sustainable later-stage development gains[12].

Here, we teeter to the very edge of what it means to be critical; well beyond what Peet & Hartwick – with their old-school calls for collective ownership of all means of production and all social institutions – would recognise.  The key dividing line lies between those who think capitalism is the problem, and those who think it will be – in some form – part of the solution.

(Likewise politically.  Critical modernism eschews kneejerk direct democracy in favour of reasoned, deliberative democracy.  But belief in evidence would accept this form of participative democracy only where – in practice – it proves more effective than representative democracy at delivering development.)

To summarise a (revised) critical modernist approach to development studies:

– Critical through central attention to the distributions of power that underlie distributions of development outcomes; and seeking to alter those distributions in favour of the less-powerful.

– A network conceptualisation of power that includes both structure and agency; both power over and power to.

– Critical acceptance of values of modernity including reasoning and democracy, development and progress, science and technology.

– An ontology and epistemology of critical realism

– Methodology based on scientific method and evidence that listens to both the powerless and powerful.

– Perhaps, a focus more on alternative forms of capitalism than alternatives to capitalism.

Follow @CDIManchester

[1] Mumby, D.K. (1997) Modernism, postmodernism, and communication studies: a rereading of an ongoing debate, Communication Theory, 7(1), 1-28

[2] Latest edition: Peet, R. & Hartwick, R. (2015) Theories of Development, 3^rd edn, Guilford Press, New York, NY

[3] E.g. Hickey, S. & Mohan, G. (2004) Relocating participation within a radical politics of development: critical modernism and citizenship, in: Participation – From Tyranny to Transformation?, S. Hickey & G. Mohan (eds), Zed Books, London, 59-74

[4] Langford, M. (2015) Rights, development and critical modernity, Development and Change, 46(4), 777-802

[5] Thompson, M. (2008) ICT and development studies: towards development 2.0, Journal of International Development, 20, 821-835

[6] Reflecting the views of many social movements that want not a rejection of development, but progress, material gains, and which often believe strongly in the power of science and technology (Hickey & Mohan (ibid)).

[7] Bennett, W.L. & Segerberg, A. (2012) The logic of connective action, Information, Communication & Society, 15(5), 739-768

[8] Peet & Hartwick (ibid:313).

[9] Hulme, D. (2016) Should Rich Nations Help the Poor?, Polity Press, Cambridge, UK

[10] Henderson, J. (1996) Globalisation and forms of capitalism, Competition & Change, 1(4), 403-410

[11] Peet & Hartwick (ibid:314).

[12] O’Neil, P.H. (2015) Essentials of Comparative Politics, WW Norton & Company, New York, NY; Kornai, J. (2000) What the change of system from socialism to capitalism does and does not mean, The Journal of Economic Perspectives, 14(1), 27-42

The Death of International Development?

I have a joke – it’s not a very good one – that in ten years’ time new staff joining my development studies department will have to don the equivalent of the Hogwarts sorting hat.  This will allocate them to either Somalia, Chad, DRC or Afghanistan, which by then will be the only developing countries left.

Sadly, this will probably be an exaggeration but there is a sense in which the development industry is succeeding, and thus steadily putting itself out of business.  Take a look at the ‘five good things’ tab by Hans Rosling at: http://www.bbc.co.uk/news/magazine-24835822; or take a look at the changes in the DAC list of aid recipients: the number of low-income countries on the list shrank from 72 in 1997/99 to 54 in 2012/13[1].

So we have an external, objective reality of fewer developing countries over time.  We have some evidence of shrinkage within development discourse: comparing top-10 tag clouds for the MDGs vs. the post-2015 agenda, it is notable that “developing countries” no longer appears.

And we may also have an internal, subjective reality.  There must be better data on this – please comment to supply – but I was struck by an anecdotal report from a colleague who has travelled to a number of African countries in the past few  months.  She found the young Africans she met were unwilling to accept the “developing country” label for their nations, which they saw as stigmatising.  They saw the development industry typically focusing on the negative[2], and missing out on enterprise, innovation, investment, opportunity and other related keyword identities.

In part, this data just reinforces past messages on development becoming a more intra- than inter-country phenomenon e.g. that the majority of the world’s poor live in middle-income countries, and may well continue to do so in coming decades.  So we see a decline of the national, geographic identity of development both externally and internally: perhaps a better title would have been “the death of the developing country”.  But we also see signs – in Rosling’s presentation but more generally in progress on the MDGs – of fewer individuals overall suffering at the wrong end of the socio-economic development continuum.

While much of this is contested and could be reversed, the direction of travel at present seems clear and it will have important implications.

First, for the notion of international development.  One can see this reflected in the changes from the MDGs to the post-2015 agenda, as discussed in a number of prior blog entries.  While mainstream notions of economic development and social development remain; they now sit alongside other views: sustainable development, inclusive development, open development, institutional development.  The development hegemony of the global North is also being challenged by Southern models of development.  In other words we have an increasingly pluralistic notion of development which must bring with it niches and fragmentation in the field.  In turn, these could well bring a lack of focus and lower profile for international development[3].

Second, for development studies departments[4].  Do they shrink themselves down alongside the shrinking of low-income countries?  Do they convert themselves into area studies departments?  Do they convert themselves into more cross-cutting departments that focus on global challenges: poverty, environment, inequality?  Do they embrace the intersections with business schools and expand to encompass the “emerging economy” notion?  Linked to all these possibilities is again the question of identity: how do development studies departments brand themselves externally, and think of themselves internally to match the changing development context?

Follow @CDIManchester

 

[1] I’ll leave aside the discussion about the extent to which the development industry is the cause of the improvements Rosling and the DAC list describe.

[2] Negative external projections of Africa are, of course, a long-term concern – see, e.g., Pratt, C.B. (1980) The reportage and images of Africa in six US news and opinion magazines, Gazette, 26(1), 31-45

[3] For analysis of the loss of focus in development see, e.g., http://www.societies.cam.ac.uk/cisa/documents/Chang_Hamlet_Paper.pdf

[4] For an analysed defence of development studies against dissolution, albeit from 2003, see: http://www.eadi.org/fileadmin/WG_Documents/Reg_WG/lister.pdf

Hot, Warm and Cooling Topics on the Post-2015 Development Agenda

The analysis presented in my previous blog entry helped understand the post-2015 development agenda.  But it was static, giving no sense of the dynamics and trends within that agenda.  Those dynamics are important to all development stakeholders: “hot” topics garner funding and attention and political support, and so can gather momentum and produce real-world impact.

So, using the MDGs as the comparison point, what topics are falling down, continuing on, and rising up the international development agenda?

A textual analysis – for details see “From the MDGs to the Post-2015 Agenda: Analysing Changing Development Priorities” – was undertaken comparing core MDG with core post-2015 documentation.  The figure below shows the results of that comparison for 25 development issues (each of which aggregates a number of separate terms).

Figure 1: Averaged Issue Change in Frequency from MDG to Post-2015 Core Documentation

These can then be grouped into three types of issue and into four categories of change, as summarised in the table below.

MDG to PTDA Change	Development Goals	Development Mechanisms	Development Perspectives
Diminution	– MDG 8 with ICTs/Digital – Manufacturing – Insecurity	– Traditional Development Finance – Development Strategy
Continuity	– Wellbeing – Infrastructure – Urban Development – Institutional Development – MDGs 1-6	– Informatics
Some Expansion	– Rural/Agricultural Development – Services – Livelihoods – Growth and Jobs – Rights and Justice	– New Development Finance – Technovation inc. Data and Mobile	– Complex Adaptive Systems
Significant Expansion	– Open Development – Inclusive Development – Migration – Environment and Sustainability	– Development Projects – New Stakeholders

Table 1: Summarising Changes in Development Issues from MDGs to Post-2015 Agenda

A blog is not the place for lengthy explanations: if you’d like to understand what each of these issues represents, then refer to the working paper.

Instead, I’ll comment on the bigger picture of change.  The post-2015 agenda represent a richer, more multi-faceted view of development.  This reflects the breadth of consultation behind post-2015; criticisms of what the MDGs missed out; and the ongoing complexification of development.

Other context also matters.  In relative terms, the MDGs were written at a time of stable politics and growing economies.  The post-2015 agenda is being created within a world suffering an ongoing series of economic, environmental and socio-political shocks.

So some of the agenda dynamics reflects real-world change – aid is no longer as important as it was; there has been some decline in war and conflict; services have grown relative to manufacturing; migration and mobile use are rising; the private sector has an ever-larger role in developing countries.  Some of the agenda trajectory reflects a mix of real-world change and the moving political spotlight: growth, jobs, inclusion and inequality are rising because of new evidence and a new economic context, but also because political insecurities have made them more salient.  Climate change and sustainability also fall into this category, though the political impetus to address them remains distributed and volatile.

And some trends seem to fall more into the realm of fads and fashions.  There are long-burn issues that have taken a while to arrive at the centre of development debate: livelihoods, capabilities, rights, justice and systems are all candidates here.  Others are more cyclical – development projects and management, science and technology were central to development debate from the mid-20^th century, then faded, and are only just returning.  Indeed, for these and other issues, we might invoke the Gartner hype cycle (see Figure 2).  ICTs, for instance, are much more important to life in 2014 than 1999 but are only just recovering from their over-hyped peak at the turn of the century.  Resilience and other recent arrivals on the development agenda may follow a similar path (see Dave Algoso’s analysis for more on this.)

Figure 2: The Hype Cycle

We can try to reach into the data to find the changing narratives of development.  One – which we can associate with the fastest-rising terms including sustainability, resilience and uncertainty – is that development in 2000 was about moving forwards.  Development in 2015 will be about that, but will also be about not slipping backwards.  With disability, inclusion/exclusion, partnership and stakeholders as other fastest-rising terms, we can also see a changing narrative from “development for many” to “development for all”.

In turn, the events and changing priorities of the 2000s could be seen as a(nother) challenge to the neo-liberal model that has been the dominant development paradigm.  Perhaps we have finally reached a point of inflection for that model in which the weight of its associated externalities give rise to some alternative.  Of course claims of such a point are arguably continuous from Marx onwards, and the MDGs themselves – while not really challenging the neo-liberal model – spoke as much from the human development paradigm as any other.

There is certainly an expressed desire to move from an incremental to a more transformative notion of development: that is a core leitmotif of the High-Level Panel report but it appears throughout the post-2015 discussions.  In practice, the aspiration for transformation sometimes means more of the same but if there is a paradigmatic transition, it is most likely to be to a sustainable development worldview.  How much political traction this will have with Western governments still likely to see themselves as fragile and emerging from recession during 2014 and 2015 remains to be seen.

There is additionally the sense that opposition to neo-liberalism is somewhat divided.  The post-2015 documents echo other development worldviews that could be transformational if they were the centrepiece for the future of development but which currently sit as one ingredient of the mix: inclusive development, rights-based development, perhaps even open development if it were able to deliver a well-grounded and broad narrative.

Returning to a main theme, above all, the post-2015 agenda – like the MDGs – reflects the world in which it is being created.  A world of growing climate change and growing inequality, of increasing global flows of capital and labour, of increasing complexity and connectivity in which a rising number of stakeholders want their voice to be heard and their views taken into account.  So alongside paradigms like sustainable, inclusive and open development will need to be a worldview that accepts development as a complex adaptive system, and seeks ways to manage that emerging reality.

Follow @CDIManchester

Analysing the Post-2015 Development Agenda

In two earlier posts, I outlined the current process of creating the post-2015 development agenda, and analysed how important it will be to development practice and research.

But what will that agenda be?  The best guide at present appears to be four key documents that emerge from the totality of post-2015 activity as previously summarised:

• The foundational “Realizing the Future We Want for All” document and its update “A Renewed Global Partnership for Development”: these are the products in 2012 and 2013 respectively of the UN System Task Team; the core of the post-2015 process.
• As part of that process a High-Level Panel was set up based around the leaders of the UK, Indonesia and Liberia, which produced a report, “A New Global Partnership” in mid-2013.
• The Open Working Group, and High-Level Political Forum, and Expert Committee associated with Rio+20 and the Sustainable Development Goals are all in mid-process, so the best guide as yet is the outcome of the Rio+20 conference; a UN General Assembly resolution of 2012 entitled, “The Future We Want”.

Textual analysis of these documents was undertaken.  A simple approach to this was the creation of tag clouds: the cloud for the combined post-2015 documentation is in the figure below.

Tag Cloud for Combined Core Post-2015 Documentation

 A more detailed analysis was then undertaken via word counts within the documentation.  In all, roughly 200 terms were analysed.  The term list was developed via:

a)   selection from the top 500 words counted in the document using Wordle, which also produced the tag cloud; eliminating all non-discriminatory terms (both simple terms like “and”, “the”, “of”, etc, but also those which relate to development but do not provide any particular guide to a development agenda such as “development”, “developing”, “countries”, etc), plus

b)   similar selection from the top 500 words within the MDG documentation (see future posts), and

c)   cross-checking with terms used in a set of other current development reports and journal paper titles.

The frequency of all terms was normed to a mean count per 10,000 words.

All meaningful terms which appeared more than 10 times per 10,000 words (i.e. with a frequency of more than 0.1% of the text) are shown in the table below.

Term	Freq. per 10,000 Words	Term	Freq. per 10,000 Words	Term	Freq. per 10,000 Words
Sustainab*	94.6	Women	19.9	Water	14.8
System*	38.2	Implementation	19.5	(In)equalit*	14.0
Partnership	36.1	Food	18.7	Security	14.0
Environment*	33.9	Education	18.6	Communit*	13.5
Social	31.5	Rights	18.2	Trade	13.3
Economic	31.2	Growth	17.6	Particip*	13.1
Finan*	29.3	Energy	17.5	Business	12.5
Poverty	28.8	State/States	17.5	Local	12.3
Policy/Policies	23.7	Public	17.1	Income	11.7
Health	21.3	Inclusi*	17.0	Transform*	11.7
Technol*	21.2	Accountab*	16.7	Job	11.5
Government	21.1	Process*	16.1	Information	11.0
		Cooperation	15.7	Private	10.8
		Child	15.5	Agric*	10.6
		Institution	15.5	Impact	10.6

Most Frequent Development Terms in Post-2015 Documentation

 Detailed discussion of the dynamics of the post-2015 development agenda will be undertaken in a future post.  Here, I note the following ten conclusions:

1. The importance of sustainable development as a core model, of course arising particularly because of the presence of the Rio+20 track within the post-2015 process; with some recognition of the role of inclusive development.
2. Poverty and environment being the two most important individual development issues on the agenda.
3. Perhaps, a reasonable parity between three of the main domains of development: environmental, social, and economic.  But a question mark over the place for political development: “politic*” scores just 8.3 and so does not appear; but “govern*” would score 31.2.
4. A strong presence for items related to MDGs 1 to 6: e.g. poverty, health, women, food, education.
5. A strong recognition of the importance of technology within development.
6. A strong presence for what one might term the mechanisms or processes of development: the need for partnerships and cooperation and participation, the role of policies, but also of processes and implementation and impact.
7. Despite moves towards a more multi-stakeholder perspective on development and the presence of business and communities; still a dominant role for the state in its various guises: state, government, public sector.
8. Some sense of a systems perspective on development.
9. Maslow’s shade – or at least the importance of basic needs – stands over the agenda given the presence of poverty, health, food, energy, water, security.
10. The recognised importance of data (just outside the list at 9.2) and information as the foundation for decision-making and action in development.

Readers are encouraged to make their own analysis of the findings presented in the table, and to draw any other big picture conclusions.

Follow @CDIManchester

How Important Will The Post-2015 Agenda Be For Development Research And Practice?

In an earlier post, I outlined the current state of the post-2015 development agenda (PTDA) process.  Later posts will look at the content of that agenda and its implications for development – particularly development informatics – research.

Before getting to that, though, it is appropriate to ask a couple of foundational assumption-checking questions.

Question number 1: “How important will the PTDA be to international development?”.  If it is just going to end up gathering dust on a shelf, or if it is just  a side-show, then there is little point using it to shape our research priorities.  We will not know the answer to that question until something like 2020 at the earliest but we have two current guides.

The first is how important the post-2015 agenda is currently perceived to be.  One set of evidence is the extent of participation in the consultation process.  There have been nearly 100 national, six regional and eleven thematic consultations, with each of these typically involving many hundreds of organisational participants[1] plus thousands of online contributions[2].  It is hard to benchmark this against other activities but it must represent one of the most substantial exercises in global consultation.  Other evidence comes from polling perceptions: for example, of more than 100 civil society organisations surveyed in 27 developing countries, 87% wanted a post-2015 development framework[3].

A second guide is historical: investigating how important the Millennium Development Goals (MDGs) have been to international development, given they are by far the closest historical phenomenon to the PTDA.  There is a generalised assumption about the MDGs’ importance: “the MDGs … have an incontestable strength”[4]; “the Millennium Development Goals … have unified, galvanized, and expanded efforts to help the world’s poorest people”[5].  However, in the complex field of influences that exists within international development, attribution is problematic: “the direct development impact of the MDGs is difficult to determine”[6].

Those who have sought to study this come up with differentiated conclusions depending on the area of influence investigated.  For example:

• Debate/Discourse: “There is widespread agreement that the MDGs have placed broad-based poverty reduction at the center of the development agenda at least in international discussions and policy discourse”[7]; “There is plenty of evidence of the influence of the MDGs on policy discourse, if this is measured by mention of the goals or their presence in donor policy documents, PRSPs and developing country government goals”[8].
• Aid Flows: “The MDGs have mobilized government and business leaders to donate tens of billions of dollars”[9]; “We argue that the MDGs may have played a role in increasing aid”[10].
• Policy: “For better or worse, the Millennium Development Goals (MDGs) have constituted the longest standing paradigm that has ever emerged in development thinking. The goals have been an organising framework for international aid over the last ten years. At the core of countless policy documents, plans and announcements”[11]; “policy statements of major bilateral donors align with the MDG priorities only partially and in varying ways … there is a considerable adoption of MDG priority areas, however there is equal or higher adoption of priorities not in the MDGs”[12].
• Outcomes: “the most powerful impact of the MDGs appears to have been on aid flows, but the impact of that aid on outcomes is difficult to assess and plausibly muted”[13]; “In some areas, such as vaccination or primary education enrolment in sub-Saharan Africa, the links between the MDGs, the mobilisation and focusing of additional aid, and subsequent impacts seem convincingly close. But in others, the links seem less plausible”[14].
• Practice: “The research shows that in the organisations studied [small number of faith-based NGOs], the extent of influence of the MDGs has been minimal upon development activities in a direct sense, although some indirect influence due to donor funding requirements has been reported”[15].

Drawing on these sources and others[16], a subjective summary assessment of MDG impact can be drawn up as shown in Figure 1.

Figure 1: Relative Impact of MDGs on Differing Aspects of International Development

Question number 2: “How important will the PTDA be to development research agendas and funding?”.  Again, we can look at current evidence about PTDA activity, plus also historical evidence relating to the MDGs.  At the time of writing, many of the major development research institutes – those with a majority focus on international development and lying at the top of the Think Tanks and Civil Societies Program rankings[17] – have post-2015 initiatives underway.  This seems much less true of US-based institutes, probably reflecting the lower levels of US engagement with the MDGs and the post-2015 agenda[18]:

• Center for Global Development: has a number of blog posts on post-2015 and some publications on the MDGs which include thoughts on post-2015, but no main topics or initiatives.
• Kennedy School Center for International Development: has no apparent research programmes or specific activities related to the post-2015 agenda.
• International Food Policy Research Institute: has its own 2020 agenda but no major post-2015 research activity.

The picture is very different for development research institutes outside the US.  Listing these in descending TTCSP rank order:

• Overseas Development Institute (UK): runs www.post2015.org and has a major programme on “The MDGs to 2015 and Beyond”.
• United Nations University World Institute for Development Economics Research (Finland): does not have a specific initiative but is positioning its future research in relation to the post-2015 agenda.
• German Development Institute (Germany): has the “What Will Be After 2015?” research project on the post-2015 agenda.
• North-South Institute (Canada): has a “Post-2015” initiative of research, briefings, events, etc.
• Institute for Development Studies (UK): has a topic focus on “Millennium Development Goals (MDGs) and Post 2015 Agenda”.

Alongside this snapshot of current activity, we can look at historical impact of the MDGs on research agendas and funding.  Data on the output side is not particularly clear.  A review was undertaken of articles in the three top development studies journals – World Development, Development and Change, and Journal of Development Studies – published during 2008-2013.  This suggested that 1-2% of articles had a specific engagement with the MDGs (mentioned in the title or abstract), and 10-15% mentioned the MDGs somewhere in the main text.  In the absence of other benchmarks, not much can be concluded from this data.

A stronger sense of the importance of the MDGs comes from the input side; from analysis of funder research strategies.  For this activity, analysis was undertaken of the research strategies of three key development research funders – Canada’s International Development Research Centre (IDRC), the Swedish International Development Cooperation Agency (SIDA), and the UK Department for International Development (DFID) – during the period 2002-2012.  This suggested a continuum of MDG influence as summarised in Figure 2:

• IDRC: research strategy documents have just one or two passing references to the MDGs, and the MDGs do not frame research strategy.  For example: “Although not explicit nor an underpinning of IDRC’s health programming, there is an implicit interest in the health-related Millennium Development Goals”[19].
• SIDA: the MDGs are one among a number of components that have shaped research strategy.  For example, a core overview[20] lists three foci for research: matters of relevance to low-income countries; research issues arising from international commitments as defined by the MDGs and UN conventions; and cooperative arrangements that identify new research of relevance to developing countries.
• DFID: “The current effort is … using the Millennium Development Goals as the main framework for determining research strategies and priorities”[21].  “All DFID’s efforts are directed towards achieving the targets set by the world community in the Millennium Development Goals by 2015. They are the basis for choosing research topics”[22].  “The purpose of DFID’s research is to make faster progress in fighting poverty and achieving the MDGs”[23].

Figure 2: MDG Influence on Development Research Strategies

Taking together all of the evidence, it seems reasonable to conclude that – whatever its absolute strength and with acknowledgement to local variations – the post-2015 development agenda will be the single most important force shaping the future of development and of development research.  It is certainly of sufficient importance to take very seriously in the planning of future development-related research agendas.  If our own future research is in synch with post-2015, at the least we can use that to boost the credibility and perceived relevance of our research; at the most, we will gain greater funding and a wider audience for our research.  Future posts will explore this further.

Follow @CDIManchester

---

[1] e.g. TWWW (2013a) Global Thematic Consultation on Governance and the Post-2015 Development Framework,The World We Want http://www.worldwewant2015.org/governance/finalreport; and TWWW (2013b) Health in the Post-2015 Agenda, The World We Want http://www.worldwewant2015.org/health

[2] See e.g. http://trends.worldwewant2015.org/discover/#mode=type

[3] Pollard, A., Sumner, A., Polato-Lopes, M. & de Mauroy, A. (2011) 100 Voices, CAFOD, London http://www.cafod.org.uk/Media/Files/Resources/Policy/100-Voices

[4] Prammer, E. & Martinuzzi, A. (2013) The Millennium Development Goals (MDGs) and the Post-2015 Debate, Case Study no.13, European Sustainable Development Network, Vienna

[5] McArthur, J. (2013) Own the goals: what the millennium development goals have accomplished, Foreign Affairs, March/April http://www.brookings.edu/research/articles/2013/02/21-millennium-dev-goals-mcarthur

[6] Higgins, K. (2013) Reflecting on the MDGs and Making Sense of the Post-2015 Development Agenda, The North-South Institute, Ottawa, ON

[7] Kenny, C. & Sumner, A. (2011) More Money or More Development: What Have the MDGs Achieved?, Working Paper 278, Center for Global Development, Washington, DC http://international.cgdev.org/files/1425806_file_Kenny_Sumner_MDGs_FINAL.pdf

[8] Lockwood, M. (2012) What have the MDGs achieved?  We don’t really know, From Poverty to Power, 31 Aug http://www.oxfamblogs.org/fp2p/?p=11498

[9] McArthur 2013

[10] Kenny & Sumner 2011

[11] Pollard et al. 2011

[12] Kenny & Sumner 2011

[13] Kenny & Sumner 2011

[14] Lockwood 2012

[15] Dore, M. (2011) Keeping Faith with the MDGs, MSc Dissertation, University of Edinburgh

[16] e.g. Gore, C. (2009) The Global Development Cycle, MDGs and the Future of Poverty Reduction, European Association of Development Research and Training Institutes, Bonn http://www.eadi.org/fileadmin/MDG_2015_Publications/Gore_PAPER.pdf; and Manning, R. (2010) The impact and design of the MDGs: some reflections, IDS Bulletin, 41(1), 7-14 http://www.humanitarianforum.org/data/files/impactanddesignofmdg.pdf

[17] McGann, J.G. (2013) 2012 Global Go To Think Tanks Report and Policy Advice, Think Tanks and Civil Societies Program, University of Pennsylvania, Philadelphia, PA http://gotothinktank.com/dev1/wp-content/uploads/2013/07/2012_Global_Go_To_Think_Tank_Report_-_FINAL-1.28.13.pdf

[18] e.g. Hulme, D. (2009) The Millennium Development Goals (MDGs): A Short History of the World’s Biggest Promise, BWPI Working Paper 100, Brooks World Poverty Institute, University of Manchester, UK http://www.bwpi.manchester.ac.uk/resources/Working-Papers/bwpi-wp-10009.pdf

[19] IDRC (2009) Innovating for Development Strategic Framework 2010-2015, IDRC, Ottawa http://www.idrc.ca/EN/AboutUs/WhatWeDo/Documents/12614275681Strategic_framework_2010-2015.pdf

[20] Regeringskansliet (2010) Research for Development, Regeringskansliet, Stockholm http://www.government.se/content/1/c6/14/60/03/eab96d0b.pdf

[21] Surr, M., Barnett, A., Duncan, A., Speight, M., Bradley, D., Rew, A. & Toye, J. (2002) Research for Poverty Reduction: DFID Research Policy Paper, DFID, London http://www.idee.ceu.es/Portals/0/Actividades/Doc_Reduccion_Pobreza.pdf

[22] DFID (2004) DFID Research Funding Framework 2005-2007, DFID, London http://globalgrn.files.wordpress.com/2009/08/dvidresearch-framework-2005.pdf

[23] DFID (2008) Research Strategy 2008-2013, DFID, London https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67757/research-strategy-08.pdf

The Post-2015 Development Agenda

This is the first of a number of related blog entries that will look at the post-2015 development agenda and its implications.  This entry describes the process of setting that agenda.

In theory, the origins of the post-2015 process could be traced back many years to the setting of the Millennium Development Goal deadline.  It was obvious then that there would be a post-MDG world from 2015.  However, it seems more appropriate to date the timeline (see Figure 1 below, and more detailed timeline in Table 1 at the end) from September 2011, with the formation of the UN System Task Team: the body charged with overseeing the post-2015 process.

Figure 1: Post-2015 Process Outline Timeline

The MDGs were an integration in 2001 of two rather separate processes: the OECD Development Assistance Committee’s work on International Development Goals, and the UN’s work to develop the Millennium Declaration[1].  This added to the time and effort required to produce the MDGs, yet the same is happening again with the post-2015 process, as summarised in Figure 2 below (adapted from an original by Claire Hickson[2]).

Figure 2: Post-2015 Development Agenda and Sustainable Development Goals Process Map

The timeline shown is therefore a single representation of multiple strands.  The post-2015 development agenda process is relatively well-advanced.  Following the UN System Task Team’s formation, a series of thematic and national consultations on the agenda have already been conducted, with two key reports produced in 2012 (“Realizing the Future We Want for All”) and 2013 (“A Renewed Global Partnership for Development”).  A High-Level Panel was set up by UN Secretary-General Ban Ki-moon.  Chaired by the Presidents of Indonesia and Liberia and the UK Prime Minister and involving 24 other “eminent persons”, this produced its report mid-way through 2013.  These documents were placed before the UN General Assembly when its 68^th session began in September 2013; a session which included special meetings and events on the MDGs and after.

At the time of writing, the sustainable development goals (SDGs) process was not quite so well developed.  Emerging from the 2012 UN Conference on Sustainable Development (“Rio+20”) and its General Assembly resolution in July 2012, this led to formation of a UN Open Working Group.  The Group has been supported by a UN System Technical Support Team, which provides a link to the post-2015 activity since it works under the UN System Task Team.  It has also been supported by an “Intergovernmental Committee of Experts on Sustainable Development Financing” and a “High-Level Political Forum” that provides political momentum for the process.  The Open Working Group has a series of eight sessions being run during 2013-2014, and structured along thematic lines.  This will report towards the end of 2014.

At that point – during 2015 – an integration of the two processes and political negotiation of the final post-2015 agenda should occur, leading to a new post-MDG framework to run from the start of 2016.  It is worth just asking whether such a framework might not emerge.  Present signs are that this would be extremely unlikely: process, timeline and structures are all in place; and significant political capital – plus other resources – has already been invested.  It would take something huge and unexpected to derail the process.  We can therefore work on the assumption that there will be a post-2015 agenda.

Table 1: The Post-2015 Process Schedule

Sourced largely from Hickson (2013)

Date	Activity
Sep 2010	UN MDG Summit
Sep 2011	UN System Task Team established to lead post-2015 process
May 2012-Apr 2013	Post-2015 thematic global consultations
Jun 2012	Rio+20 summit; working group on Sustainable Development Goals (SDGs) set up
Jun 2012	UN System Task Team “Realizing the Future We Want for All” report
Jun 2012	National post-2015 consultations begin
Jul 2012	Rio+20 “The Future We Want” resolution to UN General Assembly
Aug 2012	High-Level Panel (HLP) set up by Ban Ki-moon
Sep 2012	HLP convened
Nov 2012	HLP first substantive meeting (London)
Jan 2013	SDG Open Working Group created
Feb 2013	HLP second meeting (Monrovia)
Feb 2013	EU post-2015 communication “A Decent Life for All”
Mar 2013	UN System Task Team “A Renewed Global Partnership for Development” report
Mar 2013	HLP third meeting (Bali)
Mar 2013-Feb 2014	Eight sessions of SDG Open Working Group
May 2013	Draft SDG report
May 2013	HLP “A New Global Partnership” report
Jul 2013	Progress report of SDG Open Working Group to UN General Assembly
Sep/Oct 2013	New UN General Assembly session and MDG Review Summit
Sep 2013	First session of High-Level Political Forum on sustainable development
Sep 2014	SDG Open Working Group to report to UN General Assembly
Jan 2015	MDG deadline
Jan-Dec 2015	Intergovernmental negotiations via UN General Assembly on Post-2015 Agenda
Sep 2015	High-Level Political Forum Meeting
c.Jul-Sep 2015	UN General Assembly Post-2015/MDG Review Summit
Jan 2016	New Post-2015 framework in place

Follow @CDIManchester

---

[1] Hulme, D. (2009) The Millennium Development Goals (MDGs): A Short History of the World’s Biggest Promise, BWPI Working Paper 100, Brooks World Poverty Institute, University of Manchester, UK http://www.bwpi.manchester.ac.uk/resources/Working-Papers/bwpi-wp-10009.pdf

[2] Hickson, C. (2013) Post-2015 development goals process and timeline, Trio Policy, 11 Jul http://www.triopolicy.com/post-2015-development-goals-process-and-timeline/

Development Studies Journal Ranking Table

The following represents a first attempt at a “league table” for development studies journals.

Rank	Journal	Citation Score
1	World Development	6.04
2	Journal of Development Studies	4.90
3	Oxford Development Studies	4.06
4	Development Policy Review	3.20
5	Studies in Comparative International Development	2.40
6	Sustainable Development	2.39
7	European Journal of Development Research	1.90
8	Development and Change	1.89
9	Information Technology for Development	1.58
10	Information Technologies and International Development	1.55
11	Journal of International Development	1.46
12	Development	1.33
13	Third World Quarterly	1.30
14	Public Administration and Development	1.21
15	Development in Practice	1.03
16	Progress in Development Studies	0.88
17	Electronic Journal of Information Systems in Developing Countries	0.81
18	African Development Review	0.79
19	Gender and Development	0.58
20	Enterprise Development and Microfinance	0.45
21	Canadian Journal of Development Studies	0.45
22	IDS Bulletin	0.40
23	Information Development	0.37
24	Forum for Development Studies	0.17
25	Journal of Third World Studies	0.11
	Comparator Journals
	Journal of Development Economics	10.90
	Human-Computer Interaction	4.06
	Environment and Planning D	3.42
	Information Systems Journal	2.89
	The Information Society	1.64
	Mountain Research and Development	0.91

Basis

– Selection was on the basis of development studies journals that appear in various other tables or lists.  However, development economics journals (inc. Economic Development and Cultural Change, Journal of Development Economics, Review of Development Economics, and The Developing Economies) were not included.  If you have suggestions for additions (or deletions), then let me know.

– Citation score is calculated by taking papers published in each journal in 2008 and identifying how many times each paper is cited in Google Scholar.  The average number of cites per paper was then divided by the average number of years since publication.  Very roughly, then, the score equates to average number of GS citations per paper per year.

– All papers published in 2008 were used if less than 20 were published; a sample of at least 20 building outwards from the mid-year issues was used if more than 20 were published.

– One anomalous paper, with over 10 times the citations of any other (a pattern not seen in any other journal), was omitted from African Development Review.  Had this been included, ADR would place seventh.

– This exercise will be repeated and expanded in future years.  What is presented here should only be seen as a first, fairly rough-and-ready set of figures.  The original data used for the calculations can be found here.

Notes

– The raw figures shown here should not be compared with the impact factor scores under Planning and Development provided in ISI’s Journal Citation Reports.  The rankings can be compared.

– Different disciplines have different citation habits and norms.  Specifically, if economists cite more highly, then those development studies journals that include a greater proportion of development economics papers may gain a greater overall citation score.

– Conversely – and requiring further investigation – in compiling the figures, I got some sense that papers in special issues tend to receive fewer citations.  Journals that have a lot of special issues may receive a lower overall citation score.

Reflections

– These average figures provide no guidance on whether your individual paper would be cited more highly if published in one journal or another.  However, the rankings could be used to provide guidance or evidence on the general impact of a selected journal.  (Of course recognising that overall impact is about more than just citations.)

– The figures suggest that, beyond the obvious top two of JDS and World Development, there may be some mismatch between previous subjective ratings and actual impact.  For example, Oxford Development Studies and Development Policy Review rank 3^rd and 4^th here, yet are unrated by most other journal rating schemes.

– There is a moderate mismatch with the ISI JCR 2008 impact factor ranking.  Most notably, four of the top ten journals here do not appear at all in the ISI list including the two top-cited ICT-for-development journals.

Other Data

– The table below gives details of other ranking and rating data on development studies and some development economics journals.

High->Low	Aston 2008 (4->0)	CNRS 2008 (1*->4)	Ideas 2010 (/731)	SJR 2010 (/118)	WoK 2010 (/43)	ABDC 2010 (A*->C)	ABS 2010 (4->1)	SoM 2010 (4->1)	Heeks 2010 (/25)
African Development Review				65	43			2	18
Canadian Journal of Development Studies				78	42				21
Development			666	28					12
Development and Change	2	2		15	19	B	2		8
Development in Practice				32					15
Development Policy Review			270	10	8				4
Economic Development and Cultural Change			117		24	A	3	4
Electronic Journal of Information Systems in Developing Countries									17
Enterprise Development and Microfinance									20
European Journal of Development Research			438	48					7
Forum for Development Studies									24
Gender and Development				73					19
IDS Bulletin				70	37				22
Information Development									23
Information Technologies and International Development									10
Information Technology for Development									9
Journal of Development Economics			43		36	A*	3	4
Journal of Development Studies	2	2	152	2	26	A	3	4	2
Journal of International Development	1	3	292	22		B	1	1	11
Journal of Third World Studies				86					25
Oxford Development Studies			192	58				1	3
Progress in Development Studies				30					16
Public Administration and Development				62	39	A	2	2	14
Review of Development Economics			129	26	32			1
Studies in Comparative International Development				23	31	A			5
Sustainable Development				9	11				6
The Developing Economies			474		35	B
Third World Quarterly		2		29	30	A	2		13
World Development	3	1	134		9	A	3	3	1
High->Low	Aston 2008 (4->0)	CNRS 2008 (1*->4)	Ideas 2010 (/731)	SJR 2010 (/118)	WoK 2010 (/43)	ABDC 2010 (A*->C)	ABS 2010 (4->1)	SoM 2010 (4->1)	Heeks 2010 (/25)

Key
 
– ABS – UK Association of Business Schools: http://www.the-abs.org.uk/?id=257

– Ideas – citation data from RePEc project of paper downloads: http://ideas.repec.org/top/top.journals.simple.html (economics and finance research)

– SJR – Scopus-based citation ranking: http://www.scimagojr.com/journalrank.php?category=3303&area=0&year=2008&country=&order=sjr&min=0&min_type=cd (development journals)

– SoM – Cranfield School of Management: https://www.som.cranfield.ac.uk/som/dinamic-content/media/SOM%20Journal%20Rankings%202010%20-%20alphabetical.pdf

– WoK – 2008 impact factor in ISI Journal Citation Reports under Planning and Development

– All other data from Harzing’s Journal Quality List: http://www.harzing.com/jql.htm