Home > Researching ICT4D > ICT4D Research: How Should I Publish?

ICT4D Research: How Should I Publish?

In what form should you publish your ICT4D research for maximum impact?  A book; a book chapter; a journal article – if so what kind of journal; a conference paper?

You are welcome to comment your own evidence, but I’m going to answer that question through analysis of the publications of my favourite author: me.  Why me?  Because almost everything I’ve published is in the ICT4D field; there are a large number of items, over a long period, and in many different formats; and careful selection of my name at birth ensures very few false positives in citation searches.  But most importantly, I have access to the list of all items I’ve ever published, and so can include those that have never been cited.

The method: dividing out all my publications into different categories.  Then, for each one, identifying how many citations are shown on Google Scholar; and then working out the average number of citations per publication for each category.  The results are as follows:

Publication Type Mean Citations per Item Median Citations per Item % Items Never Cited n
Single Authored Book   96   48   0% 4
Refereed Journal Article (WoK-listed journal)   54   30   6% 17
Working Paper (available online)   27   8   22% 46
Refereed Journal Article (non-WoK-listed journal)   9   5   20% 15
Report / Handbook (available online)   6   3   29% 14
Book Chapter   9   0   55% 40
Magazine / Professional Journal Article   1.4   0   69% 80
Conference / Seminar Presentation   0.6   0   92% 98

 Notes on the data:

- I’ve excluded web sites (like http://www.egov4dev.org) and this blog.  If I did analyse them, then putting all such items together, the vast majority (90%+) of individual pages and blog entries remain uncited, but altogether there are 113 citations, 82 of those from the egov4dev web site.  So a successful web site can be as highly-cited as a book.

- I’ve also edited two books, but not included them in the table for two reasons.  First, because – in a strict sense – I don’t think one should cite an edited book; one should cite from the individually-authored chapters within that book.  Second, because the two had such wildly-divergent fates, “average” makes no real sense: one is cited 9 times; one is cited 363 times.  (And, yes, I have sample checked that those citations refer just to the overall book and not to any individual chapter.)

- That pattern of very skewed data between items – some scoring well, others totally flopping – is found across virtually all types of output.  That’s why the table includes both mean and median scores.

- WoK is the ISI Web of Knowledge; inclusion of a journal in this is a rough quality benchmark.  I could have redone the citation figures using WoK instead of Google Scholar but I’ve looked at the data and the overall pattern is the same; just that citation numbers are around one-quarter of those for GS.

Conclusions From The Data:

- This is data from just one person: caveat lector.

- Trying to work out what differentiates a citation hit from a citation flop, one might think research quality – see separate entry on that – was part of the story (not, of course, that my publications would ever be anything but very high quality!).  But I don’t think it is.  To me, timing looks a big factor.  Write something half-decent in a sub-area that subsequently grows, and you’ll get a lot of cites (and, arguably, have some degree of influence in that sub-area).  Write something great in a mature field and you’ll get many fewer citations (of course in part because you’re then fighting for attention with far more “competing” publications).  The slightly depressing conclusion: if it’s citations you’re after, keep up with research fads and fashions.

- And then what about the answer to the initially-posed question on type of publication to choose?  I’ve arranged the table in what I would say is decreasing overall impact order, and I’ll comment on each category

- – Authored book: still the best way to have an impact, and the data here probably understates since it includes two quasi-turkeys from the 1980s when I had 100% no clue what I was doing (now I just have 50% no clue).  The two published more recently have an average 185 cites each and rising.  If we compared that to the median journal figure, then you would ask yourself – is the effort of writing a book more or less than that for writing six refereed articles in decent journals?

- – Pukka journal article: still the most obvious way to build a citation profile.

- – Edited book: editing a book is easier than writing one.  Editing an impactful book is probably not.  I’d suggest you need to write or co-write a number of start and end chapters yourself; and impose such a degree of consistency of argument, style and structure that you are often almost rewriting other contributions from scratch.  Plus see comment on book chapters below.

- – Working paper: can be highly effective but a lot probably depends on the profile and dissemination of the series in which you publish.  You get zero academic brownie points, but you can maybe modify into a journal article.

- – Not-so-pukka journal article: can be worth doing, particularly as it’s likely easier to get accepted than in a top journal.  But note that publishing in the specialist journals for your field may be much less effective than aiming for higher-quality journals in the “parent” disciplines.  And – taking those averages at face value – ask yourself; will my purposes be better served by spending one month on a not-so-pukka journal; or spending six months getting my paper into a decent journal.  The ratio of time invested:average citation is the same.  The impact at least on an academic career is far higher for the latter.  (See below for an ICT4D specialist journal list.)

- – Reports, handbooks, and professional magazine articles: you may often be reaching out to a practitioner audience, so citation rates may be a poorer reflection of impact.

- – Book chapters: a high-quality edited book project in which the editor invests a lot of time might be worthwhile.  Otherwise, leave this to those who are just starting to build a publication record.  As for edited books from publishers who churn out an endless series of such items: caveat scriptor.  In citation terms, your chapter is likely to disappear from view faster than a golfing superstar after a car accident (this joke will self-erase when Tiger wins the Masters).

- – Conference and seminar presentations: I already discussed this in the previous blog entry on ICT4D research quality and impact.  If your presentation will lead on to a journal publication, it’s a slightly different matter.  Otherwise, attend conferences for lots of reasons.  But not in order to get cited.

ICT4D Specialist Journals:

If you’re looking to get published in a specialist ICT4D journal, then the following represents the current English-language list I’m aware of (please comment to add others):

What shall I say about this curate’s egg of a list?

Perhaps just that I have only had the pleasure of publishing in EJISDC, ITforD, and ITID.  That I have only had the pleasure of reading articles from these three plus AJIC, AJIS, AJC and ID.  That ID is the only that is WoK-listed.  And that AJICT, AJIT and IJAICTER look fairly techie, though with the occasional softer article.  Most are open access except for AJC, AJIT, ID, ITforD, IJICTRDA and IJICTHD.

If you want to cross-check against a journal ranking list, see: http://www.developmentinformatics.org/resources/journalranking.html – I could only spot AJC ranked A, and AJICT, ID and ITID ranked C.

You can find much longer unranked lists which include journals in parent informatics and development disciplines plus adjacent areas such as telecoms maintained by:

The IPID network provides a regular and very useful circulation of journal calls and contents.  If you’re not linked in to IPID, you should be.  To get on the circulation list, email: gudrun.wicander {at} kau.se

About these ads
  1. 31 March 2010 at 1:39 pm | #1

    Now that was an interesting post.

    But I think leaving out blogs and website misses something key in this ICT world of ours. I was sent this blog from 3 different sources within 24 hours of your email to me. As a flow of information isnt that rather good going? – hasnt it effectively found its way into the minds of important people (not that I am claiming to be important of course)?

    I dont think your indicator – “citation” really captures influence. (And you kind of acknowledge that in your comment about development practitioners)

    As discussed in Sweden – while I accept that publishing research in journals is important for academic career, in terms of policy influence isnt a blog or website potentially just as influential?

  2. Hector Mongi
    31 March 2010 at 4:36 pm | #2

    Thanks Prof Heeks

    This is really a wounderful resource. I may agree with the commentator above on the importance of blogs and websites Vs Journal and books as far as impacts are concerned. Broadly speaking, when one googles, the common results are those from websites and blogs. Very often, it is under refined search, example through google scholar that journal articles can be accessed.

  3. Richard Heeks
    31 March 2010 at 6:03 pm | #3

    Seems like there are two related questions in here.

    First, are there ways to measure impact other than via citations? IDRC and DFID (and no doubt others) have both done work looking at how research impacts development policy-making, for example. But that comes up with a lot of quite different answers to the one’s in this blog; probably worth a separate entry on that, later.

    Second, how could we measure the impact of blogs, wikis, websites, etc? I know one can use hits and links and the like, but I would make some defence of Google Scholar. As I noted, it does show the egov4dev website has had about as much citation impact as a book. I also checked the type of documents it covers using a sample of things in which I’ve been cited. By far the majority are academic sources, but there are international agency – UN, World Bank, etc – reports too.

    Of course GS does only include specific citation (and probably only citation that comes close to academic norms); and certainly not more nebulous impact like the general influence of ideas. To trace that, you’d have to adopt the more in-depth, qualitative approach IDRC and DFID use to track research impact.

  4. Richard Heeks
    27 April 2010 at 12:58 am | #4

    A clarification of something implicit – if you publish via a particular publication type, these figures do not, of course, mean that you will necessarily achieve a particular number of citations.

    The type of publication will have some effect on the likely level of citations, but much of the cite-ability of your paper will depend on other factors. As noted in the blog, timing is one. The quality of your conceptualisation, research methods, writing, etc will of course matter. If your ICT4D research is crap, you won’t get cited, no matter what type of publication you use: see the earlier blog entry on what constitutes good ICT4D research.

  1. 14 April 2010 at 1:12 am | #1
  2. 28 April 2010 at 11:13 am | #2
  3. 8 June 2010 at 10:41 am | #3
  4. 7 February 2013 at 9:35 am | #4

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 555 other followers

%d bloggers like this: