Month: March 2010

ICT4D Research: How Should I Publish?

~ Richard Heeks ~ 8 Comments

In what form should you publish your ICT4D research for maximum impact?  A book; a book chapter; a journal article – if so what kind of journal; a conference paper?

You are welcome to comment your own evidence, but I’m going to answer that question through analysis of the publications of my favourite author: me.  Why me?  Because almost everything I’ve published is in the ICT4D field; there are a large number of items, over a long period, and in many different formats; and careful selection of my name at birth ensures very few false positives in citation searches.  But most importantly, I have access to the list of all items I’ve ever published, and so can include those that have never been cited.

The method: dividing out all my publications into different categories.  Then, for each one, identifying how many citations are shown on Google Scholar; and then working out the average number of citations per publication for each category.  The results are as follows:

Publication Type Mean Citations per Item Median Citations per Item % Items Never Cited n
Single Authored Book   96   48   0% 4
Refereed Journal Article (WoK-listed journal)   54   30   6% 17
Working Paper (available online)   27   8   22% 46
Refereed Journal Article (non-WoK-listed journal)   9   5   20% 15
Report / Handbook (available online)   6   3   29% 14
Book Chapter   9   0   55% 40
Magazine / Professional Journal Article   1.4   0   69% 80
Conference / Seminar Presentation   0.6   0   92% 98

Notes on the data:

– I’ve excluded web sites (like http://www.egov4dev.org) and this blog.  If I did analyse them, then putting all such items together, the vast majority (90%+) of individual pages and blog entries remain uncited, but altogether there are 113 citations, 82 of those from the egov4dev web site.  So a successful web site can be as highly-cited as a book.

– I’ve also edited two books, but not included them in the table for two reasons.  First, because – in a strict sense – I don’t think one should cite an edited book; one should cite from the individually-authored chapters within that book.  Second, because the two had such wildly-divergent fates, “average” makes no real sense: one is cited 9 times; one is cited 363 times.  (And, yes, I have sample checked that those citations refer just to the overall book and not to any individual chapter.)

– That pattern of very skewed data between items – some scoring well, others totally flopping – is found across virtually all types of output.  That’s why the table includes both mean and median scores.

– WoK is the ISI Web of Knowledge; inclusion of a journal in this is a rough quality benchmark.  I could have redone the citation figures using WoK instead of Google Scholar but I’ve looked at the data and the overall pattern is the same; just that citation numbers are around one-quarter of those for GS.

Conclusions From The Data:

– This is data from just one person: caveat lector.

– Trying to work out what differentiates a citation hit from a citation flop, one might think research quality – see separate entry on that – was part of the story (not, of course, that my publications would ever be anything but very high quality!).  But I don’t think it is.  To me, timing looks a big factor.  Write something half-decent in a sub-area that subsequently grows, and you’ll get a lot of cites (and, arguably, have some degree of influence in that sub-area).  Write something great in a mature field and you’ll get many fewer citations (of course in part because you’re then fighting for attention with far more “competing” publications).  The slightly depressing conclusion: if it’s citations you’re after, keep up with research fads and fashions.

– And then what about the answer to the initially-posed question on type of publication to choose?  I’ve arranged the table in what I would say is decreasing overall impact order, and I’ll comment on each category

– – Authored book: still the best way to have an impact, and the data here probably understates since it includes two quasi-turkeys from the 1980s when I had 100% no clue what I was doing (now I just have 50% no clue).  The two published more recently have an average 185 cites each and rising.  If we compared that to the median journal figure, then you would ask yourself – is the effort of writing a book more or less than that for writing six refereed articles in decent journals?

– – Pukka journal article: still the most obvious way to build a citation profile.

– – Edited book: editing a book is easier than writing one.  Editing an impactful book is probably not.  I’d suggest you need to write or co-write a number of start and end chapters yourself; and impose such a degree of consistency of argument, style and structure that you are often almost rewriting other contributions from scratch.  Plus see comment on book chapters below.

– – Working paper: can be highly effective but a lot probably depends on the profile and dissemination of the series in which you publish.  You get zero academic brownie points, but you can maybe modify into a journal article.

– – Not-so-pukka journal article: can be worth doing, particularly as it’s likely easier to get accepted than in a top journal.  But note that publishing in the specialist journals for your field may be much less effective than aiming for higher-quality journals in the “parent” disciplines.  And – taking those averages at face value – ask yourself; will my purposes be better served by spending one month on a not-so-pukka journal; or spending six months getting my paper into a decent journal.  The ratio of time invested:average citation is the same.  The impact at least on an academic career is far higher for the latter.  (See below for an ICT4D specialist journal list.)

– – Reports, handbooks, and professional magazine articles: you may often be reaching out to a practitioner audience, so citation rates may be a poorer reflection of impact.

– – Book chapters: a high-quality edited book project in which the editor invests a lot of time might be worthwhile.  Otherwise, leave this to those who are just starting to build a publication record.  As for edited books from publishers who churn out an endless series of such items: caveat scriptor.  In citation terms, your chapter is likely to disappear from view faster than a golfing superstar after a car accident (this joke will self-erase when Tiger wins the Masters).

– – Conference and seminar presentations: I already discussed this in the previous blog entry on ICT4D research quality and impact.  If your presentation will lead on to a journal publication, it’s a slightly different matter.  Otherwise, attend conferences for lots of reasons.  But not in order to get cited.

ICT4D Specialist Journals:

If you’re looking to get published in a specialist ICT4D journal, then the following represents the current English-language list I’m aware of (please comment to add others):

What shall I say about this curate’s egg of a list?

Perhaps just that I have only had the pleasure of publishing in EJISDC, ITforD, and ITID.  That I have only had the pleasure of reading articles from these three plus AJIC, AJIS, AJC and ID.  That ID is the only that is WoK-listed.  And that AJICT, AJIT and IJAICTER look fairly techie, though with the occasional softer article.  Most are open access except for AJC, AJIT, ID, ITforD, IJICTRDA and IJICTHD.

If you want to cross-check against a journal ranking list, see: http://www.developmentinformatics.org/resources/journalranking.html – I could only spot AJC ranked A, and AJICT, ID and ITID ranked C.

You can find much longer unranked lists which include journals in parent informatics and development disciplines plus adjacent areas such as telecoms maintained by:

The IPID network provides a regular and very useful circulation of journal calls and contents.  If you’re not linked in to IPID, you should be.  To get on the circulation list, email: gudrun.wicander {at} kau.se