Activism or science? A debate on open access.

Yesterday, I hit a nerve on Twitter. Ok, more than one. But it resulted in a great discussion about open access and brought up some interesting questions. I’d like to take the opportunity to explain in more detail what I meant and did not mean by my tweet. And then I’d like to open up the discussion further. But first, the backstory.

I am writing a systematic review. For those not familiar with the concept, this is not simply summarizing work others have done in a particular area of research. It involves designing searches, implementing filters, and clearly outlining criteria for selecting or excluding articles. The idea is to give a complete overview of the literature and be able to quantify, for example, what percentage of studies in the research area used a certain technique, or arrived at a common conclusion.

To give you an idea of the scale of my review, my searches retrieved 370 results. 73 of these were duplicates and excluded, leaving me with 297 unique articles to screen. From these, I excluded 183 studies whose results were irrelevant to the questions my review seeks to answer, a further 8 which were review or comment articles, and another 7 studies which used techniques other than the ones to which I’m restricting my review. Many of the articles I retrieved and screened were published in subscription journals, but I gained access via a university affiliation. Even so, there were some articles I couldn’t retrieve. So, I tweeted this:

The responses soon came in.

I think Dennis and Matt are right. Papers shouldn’t be excluded due to non-scientific reasons, and systematic reviews should be complete. But I argue that my reason for excluding articles to which I don’t have access is scientific and related to completeness. First, from a practical standpoint, I can’t extract the information I need. My review involves specifying certain methodological details of each study – details which are not available in the abstracts. Second, and more generally, I cannot evaluate the quality or validity of a study if I can’t read it in full. There are other studies which I have excluded from the review after reading the full text and finding inconsistencies in the methods, or disagreement between statements in the methods and results sections. As a scientist, I think it is irresponsible of me to cite work I haven’t properly evaluated.

With respect to the review, my thinking was as follows. Typically, such reviews include a schematic with boxes enumerating at each stage how many articles were excluded and for what reasons. I planned to include a box for those 7 (I have now decreased that number to 5) articles that I could not access even with my university account. This seemed fair to me. Based on a lack of information, I couldn’t very well include them nor assign them to any other exclusion box, and I was clearly stating the reason for exclusion.

This is where the nerve was hit. Some suggested I was practicing “activism” rather than scholarship, “shunning paywalled literature” for the cause of open access. This is where I think my tweet was misunderstood. It’s no secret I am an advocate of open access, and perhaps I brought some of the misinterpretation on myself for having included the #openaccess hashtag in my tweet. But activism was not my intention here. I was not suggesting that people exclude articles from their reviews just because they are not open access. In fact, I am including at least 40 articles in my review which are paywalled. Unfortunately, because such a large proportion of biomedical research is currently paywalled, it would not be a very thorough or informative review if I excluded these articles. My intention was instead two-fold: (1) simply to point out that a scholar – and even one who does have access to a large portion of the paywalled literature – was still struggling to do a thorough review because of a lack of access, and (2) to alert (or remind) authors who publish in paywalled journals that their work may not get cited not because other researchers aren’t interested, but because they can’t read it.

Some tweeters responded that my job as a researcher doesn’t end when I hit an electronic paywall:

Many tweeted that I should contact the authors directly to request reprints or try interlibrary loans. I wondered whether, given that I have 67 other eligible studies I can read, is it worth the time and effort to track down the 7 (now 5) I can’t? In the interest of thoroughness, I suppose the answer is ‘yes’. But some questioned whether the burden was on me to do this:

While I wasn’t intending to argue that papers should be excluded just because they are paywalled, some were arguing exactly this point:

Others disagreed:

The debate hinged on communication. As a scientist, is it your responsibility to communicate your science? If you lock up your results, shouldn’t that be considered a failure of communication and therefore unscientific? Or, is it the responsibility of the person conducting the review to communicate effectively, including all relevant literature whether paywalled or not?

In fact, others suggested that including paywalled articles in open review papers could be seen as supporting open access by bringing those results out into the light:

People got passionate about this one. And I can understand why. I am passionate about open access. There are few things more frustrating as a researcher than seeing that nasty paywall come between you and science, except perhaps seeing that frustration in your students or your patients.

But they are ways you can actually hurt the cause you are trying to support. There are simple issues and complicated ones. Should researchers lock their work behind paywalls? No. I think that’s pretty simple. And if I’m expressing my personal views on the matter, I also think it’s unscientific. I express that few frequently with colleagues, collaborators. I write about it. I try to be as open as I can with my own work. That is my form of activism. However, given that a lot of relevant research is currently locked up, how should researchers like myself deal with that? That’s a little more complicated and I don’t pretend to have all the answers. I do believe we have to be careful that our views of what should be with respect to open access do not cause us to act unscientifically. Do I like that biomedical research is paywalled? No. Does it make me angry? Yes. Does that mean I have the right to exclude paywalled research from my review? I don’t think so.

So, I will make additional efforts to retrieve those articles I am missing. If I get them, great. If I don’t, then I can say my reason for excluding them was purely scientific. I could not evaluate the work. Period. And in the meantime, I’ll keep advocating for open access. Hopefully, the next time I do a systematic review access won’t be an issue.

About these ads

21 thoughts on “Activism or science? A debate on open access.”

  1. Excellent summary! I want to add that I am also completely for open access publishing. However, as long as articles are paywalled, it is unreasonable to make it some author’s responsibility to provide open access to their work. There is work published before the invention of open access. This work is bound to copyrights owned by the journals. It is not the author’s fault these are not openly accessible.

    As you said, you can be vocal about open access and you can be a role model by chosing open access and everybody should. But ignoring everything you can’t access without some hurdles undermines your reputation as a scientist. And the point of Benjamin de Bivort is a good one, too.

    1. It is totally the author’s fault! No one held a gun to their head to forfeit their copyright to publishers. No one is coerced to publish closed access.

      Now, the misaligned incentives system forces many academics’ hands. But let’s not kid ourselves that they do so with full awareness of the consequences. And if they don’t think through the consequences of paywalling publicly funded research, then they are acting as poor stewards of taxpayer money, and defeating the point of the original investment in the basic research enterprise.

      1. I think in many circumstances it can be the fault of authors that their work is not accessible, as I mentioned below. But I’m not sure it’s fair to say “no one is ever coerced to publish close access”. For example, graduate students may be coerced into publishing in certain subscription journals by advisors. In this case, I would not fault the student author, but I would fault the advisor.

        I agree with you about misaligned incentives. We need changes in attitude, not just from individual authors, but from those at the top who decide hiring and tenure policies. We have to evaluate people on the quality and real impact of their work, not on which journal it was published in. When that attitude changes, I think we will see more progress in making the literature open.

    2. Thanks! Yes, I should mention that I don’t think anyone involved in the discussion was arguing against open access per say. In fact, the need for open access seemed to be the one thing we all agreed on!

      I have to disagree, though, with the idea that authors are not at fault for articles not being accessible. First, authors often (though not always, depending on circumstances) have a choice in where they submit their work. Second, even if authors publish in subscription journals that retain copyright, most journals allow some form of self-archiving. If archiving is allowed and authors still don’t make their work freely accessible, then fault does lie in part with them.

      1. Apart from your response to Ethan, that some authors may feel pressure from co-authors, there is also the point that up to quite recently there was no open access publishing. Relevant studies may be older than 10 years.

        Finally, ppen access journals are mostly relatively new and there is in general a conservative tendency in academia against ‘radical’ changes in procedures. You shouldn’t forget the large portion of quite senior tenured professors who have their habits and are entitled to them. You can not ignore their work just because you don’t like their publishing habits.

  2. I’m curious what is considered a reasonable “additional efforts to retrieve those articles I am missing.”

    Because the only ways to legally get access to an article for which your institution doesn’t have a subscription are (a) purchasing the article or (b) directly from someone (such as the author) who has a licence to legally distribute the content.

    Anything beyond that (e.g. #icanhazpdf) you then have to balance the ethics of illegally acquiring content versus leaving the literature review incomplete.

    1. Tough question. I’d say you can reasonably expect authors to explore legal channels such as ILL or contacting authors to request reprints. Even though I’m a fan of #icanhazpdf, I don’t think you can oblige researchers to use illegal means to obtain articles. (Whether they decide to use that route is up to them.) Payment as an option is trickier. Of course, it’s legal. But it can represent a significant burden to the researcher. In my case, I have no research funds. Assuming that each article costs on average $30 for me to purchase access, that would be a total of $150 out of my own pocket for those missing articles. Now imagine the case for researchers that do not have any institutional access. If they had to purchase all 297 articles, for example, the cost would be nearly $9,000. Can we really require researchers to pay that kind of money to access articles? I think that’s unethical.

      1. I actually went to a college where my access to articles involved requesting articles via pen and paper at the library and waiting 3 – 5 days for them to arrive, on a loaner basis. The campus library ate up the cost, but the time itself was extreme. Needless to say, so were the copier fees at times since we’d copy all the articles we requested. This sounds more like #firstworldproblems when I type it out, but the bottom line is that access was limited and unfriendly to literature review.

        Now, did we do this for all our articles? No. But we definitely “borrowed” articles and thereafter deciding it was relevant to our study, we requested for it through the library. That experience definitely teeter-totters on the line of ethics. It also tells me that regardless of your access, whether you’re a student or full prof, you will inevitably find a way to access what you want. And sooner or later, there will be a shift towards or away from the paywall.

  3. Great post, Erin! Really thoughtful blog post extension of a spirited Twitter thread discussion.

    For the record, I 100% agree with your approach. I only chimed in when some participants of the thread seemed to be tone deaf on the paywall crisis. Granted that’s a tangent to your specific difficulty wrangling a few stubborn paywalled papers.

    1. Thanks, Ethan! “Spirited” is a good word for it :) I was honestly a bit surprised by the response the tweet generated. But I’m glad people are talking openly about these issues. We need more of that.

  4. Awesome post! I’m torn. If I were in your shoes, I think I’d probably use #icanhazpdf, simply because I want to know the content of the papers without spending too much time. I’d want the papers not only for the review, but for myself. However, if that shouldn’t work, I’d cite the papers in the review as unavailable.

    1. Thanks, Björn! Yes, I may go that route…especially since I’ve noticed that some of the article records do not include corresponding email addresses. (Isn’t that brilliant?!)

  5. Here’s one important point that’s hinted at here but needs drawing out: if you are an author, don’t put meta-analysis authors through this sort of idiot time-wasting. Make it so that there is no question whether your work is available.

    BTW., Erin is making a special effort to track down all the non-OA sources because this is a systematic review. You can bet that in more informal reviews, such as the discursive introductions of most papers, the anti-paywall bias is much stronger.

    1. Well said, Mike. This is definitely frustrating. I’ve been going in circles. Track down a lead, hit a paywall. Go another route, hit a few promising sites, and then back to the same paywall. It’s far from being a good use of my time. And yes, if this was anything other than a systematic review, I probably would have given up on these articles awhile ago. If all I needed were references to cite, I have several other articles I can access which give me similar information. My citations would be going to those articles – not because of activism, but simply because of practicality.

  6. In my 12 years researching I haven’t found yet a paper that I can’t access in a way or another: friends, university access, interlibrary photocopy loan (completely legal) and so on. Even book chapters; old and new, online or printed. It’s a matter of time and effort… Exactly what looking for literature is.

    I am also open access, of course, but as someone mentioned before, open access came after still a majority of older papers published in closed access journals (when open access did not even exist), most of them not even digitalized.

    Thus, for me there is no debate about citing or not: they must be found, revised and cited.

    1. I’m not at all sure it’s that clear. Many results are obtained by never published at all: no-one thinks that authors of reviews have an obligation to read and include these. I don’t see how it’s qualitatively different in the case of a result that is nominally published but not actually available.

      1. How can an author include a result not published?? That’s a huge qualitative difference: published or unpublished. Just as an example: in the field of comparative morphology and anatomy there is a huge amount of old literature that cannot be found on the internet. But it’s there, and good scientists still get them and take them into account for their work (even being these in German). What I say is that it’s not profesional to try to get your bibliography only comfortably sitting down in front of your computer. Going to your library and ask your librarian can be a good exercise.

        Not that I am saying this is the case, though.

        By the way, why don’t you list the papers you cannot access, everyone could check if we have access through our institutions :)

      2. Juan asks: “How can an author include a result not published?”

        It might be in a dissertation. It might be presented at a conference. It might be summarised in an abstract. It might be mentioned in an email. It might be overheard in a pub. It might be hidden behind a paywall.

        I don’t see such a clear line here as you do.

  7. Cheers for this post Erin, the thoughts and discussion are quite useful. I’m not sure many ECRs would find themselves in similar situations though ;)

    Personally, I’ve had to exclude numerous articles from the ‘lit-review’ part of my thesis, which I consider to be a contextual systematic review of the literature anyway. I’ve got a list of papers written on unobtanium, and will probably have to go to one of them ‘physical libraries’ to track them down.

    The same applies for the database our research group uses, the Palaeobiology Database. All the data is sourced from the primary literature – if we can’t access it, our data is incomplete. I found this to be a bit of an issue when I started adding to it, as difficult to acquire papers were often excluded.

    Despite some of the comments written above, I think this is just the way science is atm – if we can’t access something, don’t reference it. Authors lose out (no citation boosts), and the researchers suffer a minor impediment (we’d hope) to an otherwise largely-complete data/literature source.

    Where I become torn on this subject is with blogging. I write quite a bit about developments in the field of Palaeo, and can’t decide if I should only blog about OA research or not. The idea being that as authors haven’t made their research openly available, why should I do their job for them by making it accessible? On the other hand, I see it as a duty of a member of a field to make it accessible, and therefore should ‘cover the assess’ of the authors who didn’t make their work accessible by writing about it.

    In the mean time, this: http://sci-hub.org/ has not once failed to get me a pdf that’s actually available in some manner online. It’s good.

    1. Yes, I’m torn on the blogging issue, too. As you point out, and as Benjamin de Bivort suggested in his tweet, I can see the argument for ‘jailbreaking’ closed results by writing about them in public venues. At one point, I considered establishing a series on this blog dedicated to doing just that with closed neuroscience literature. On the hand, I think there’s a scientific issue worth considering. That is, if the work is not accessible to many, then they simply have to take my summary and interpretation of the research at face value. They have no way of verifying (i.e. by reading the paper) whether my writing fairly represents the work. This is especially important if I am being critical of the experimental design or conclusions. I don’t know, this is a tough one. I’m still not decided.

      Thanks for the link to Sci-hub! Looks like it could be a great resource!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s