EDIT 14/03/12: As pointed out in the comments, the accuracy of the PubMed’s database needs to be considered when analyzing retractions. Thanks L. Wynholds!
Unless you’ve completely been avoiding all social media platforms this week, you’ve likely come across the #StopKony/#Kony2012 campaign. In short, a group called Invisible Children created the video above that was meant to make Joseph Kony infamous, and encourage governments and people to act against him. By raising awareness, you can make a difference, the filmmakers argue.
The video went up, and almost immediately went viral. It was uploaded on March 4th, 2012. By March 5th, it had over 25 million views. As of March 11th i.e. one week, it has over 70 million views.
What I find most interesting though, is that almost immediately after the video went up, people started digging. And suddenly, this campaign was being questioned and criticized. Sites such as Visible Children found questionable information about the campaign, and it wasn’t long before the media started asking questions; Jezebel, NY Daily News, CTV, The Huffington Post (which featured comments from Ugandans) and The Atlantic all penned articles ranging from slamming Invisible Children (Jezebel) to presenting both sides (CTV). People were critical, they dug up information, and they presented facts to back up their criticisms.
Scientists are familiar with this process: It’s peer review.
There have been several high profile cases where scientists have either fabricated or cherry-picked data. Most, if not all, epidemiologists are familiar with the controversy surrounding Dr Andrew Wakefield. He ran a study in 1998 that showed a link between the MMR vaccine and autism in children (1). However, it was later found out to be fraudulent, and was retracted (2). There’s also the story of Dr Anil Potti, who claimed to be a Rhodes Scholar. Turns out he made that up/misrepresented himself, and now has a slew of papers that have either been retracted or are under investigation.
But how common is this?
Turns out it happens more often than you think. And not only does it happen a lot, it has also been going up over time, to the point where Retraction Watch is a blog dedicated to it. Part of this is attributable to the fact that more articles being published should result in more retractions (3), and with a wider audience any “issues” are more likely to be discovered (4).
Edit: Note that the denominator here is the number of papers indexed in PubMed. This should be considered when drawing conclusions; how are papers reported to PubMed when they are retracted, i.e. is this voluntary or automatic, how good is PubMed’s data back to 1977, and how broad a field does PubMed cover.
So what does this all mean?
With more and more information being available at our fingertips, and the absolute number of articles being published rocketing skyward, combined with a 24 hour news cycle that focuses on sound bites and headlines rather than delving into issues deeply, we, as scientists, need to be prepared and ready to critically analyze and evaluate information presented to us. The same way #StopKony was investigated and tested, we should evaluate information the same way, doing our due diligence and ensuring that the study has been performed to a high level.
In another news, I feel really sorry for this company: www.kony.com
References
1. Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., Valentine, A., Davies, S., & Walker-Smith, J. (1998). RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children The Lancet, 351 (9103), 637-641 DOI: 10.1016/S0140-6736(97)11096-0
2. Godlee F, Smith J, & Marcovitch H (2011). Wakefield’s article linking MMR vaccine and autism was fraudulent. BMJ (Clinical research ed.), 342 PMID: 21209060
3. Cokol M, Iossifov I, Rodriguez-Esteban R, & Rzhetsky A (2007). How many scientific papers should be retracted? EMBO reports, 8 (5), 422-3 PMID: 17471252
4. Cokol M, Ozbay F, & Rodriguez-Esteban R (2008). Retraction rates are on the rise. EMBO reports, 9 (1) PMID: 18174889
Mr Epidemiology 
March 12, 2012 at 1:41 pm
Organizations like charity watch are getting popular even with those that are less “plugged in”. I remember when I first stumbled upon humanewatch.org and petakillsanimals.com. It’s inspiring to see so many non scientists asking questions about information presented to them, and then making arguments based on evidence.
And you know, we really do have mythbusters to thank for much of this.
March 12, 2012 at 2:32 pm
Absolutely! Even something as simple/innocuous as chain forwards that go around on Facebook can be easily disproved with sites like Snopes. It makes me happy that people think critically and don’t just accept the information presented as fact; especially when the information is flawed or just downright wrong.
Thanks for the comment!
March 14, 2012 at 4:45 am
The claims made here about retractions of papers could use some of the same scrutiny that is being promoted. Errors in papers may or may not be on the rise. There are two main concerns that still need to be addressed in order to make that claim:
1) This data isn’t the number of retractions per year per 100’000 papers. It is the number of retractions per year per 100’000 papers per pubmed’s records. Did anyone verify that their record keeping processes remained consistent over that period? Retractions were not originally indexed in pubmed (hence the lack of records prior to 1977). I would suspect that they were added by hand or when notified by the publisher initially until the records could be loaded automatically. When did they start loading retraction records automatically? Was the loading systematic or voluntary? Without knowing whether these numbers represent ALL the retracted publications for that period vs. the ones voluntarily contributed by individual publishers, the data is meaningless.
This article suggests that pubmed didn’t start incorporating retraction data until 2002 (http://www.nlm.nih.gov/pubs/techbull/so06/so06_pm_10.html ), but the language is unclear as to whether that was when they started making it a part of pubmed or whether that was when they started the collection of the data.
This help sheet states that medline (the underlying database) didn’t start indexing data on errata until 1987, and comments until 1989, but fails to state when they started indexing data on retractions (http://www.nlm.nih.gov/pubs/factsheets/errata.html ).
2) Some disciplines have heavily favored publishing errata over retractions in almost all cases. There may have been a culture change around this point, given that there is a paper from 2004 that studied the policies of publishers for the retraction of articles and was unable to find policies for 62% of the journals (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC385306/ ) In order to be able to assert that there are more problems with papers, one would need to plot not only the retractions, but also the errata.
March 14, 2012 at 9:28 am
These are all great points. The denominator used may not be completely accurate. I would suggest that, at least for more recent years, it would be fairly close. Also, you would hope that a database as large as PubMed would be kept up to date with retracted publications, and that the “honour system” would encourage publishers to report retracted publications. But, as you rightly pointed out, this may not be the case. I would definitely be interested in seeing the data on errata and compare that to retractions.
Thanks for the comment – I hadn’t considered the denominator being inaccurate. I’ve added your points to the original post. Thanks! 🙂
March 14, 2012 at 3:26 pm
I know I probably sound like I’m harping on this, but the problem isn’t one of accuracy, the problem is one of understanding the constraints of the data and operating within them. It’s easy to mistake an artifact in the data for a phenomena, especially if the data is understood to be cold fact rather than a representation of a record of evidence.
This editorial from a couple of Nature editors outlines what I’m talking about pretty well:
http://dx.doi.org/10.1038/nphys1633