The URL posted by Calacanis links to a Reuters UK news release entitled “Wikipedia Often Omits Important Drug Information“. Obviously, with such a title, my interest was piqued.
The article — which is based on findings presented in The Annals of Pharmacotherapy, December 2008 — basically states…
Consumers who rely on the user-edited Web resource Wikipedia for information on medications are putting themselves at risk of potentially harmful drug interactions and adverse effects, new research shows…
The researchers compared Wikipedia to Medscape Drug Reference (MDR), a peer-reviewed, free site, by looking for answers to 80 different questions covering eight categories of drug information, for example adverse drug events, dosages, and mechanism of action.
What the researchers found, was that MDR answered more than 80% of the questions, while Wikipedia only answered 40%, and Wikipedia answers were less likely to be complete — for example, the Wikipedia entry for the anti-inflammatory drug, Arthrotec (diclofena and misoprostol), was missing information that it can cause women to miscarry.
Thus, the researchers advised that “If people went and used this as a sole or authoritative source without contacting a health professional… those are the types of negative impacts that can occur“.
While I would definitely agree with that statement, one would probably conclude from what was stated so far, that we should NOT rely
on user generated and maintained content sources, such as Wikipedia; but rely more on professional, peer-reviewed resources, such as Medscape Drug Reference.
However, the article goes on to say that although Wikipedia tended to have a higher number of “errors of omission” (48 omissions, compared to MDR’s 14 omissions), it had FEWER factual errors, while MDR had 4 factual inaccuracies!!! This is a peer-reviewed professional resource, folks… It’s what physicians and other medical professionals use to look up drug information… And it’s got more errors than a user generated/maintained wiki. WOW!
“Of the answers the researchers found on Wikipedia, none were factually inaccurate, while there were four inaccurate answers in MDR.”
Additionally, the researchers also found that “…after 90 days, the Wikipedia entries showed a “marked improvement” in scope“. Not sure exactly what they mean by that, but I can only assume that some of the wiki-contributors submitted more information about the drugs they were evaluating(perhaps reducing the amount of info that had previously been omitted), which IMHO demonstrates the true power (multitude of “editors”), flexibility + speed (no editorial/review committee delays), and wisdom-of-crowds type network effect of a wiki-based resource.
So, I’ve got a few issues with this study (in case you can’t tell)…
- UNFAIR COMPARISONS
First and foremost, it’s not really fair to compare Wikipedia against a professional, peer-reviewed resource like MDR, especially when the researchers are obviously evaluating from the perspective of a consumer and not a medical professional. Instead, why didn’t they compare Wikipedia to WebMD or a similar Health 2.0 drug directory? That would probably be more balanced.
- OMISSIONS vs. INACCURACIES
It could be argued that MDR has a higher chance of inaccuracies because it contains more content and more detail (>80% answers), but then I have to question how a peer-reviewed professional resource that goes through a thorough editorial review process can end up with 4 inaccuracies in 80 questions (that’s 1 inaccuracy for every 20 questions).
Furthermore, at least Wikipedia entries are probably closely monitored and updated, allowing for additional updates and/or corrections to occur on a regular basis; whereas I would imagine that not much gets changed to content on MDR after it’s passed through the editorial process and published (unless there are changes to the label/indication or they receive complaints, of course). In fact, here’s a quote from the article that certainly implies that the inaccuracies are as dangerous as the omissions, if not more so (infer what you will from this):
“I think that these errors of omission can be just as dangerous” as inaccuracies, Clauson told Reuters Health.
On a similar note to #2, if the worry is that omissions by Wikipedia may lead to users being misinformed about a drug, then I think there also needs to be an examination of the types of perception that people have when using each application. I can’t say for sure, but I would think that most people use Wikipedia as an entry point for research, much like Google. Most people probably understand the wiki (user-maintained) concept and therefore treat the information as “exploratory”. They may then go on to find more in-depth information on more trusted, authoritative sites… Sites like Medscape, perhaps.
On the contrary, when someone uses Medscape — being a professionally-oriented, peer-reviewed, and funded resource — they EXPECT to receive detailed, accurate information that doesn’t require further research or confirmation. It is, after all, called a drug REFERENCE. So, perhaps the title (and contents?) of the article is misleading, in that it is strongly biased against the Wikipedia omissions and does not address the issue of the MDR inaccuracies whatsoever.
- EASE OF USE
Finally, when it comes to information seeking and, more importantly, finding the right information, it all boils down to how easy it is to navigate through an application and find what you are looking for (i.e. ease of use). And to that end, when you look at MDR as a starting point for consumers to search for a drug and comprehend what they’re being presented, then I think you’ll immediately understand why people turn to Wikipedia first. MDR was really designed for medical professionals (Medscape is the professional facing arm of WebMD), so it’s built for people that understand medicine and can easily navigate through complex information.
On the other hand, Wikipedia is (mostly) written by general consumers for general consumers, so the level of complexity and the way that it’s written is completely different (AFAIK, no paid medical writers on staff at Wikipedia).If you wan to experience this, just try searching for one of the drugs mentioned in the article, darunavir (I just picked that one at random). First try MDR and you’ll see the search result screen is already confusing — is it the oral form? Good thing there’s only one choice (you should try some others with multiple options). Hmm…
Next, click on the link and look at all the confusing options — would you know where to find the info you want?
OK, now let’s take a look at the same search in Wikipedia…
As you can see, you are immediately greeted by a much less “clutter” and a definition of the term up front that confirms what you are looking for, as well as breaks it down less complexity. There are also a ton of outbound links for finding additional info/resources, as well as anchors within the page for more info on efficacy, safety, dosing, etc.
So, while I am definitely NOT saying that Wikipedia is a better resource than peer-reviewed resources, I am trying to bring balance to an article that seems to unfairly place Wikipedia in a negative light, with regards to being a resource for medical information.
In the end, the ease of use, regular updates, and basic simplicity of Wikipedia — as well as it’s high ranking on search engines — often make it the first stop for a lot of people. It may not be perfect (although it’s been shown to be as good as the Encyclopedia Britanica, which nobody questions as a source), but it definitely has many positive aspects (including accuracy) that this study fails to point out or appreciate. I will give some credit to the researchers for concluding the following, though they still try to make Medscape to be the better option despite it’s inaccuracies:
Wikipedia can be a good jumping-off point for Internet research, Clauson said, but shouldn’t be seen as the last word on any topic-and should certainly not be used as a resource by medical professionals. “You still probably want to go to medlineplus.gov or medscape.com for good quality information that you can feel confident in,” he said.