Was Study Of Wikipedia Errors Wrong?

By Katie Wike, contributing writer

A recent study claimed that Wikipedia’s health related articles were wrong 90 percent of the time. Some experts, however, says that those results should be viewed with skepticism.

Health IT Outcomes previously reported results of a study by Hasty et al. published in The Journal of the American Osteopathic Association. This study claimed that, when researchers took an in-depth look at entries for the 10 most costly medical conditions on Wikipedia, they found 90 percent of them contained errors.

“Most Wikipedia articles representing the 10 most costly medical conditions in the United States contain many errors when checked against standard peer-reviewed sources. Caution should be used when using Wikipedia to answer questions regarding patient care,” concluded those researchers.

Now, a handful of Wikipedia contributors who also happen to be respected members of the medical community are encouraging readers to view the study with skepticism. According to these experts, the previous study used partnered medical students to review a total of only 10 articles, comparing facts and checking for errors.

“While hundreds of facts were identified in the ten articles, each team of two evaluators rarely agreed on what these facts were. Often, the number of facts identified differed nearly twofold between the two evaluators,” write these skeptics in an article published by the Cochrane Collaboration.

They also defended their authors, writing, “Whereas this study was based on assessments made by a small cohort of students, Wikipedia is built by a consensus of people, many of whom are experts. A recent survey of our medical editors found that 52 percent have either a masters, PhD, or MD, and another 33 percent have a BSc (unpublished data).”

Also, the study had no comparison, it used only Wikipedia as a source to make these claims. “Hasty has created an unvalidated method to judge the quality of medical literature. He has then applied this method to a single source: Wikipedia. Even if we were to accept that getting two students to sample the literature on a topic in a haphazard way provides an appropriate basis for measurement (which we do not), without a comparator this single data point has little meaning,” the authors of the article concluded.