Study finds problems with health information on the Internet

ANN ARBOR—If you search the Internet for advice on treating your health problems, much of what you get may be inaccurate, inappropriate, misleading or unreviewed by doctors, according to a new study by researchers from the University of Michigan Health System.

The team performed a typical Internet search for information on a single type of cancer, and analyzed a sampling of the retrieved pages. They found that nearly half of the pages that contained treatment information apparently hadn’t been validated by scientific scrutiny. About 6 percent of those had wrong information, and many more were misleading. The search also turned up hundreds of frustrating dead ends, bad links and pages with no medical information.

But, the team writes in the journal Cancer‘s lead August article, their finding doesn’t mean that Internet users should stop looking for health information—or that doctors should dismiss the data their patients find. Rather, they say, the results should encourage physicians to discuss such information with their patients, and to steer them toward trustworthy sites.

“The Internet is becoming a powerful force in medicine, and doctors should view this development as an opportunity,” says lead author J. Sybil Biermann, M.D, U-M assistant professor of orthopaedic surgery. “For the public’s sake, we should work to improve the quality of health information on the Internet, and to increase public understanding of how important it is for medical data to go through the process of scientific verification. In the meantime, the best advice for the public is, ‘Consider the source.'”

The study, one of the first published efforts of its kind, statistically examined a sample of the pages retrieved when four popular Internet search engines searched for information on Ewing’s sarcoma, a rare and often fatal form of malignant bone cancer that occurs mostly in children and teen-agers. The uncommon disease was chosen to keep the search results manageable.

The search simulated the approach of the millions of ordinary Internet users who have been told that they or their loved ones have a particular disease, and set out to find more information. The team reviewed 400 Web pages for accuracy and appropriateness, and determined whether doctors had verified the information on them—a process known as peer review.

Peer review aims to squelch inaccurate or misleading information by exposing each piece of research to the anonymous review of disinterested experts before it can be published in a medical journal or presented at a scientific meeting. For example, reviewers scrutinized the paper by Biermann and her colleagues before it could be published in Cancer.

The 400 pages evaluated in the study were culled at random from more than 27,000 retrieved when four different spellings of the disease’s name were entered into four search engines. Two researchers acting independently evaluated each of the sampled pages and rated them; a statistical test showed good agreement.

Fewer than half of the sampled pages contained information about medical treatments for Ewing’s sarcoma. Of those, nearly 60 percent had peer-reviewed information from the National Cancer Institute or other reliable sources. But the rest contained data and recommendations that came from non-reviewed sources—or didn’t list the source at all.

The researchers used their combined expertise and a standard textbook to determine that a full 6 percent of the unreviewed pages contained wrong information. What’s more, some contained information about life expectancy that could be overly distressing or falsely encouraging to the 250 Americans diagnosed each year with Ewing’s sarcoma.

Medicine in the Information Age
The exchange of health information is now a leading use of the World Wide Web and of other Internet resources such as discussion groups. According to the Internet research group Cyber Dialogue, (http://www.cyberdialogue.com) 25.5 million American adults will use the Internet this year to find health information.

In fact, the idea for the study came from Biermann’s own experience as the surgeon to hundreds of children and adults with different forms of primary and metastatic bone cancer, including Ewing’s sarcoma.

For several years, she reports, patients have been coming to her armed with printouts of pages from the Internet. Some have even asked why her strategy for treating their disease didn’t exactly match the advice they got online, or have called a new clinical trial to her attention.

She adds, “It’s no longer a situation where doctors dispense information and patients passively listen; any patient or their family can now go on the Internet. But nobody really has a handle on what’s out there for them to find, and it’s been difficult to quantify how good the information is.”

One of the hardest classes of medical information to assess, Biermann continues, is the anecdotal or “personal story” kind. The Internet abounds with newspaper feature stories and discussion group testimonials. “There is some value to anecdotes—when they’re detailed, doctors call them ‘case studies,'” she says. “But you have to make sure that appropriate conclusions are drawn from the evidence at hand.”

All in all, she concludes, there is plenty of value to be found on the Internet, and plenty of value in the interaction between patients and physicians that comes about because of it.

“I encourage my patients to use the Internet,” says Biermann. “I ask them if they found anything surprising or different from the care I’ve recommended, and I give them addresses of Web sites I’d recommend, such as government and nonprofit sites. I fully believe that physicians should be partners in this. We should view it as an opportunity to give our patients more information about their case.”

EDITORS: Dr. Biermann’s Web picks for health information:

University of Michigan Health System: http://www.med.umich.edu
Contains “Health Topics A to Z”—descriptions of numerous conditions, treatments and medical issues, approved by doctors at one of the nation’s leading medical institutions.

National Cancer Institute CancerNet: http://cancernet.nci.nih.gov
Contains the PDQ cancer information database on many cancer topics, as well as other information for patients and physicians.

Intellihealth: http://www.intellihealth.com/IH/ihtIH
Created by Johns Hopkins University’s Health Information Service, a site with information from a wide variety of sources.

American Cancer Society: http://www.cancer.org
Includes the Cancer Resource Center, information on cancer for patients and families.

Drkoop.com: http://drkoop.com
A comprehensive site published under the name of Dr. C. Everett Koop, former U.S. Surgeon General.

Health SystemCancerJ. Sybil Biermannhttp://www.cyberdialogue.comhttp://www.med.umich.eduhttp://cancernet.nci.nih.govhttp://www.intellihealth.com/IH/ihtIH