James A. Herrick
From Argumentation (Allyn and Bacon, 2000).
The Internet has emerged as an important source of evidence for advocates. Many scholarly journals, government publications, and other important materials are now available online. Some sites provide links to hundreds of journals and newspaper, offering virtual encyclopedias of information on topics ranging from the latest medical research to developments in the entertainment industry. However, evaluating Internet sources can be difficult for several reasons, not the least of which is the sheer number of sites currently available. Other obstacles to accurate evaluation are also apparent. Sometimes we know little or nothing about the parties responsible for creating a particular site on the Internet. And, Internet sites often do not have to endure the editorial process that is typical for most books and virtually all articles in scholarly journals. For these reasons, it is particularly important to evaluate the quality of an Internet site as a source of evidence.
A number of colleges and universities have excellent Internet sites devoted to just such evaluation. Among the particularly helpful sites are those created by UCLA (http://www.library.ucla.edu/libraries/college/help/critical/index.htm), Purdue (http://www.lib.purdue.edu/InternetEval/dhtmlintro/dhtmlintro.html) and Johns Hopkins (http://milton.mse.jhu.edu/research/education/net.htm).
The following questions are helpful when evaluating Internet sources.
First, we need to ask, Who created the site? As you know, millions of organizations and individuals have created sites on the Internet. In fact, you may have your own site or webpage. Ease of participation in the Internet is, of course, one of its great attractions. But, searching for evidence this vast assortment of sources also poses some problems, not the least of which is knowing which sites are reliable sources of evidence. Thus, it is vitally important to try to ascertain what group or individual stands behind a particular site. Let’s begin with organizations.
Well known and established organizations such as the American Medical Association (www.ama-assn.org), the Federal Bureau of Investigation (www.fbi.gov), The Smithsonian Institution (www.si.edu), or the National Aeronautics and Space Administration (www.nasa.gov) have developed Internet sites with a great deal of reliable information that can be of use when arguing a case.
Other kinds of organizational sites, however, may pose special problems when used as sources of evidence. For example, we may not know whether an organization is politically neutral or biased. Lobbying organizations such as the National Rifle Association or Greenpeace will have a political agenda that may influence how information is presented and interpreted on their Internet sites. This does not mean that information on such sites is unreliable. But it does mean that you should always try to identify the organization sponsoring the site you are using, assess the organization’s credibility and political stance, and determine what biases might influence the presentation of information. When possible, consult sites created by organizations with opposing views. Reference librarians can help with this process of evaluating organizational sites.
When you employ an Internet site as a source of evidence, qualifying the source will mean identifying the organization or individual creating and maintaining the site. When there might be a question about credibility, qualifying an Internet source will also mean assuring your audience of the credentials of the sponsoring individual or organization. Citing an Internet source in footnotes is closely related to qualifying your source. It is never sufficient citation simply to present a site’s URL, though the URL should always be included. For example, the URL http//:www.atsea.org as a citation for evidence regarding the management of Northern Pacific fisheries does not provide enough information about the source of the site for any reader or listener to make an informed judgment about it. The At-Sea Processors Association, a Bering Sea fishing industry lobbying organization, manages this particular Internet site. The site is highly developed and contains a great deal of information on fishing regulations and the management of fisheries. However, the organization sponsoring the site also has a definite agenda that anyone using evidence from the site needs to be aware of. When you do cite an Internet source, always identify the sponsoring organization or individual. It is helpful to add a note about the organization’s purposes if the group is not widely recognized. For sites managed by individuals, provide their names. When citing an article taken from an Internet site, provide the author’s name, as well as the title of the article, and the date it was published or posted. The organization sponsoring the site where the article appeared should, as always, also be identified.
Another important question to ask of any Internet source is, For what audience is the site intended? Some sites are intended for the general public, others for those with a special interest in a topic, and others are intended principally for experts. Sometimes the intended audience and the actual audience for a site are not the same, and problems can arise. Recently Dr. Gretchen Berland and several colleagues studied the uses of medical information on the Internet. Their findings were published in the Journal of the American Medical Association, and appeared in the online version of the journal as well.
Dr. Berland and her colleagues discovered that “the reading level of most Web-based material is quite high,” in fact, so high as to make practical use of the information difficult for most people. She writes, “all of the English-language sites had material that required at least a 10th-grade reading level, and more than half of the sites presented material at the college level. Dr. Berland adds, “according to the 1992 National Adult Literacy Survey, 92 million adults in the United States almost 48% of the population and more than 75% of current welfare recipients have low or very low reading skills.” She concludes, “thus, even if wider access to computer technologies narrowed the digital divide, the online health information currently available would be difficult for many people to understand.”1 So, the intended audience for many medical sites on the Internet appears to be a highly educated audience, a fact that renders much of the available evidence difficult for many people to understand and benefit from. When using information from such a site as evidence, translating the evidence into readily understandable terms would be important.
Dr. Berland’s findings also raise the important question of accessibility. We have already noted that the test of accessibility asks whether evidence is available for examination. This test can be applied to evidence located using the Internet. For example, suppose that a webpage with the heading, “Americans for Electoral Reform,” makes the following claim:
A full recount of the ballots cast in Florida during the 2000 presidential election proves that Al Gore won the state by a margin of more than 5,000 votes.
You wonder whether this claim can be true, and begin to look for the source of the claim. However, no source is cited or otherwise mentioned anywhere on the webpage. The fact that this claim appears in an electronic source does not by itself substantiate the claim. It is reasonable to expect to see an accessible source cited for this claim. Moreover, nothing on the webpage identifies the group, “Americans for Electoral Reform.” This page provides inaccessible evidence from a source that does not identify itself sufficiently to allow for evaluation.
Inaccessible evidence should not be considered strong evidence, whether it appears in a traditional publication such as a book or periodical, or on a webpage. As noted above, we are under no obligation to accept such evidence. And, as with more traditional sources of evidence, citation of sources of evidence should be an expectation of websites. Let’s look at an example in which evidence from an Internet source satisfies this tests of accessibility. Each year, US News and World Report publishes its rankings of America’s top universities. These rankings are available on the magazine’s website. www.usnews.com. In its Best Colleges 2001 report, US News ranked Princeton University as the nation’s top university, followed by Harvard and then Yale. Because this information comes from a reputable publication, and because US News is identified as the source of the webpage and of the information available there, it is safe to say that the criterion of accessibility has been satisfied in the following statement:
US News and World Report ranked Harvard as America’s top university in 2000, as reported on the magazine’s webpage.
But suppose that a member of the audience hearing this claim happens to be from the West Coast. Her suspicion is aroused about the evidence because Cal Tech, Stanford, UC Berkeley did not show up in the list of the nation’s top three universities. Her question upon hearing this evidence might be, “What factors were considered in arriving at these rankings?” This also is a question of accessibility. This audience member wants access, not only to the evidence and its source, but also to the criteria of evaluation US News employed in arriving at its rankings. In this case, US News publishes not only its rankings, but also its criteria of evaluation.
In fact, the US News webpage provides an easy link to a complete discussion of its selection criteria with the title, How we Rank Colleges. Two reporters, Robert J. Morse and Samuel Flanigan, write: “Our method uses 16 measures of academic excellence.” These sixteen measures and their relative weightings are readily accessible and clearly discussed on the webpage. They include such factors as academic reputation, retention, faculty resources, student selectivity and financial resources. This webpage, then, satisfies the criterion of accessibility. Any evidence derived from the page should be viewed by audience members as accessible to them, even if they disagree with the evidence or conclusions drawn from it. By the way, US News ranks Cal Tech fourth, Stanford sixth, and UC Berkeley twentieth. In addition to being accessible, Internet sources should also satisfy the criterion of consistency. As you know, it is not difficult to create a webpage. This fact makes the Internet an ideal location for publishing views that are eccentric or extreme. For that reason, it will be especially important to consider the test of external consistency when evaluating Internet sources.
As noted earlier, it is important to ask whether evidence is at odds with the best evidence from other sources. This is the test of external consistency. Consider the following example. A friend e-mails you a link to an Internet site that purports to prove that American astronauts have never set foot on the moon. The site argues that the moon landings were staged in a huge studio at a secret military base, and that faked broadcasts and photographs were employed to dupe the world into believing that Americans had reached the moon first. The page comes complete with several photographs and allegedly expert analysis of such factors in the pictures as the impossible angles of shadows and the unlikely position of the Sun. It all sounds rather convincing. Should we accept this evidence?
The fist step in evaluating this evidence would be to ask about the source, that is, to apply the test of accessibility. You look through the entire site and find no source identified. The individual analyzing the photographs is not named, nor are any credentials presented to support this expert’s analysis. No organization is identified as having created the site.
The second step in analysis would involve asking whether other reliable sources of evidence contradict the claims advanced on this site. Here the answer would have to be, yes. The National Aeronautics and Space Administration, NASA, stands behind the claim that Americans did reach the moon more than once, the first visit occurring in 1969. Moreover, thousands of people who have worked for NASA and other related agencies have had ample opportunity to debunk claims to a moon landing if, in fact, the landings had been faked. We might add that the astronauts themselves have testified repeatedly to the authenticity of their voyages to the moon. Finally, the photographs in question have been examined countless times by qualified experts, and even by experts working for governments hostile to the United States, and questions about their authenticity have never been seriously advanced. In other words, the evidence contradicting the claims of this site in question is sufficient to lead one to conclude that the site fails the test of external consistency.
Internet sources should also pass the test of internal consistency, that is, they should not contradict themselves. Let’s return for a moment to the research findings of Dr. Gretchen Berland. Dr. Berland found that a high percentage of sites providing medical information actually involved internal inconsistencies. She writes, “overall, just over half of English language Web site reviews revealed 1 or more conflicts in the information provided.” What sorts 5 of contradictions did Berland and her colleagues discover? They write, “conflicts involved treatment (present in 35% of reviews), diagnosis (13%), definitions (7%), adverse effects (5%), etiology and risk factors (5%), and incidence and prevalence (4%).” She provides the following example: “a childhood asthma site stated at one point that inhaled steroids do not stunt growth and later stated that inhaled steroids do stunt growth.” Berland adds, “materials on depression were the most likely to have conflicts on treatment, whereas breast cancer materials were the most likely to contain conflicts on diagnosis (P<.001).”2 So, it appears that even sites created and maintained by experts in a field may be internally inconsistent.
Recency remains a concern with Internet sources as with other sources of evidence. One of the great advantages of the Internet as a source of evidence is that information can be updated frequently. However, this fact does not guarantee that every Internet site is being carefully maintained and updated. It is usually relatively easy to determine when a site was last updated. But, the absence of a date on the site should cause us to do some further checking about the recency of information presented there. Of course, other tests of evidence such as relevance and adequacy still apply to evidence gathered from Internet sources.
*James A. Herrick is the Guy Vander Jagt Professor and chair of the Department of Communication at Hope College in Holland, Michigan. 1 Gretchen Berland, M.D., et al. Journal of the American Medical Association, online edition, Vol. 285 No. 20, May 23/30, 2001. http://jama.ama-assn.org 2 Berland, et al.