Exploring Online: Evaluating Information

Key Topics

Introduction

If you were looking for accurate information in a printed newspaper or magazine, would the first place you look be the Wall Street Journal or the National Enquirer? Would you expect to get factual information about cancer treatment from the National Institutes of Health or from a drug manufacturer claiming a new miracle cure? In your lifetime of information seeking, you probably have learned that some sources of information are inherently trustworthy, while others must be looked at more skeptically. The creditable sources, often described as 'authoritative,' have established a reputation for dependability and reliability. You can be confident of the accuracy of the information they present. "Consider the source" is valuable advice for information seekers.

There is an amazing amount of information available on the World Wide Web, but unfortunately much of it is not accurate, up to date, or even very good. The more you understand the characteristics that differentiate sites that deliver information (and opinion) from one another, the easier it will be to find "the good stuff," i.e., information you can trust. This section covers three topics that will help you distinguish between information you can rely upon and information that is questionable. The first section discusses a concept called critical thinking and gives you a set of tools for assessing the reliability of what you read and hear. The second section describes the process of evaluating information, both on and off the Web, and gives you a framework for deciding whether to rely on information you have found or to keep on looking. The final section discusses computer hoaxes and provides some guidance and useful resources for differentiating between fact and fiction in a world of ubiquitous email and instant messaging. Each section ends with a list of Related Resources (all on the Web) where you can find more information about the topic.

Back to Top

Critical Thinking

Definitions of Critical Thinking

Critical thinking is the thought process individuals use to decide whether what they have read or heard is true or false, authoritative or "off-the-wall," applicable or irrelevant. The process is generally the same whether the information being evaluated comes from a printed source (book, article, encyclopedia, etc.), from a digital source (a Web site on the Internet or a CD-ROM product), or from verbal communication from the TV, radio, college lecture, or personal conversation. Critical thinking is a way of evaluating what you have read or heard and forming your own opinion about it, based on an open-minded evaluation of its features and components.

Some useful definitions of critical thinking by people who write textbooks and articles on the subject can be found at the "Critical Thinking Across the Curriculum Project" Web site at Longview Community College, Lee's Summit, MO located at http://www.kcmetro.cc.mo.us/longview/ctac/toc.htm.

Here are a few samples of what these experts, who have spent considerable time thinking about this subject, have to say:

Critical thinking is deciding rationally what to or what not to believe. Norris, Stephen P. 1985. Synthesis of Research on Critical Thinking. Educational Leadership, 42 (8) (May): 40-45.

Critical thinking is careful and deliberate determination of whether to accept, reject, or suspend judgment. Moore, Brooke Noel, and Richard Parker. 1994. Critical Thinking.

Critical thinking is the examination and testing of suggested solutions to see whether they will work. Hall, Calvin S., and Lindzey Gardner. 1978. Theories of Personality.

In explaining what they mean by critical thinking, some of the writers quoted provide an operational definition. Here is one such example taken from the MCC General Education Initiatives:

Critical thinking includes the ability to respond to material by distinguishing between facts and opinions or personal feelings, judgments and inferences, inductive and deductive arguments, and the objective and subjective. It also includes the ability to generate questions, construct, and recognize the structure of arguments, and adequately support arguments; define, analyze, and devise solutions for problems and issues; sort, organize, classify, correlate, and analyze materials and data; integrate information and see relationships; evaluate information, materials, and data by drawing inferences, arriving at reasonable and informed conclusions, applying understanding and knowledge to new and different problems, developing rational and reasonable interpretations, suspending beliefs and remaining open to new information, methods, cultural systems, values and beliefs, and by assimilating information.

Back to Top

Critical Thinking Skills

What are the cognitive skills that are necessary for such critical thinking? Here are the views of two writers and thinkers as cited on the Longview Community College site (http://www.kcmetro.cc.mo.us/longview/ctac/toc.htm).

According to Daniel J. Kurland, in his 1995 book I Know What It Says…What Does It Mean?, critical thinking involves:

Frank Smith in To Think (1990) defines the key critical thinking skills as:

Additional insights into the characteristics of critical thinking can be found in an informative essay called, "Critical Thinking and Why it Counts" by Peter A. Facione, Dean of the College of Arts and Sciences, Santa Clara University, posted on the Web site of the California Academic Press at http://www.calpress.com/. This essay reports on the findings of a group of international experts who were asked to identify the cognitive skills required for critical thinking. The consensus view was that the required skills include the following:

Back to Top

Characteristics of Critical Thinkers

It becomes clear from the characteristics presented above, that critical thinkers think for themselves. They do not automatically believe everything they hear or read. They question things, using other knowledge and understanding they possess that is relevant to the current problem to help them question productively. Critical thinkers do not accept all information that they are given as fact. They question the clarity and strength of reasoning behind the information. They look at the source. They identify assumptions and values. They recognize points of view and attitudes, as well as evaluate conclusions and actions.

In the view of the same group of experts cited by Facione above (http://www.calpress.com/), the ideal critical thinker is one who is

Here are two other descriptions of critical thinkers, both from the Longwood Community College site (http://www.kcmetro.cc.mo.us/longview/ctac/toc.htm).

Critical thinkers, as seen by D. Ellis in Becoming a Master Student (1997):

S. Ferrett in the 1997 book Peak Performance says a critical thinker:

Back to Top

Evaluating and Validating Information Sources

Why You Need to Evaluate and Validate the Information You Find

In looking for information, you are faced with many decisions about the sources you might consult to find out about your topic. If you were to go to the library, you would be likely to find relevant information in such places as an encyclopedia, a reference book, a textbook, a journal or magazine, a pamphlet, or any number of other types of publications. Some of these sources may not tell you enough and some may tell you much more than you need to know. Some may provide a cursory overview of the subject and some may discuss some narrow aspect in a great deal of depth. You may be familiar with the credentials of a few of the authors or publishers you come across and may never have heard of many of the others. Even in a print-based world, trying to figure out what is authoritative, suitable, and reliable can be a challenge. Add to this the amount of information, much of it unverified, that you might find through a Web search, and you are faced with a daunting proposition.

Still, when you are looking for information, you must be able to make an informed judgment about the reliability of what you find before you accept it as fact. Basing a decision on erroneous, incomplete, or biased information can have unfortunate consequences ranging from simply embarrassing to downright catastrophic. So how can you know whether the information you find in a book, a magazine article, or on the Web can be trusted? By employing the same critical thinking techniques that you would apply to any information you read or hear. The criteria for evaluating and validating published information have been developed over many years and are often taught in high school and college. In recent years these same criteria have been modified and adopted for use in evaluating and validating the unfiltered information that is published (posted) on Web sites.

Back to Top

Techniques for Evaluating Published Material

"How to Critically Analyze Information Sources" (2001), at http://www.library.cornell.edu/okuref/research/skill26.htm, was developed for the use of students at Cornell University in Ithaca, NY by the Reference Services Division of the Olin, Kroch & Uris Libraries. Authors Joan Ormondroyd, Michael Engle, and Tony Cosgrave believe that the first thing to look at in evaluating information is where it comes from, i.e. its source. You can often do this without actually seeing the item itself, based on a reference in a book or article, or a bibliographic citation from a database search. During this initial appraisal, you are trying to determine how well this source will meet your needs based on the following criteria: author, date of publication, edition or revision, publisher, and where it appeared (e.g. in a scholarly journal versus a popular magazine). At the next level of evaluation, you begin to analyze the content itself. Start by reading the preface of a book or scanning the outline of a journal article to get an overview of the material. Then go on to see how well it meets your needs by evaluating it on the basis of:

Another university library site that helps to make sense out of the many available information options is "Critical Evaluation of Resources," (1998 - updated 2001), the Library, University of California, Berkeley, CA located at http://www.lib.berkeley.edu/TeachingLib/Guides/Evaluation.html. It also suggests evaluating suitability by looking at the scope, intended audience, and timeliness of the material. It then suggests ascertaining the work's authority by checking the author's credentials and determining what else he or she has written. Other things to look for include objectivity and evidence of scholarship as indicated by the inclusion of a bibliography or footnotes.

The issue of authority is important in evaluating all types of information, but becomes truly critical in a Web environment. The Berkeley site also includes some suggestions for applying the above criteria to content found on the Web.

Back to Top

What Makes Web Information Different?

In her workshop on the Electronic Library, Lisa Janicke of the Milner Library, University of Illinois at Urbana-Champaign (http://alexia.lis.uiuc.edu/~janicke/ElecLib.html) introduces her subject by discussing the lack of quality control over what is posted on the Web. She points out that accurate and reliable data may coexist with data that is inaccurate, unreliable or purposely false. Weeding out the bad from the good is the responsibility of the "savvy" user. In such an uncontrolled environment, users need to be "critically skilled consumers" of the information they find on the Web.

Donald T. Hawkins (1999), in "What is Credible Information?" InfoResources, Online, Inc. (http://www.onlinemag.net/OL1999/technomonitor9.html)makes the case for evaluating Web-based information by contrasting the checks for accuracy and reliability that attend the publication of a journal article or other printed documents with the immediacy and lack of filtering with which Web pages are produced. This article, which first appeared in the established print publication, Online, read by many librarians and information specialists, presents a set of Web site evaluation criteria and includes an extensive reading list of both print and Web-based materials.

When information is published in a printed media, it goes through an editorial process that acts as a filter that provides some assurance of accuracy and reliability. Different types of materials are subject to different levels of review and fact checking, but almost all have been through some type of validation process for compliance with editorial or publication standards before they are released. Information that appears on the Internet is not subject to any kind of review or checking. Since anyone can publish (post) on the Web, it is the responsibility of the user (you) to evaluate the information you find in this free-form, unfiltered environment.

Back to Top

Some Hints for Evaluating Web-based Information

Once you find information on the Web how do you know if the source is valid? What criteria should be used to determine whether the information is accurate and reliable? The sites described in this section and others in the Related Resources Section suggest ways to assess the worth of Web-based information. Several of these sites have been created by colleges and universities, and are therefore likely to be valid and unbiased sources of information.

"Evaluating Web Sites" (Copyright 1999), Leslie University, Cambridge, MA
(http://www.lesley.edu/library/guides/research/evaluating_web.html) offers useful evaluation criteria for Internet users. This site emphasizes the role the URL (Web address) plays in determining the purpose of a site and assessing its authority. A list of existing and new domain names is presented. Checking the URL with a clear understanding of the differences between, say, a dot-org and a dot-net, is a good rule to keep in mind as you as you evaluate Web-based information.

John R. Henderson also addresses evaluation criteria in "The ICYouSee Guide to Critical Thinking About What You See on the Web" (2000) at http://www.ithaca.edu/library/Training/hott.html:

The application of traditional and new evaluation criteria to Web resources is discussed by Janet E. Alexander and Marsha A. Tate in "Web Resource Evaluation Techniques" (1998), Wolfgram Memorial Library, Widener University, Chester, PA at http://www2.widener.edu/Wolfgram-Memorial-Library/webevaluation/webeval.htm. While acknowledging that reliance on Web-based information is becoming commonplace, this site warns that the quality of Web-based resources can vary within sites, as well as from site to site. The very nature of the Web with its use of hyperlinks, frames, search engines, and blending of entertainment, information and advertising, requires users to look carefully at the Web pages that they are accessing. Alexander and Tate recommend that users determine whether the pages fall into one of five categories:

"The Navigating the Net Forum: Evaluating Quality" site produced by Walt Howe at http://www.walthowe.com/navnet/quality.html provides more valuable advice about how to evaluate quality on the Web. Mr. Howe suggests you ask yourself the following questions in order to judge the quality of Web-based information:

Howe also suggests an alternative to the overwhelming amount of information that is sometimes retrieved using one of the Web search engines. Even though these search engines attempt to rank results by relevancy and bring the best hits to the top of the list, this does not always happen. Furthermore, no matter how good the search engines, they never index more than a fraction of the pages on the Web. To make searching for quality more productive, Mr. Howe suggests using the expert work done by others. Instead of searching for the basic information, search for an expert in the subject who has posted his or her own information or posted links to evaluated information.

Back to Top

Characteristics of a Quality Web Site

In addition to the accuracy of the information provided, there are a number of other factors that enter into the production of a quality Web site. A number of these considerations are pointed out by Paul Gorski of the Curry School of Education, University of Virginia, Charlottesville, VA, in "A Multicultural Model for Evaluating Educational Web Sites" (1999) at http://curry.edschool.virginia.edu/go/multicultural/net/comps/model.html. While Mr. Curry is primarily interested in helping teachers evaluate Web sites for their students, the checklist he provides applies to all sites and can therefore help end users in evaluating the quality of a site. He suggests a series of questions for evaluating a site in seven key areas:

Issues of accessibility and navigability are important non-traditional criteria for judging the quality of a Web site. Mr. Curry suggests you ask the following questions about accessibility:

He suggests asking these questions about navigability:

Carolyn Kotlas in "Evaluating Web Sites for Educational Uses: Bibliography and Checklist" (2000), UNC, Chapel Hill, NC at http://www.unc.edu/cit/guides/irg-49.html offers a checklist, with the last seven questions (in italics) applying uniquely to the Web:

Back to Top

Identifying Computer Hoaxes

About Hoaxes

As a knowledge worker in the networked information world, you may need to distinguish between what is fact and what is instead a hoax masquerading as fact. The definition of a hoax is a scam, myth or other fabrication designed to mislead. Ranging from merely annoying, to misleading and dangerous, hoaxes can be quickly and easily proliferated by electronic mail. As an information literate citizen you need to be able to practice critical thinking that will enable you to spot such chicanery, misinformation and lies.

A reason many people may fall for these hoaxes and other fabrications, according to Paul Gilster, author of Digital Literacy: Personal Preparation for the Internet Age (New York, Wiley, 1997), is the perception that computers don't make mistakes, coupled with the belief that the Internet, having been developed by the academic-scientific community under government auspices, must be a high-level information source. The result is that some people are predisposed to accept as true any information that appears on their computer screens.

Back to Top

How to Recognize a Hoax

Here are some clues that you may be dealing with a hoax, as provided by the Department of Energy's Computer Incident Advisory Capability (CIAC) at its HoaxBusters site (http://HoaxBusters.ciac.org/):

Deciding whether a warning is legitimate or a hoax requires you to evaluate the reasonableness of the claims and to verify that the person sending out the warning is a real person with the requisite expertise. While hoaxes may contain technical sounding language that make them sound legitimate, they often make fantastic claims. For example, they may claim that an attachment will damage your computer even if you do not open it. Authoritative sources for warnings about computer safety are your computer security administrator, recognized antivirus vendors, and computer security response teams, such as CIAC or the CERT Coordination Center at Carnegie Mellon University (http://www.cert.org). You can find the names and addresses of other response teams by connecting to the FIRST Web page (http://www.first.org). Warnings that are unsigned or that are by people whose names, addresses, and phone numbers cannot be verified are probably hoaxes.

Back to Top

What to Do When You Receive a Warning

Here are some on steps to follow, taken from the HoaxBuster site (http://HoaxBusters.ciac.org/), when you receive a virus alert or other warning.

The universal recommendation from computer security response teams, anti-virus vendors, and others is summarized in the advice that follows, from the Don't Spread That Hoax! site maintained by Charles Hymes (http://www.nonprofit.net/hoax/hoax_big.html):

Back to Top

Related Resources

Critical Thinking

Center for Critical Thinking
http://www.criticalthinking.org/
Articles, curriculum guidelines, and lesson plans.

CriticalThinking.Net
http://www.criticalthinking.net/index.html
Web site of the Illinois Critical Thinking Project, includes definitions and articles.

Critical Thinking on the Web
http://www.philosophy.unimelb.edu.au/reason/critical/
Links to references, articles, and related information that can found on the Web, hosted by the University of Melbourne, Melbourne, Australia.

Mission:Critical
http://www.sjsu.edu/depts/itl/index.html
Interactive tutorial on critical thinking, presented by San Jose State University in San Jose, CA.

Evaluating and Validating Information Sources

Beyond 'Cool': Analog Models for Reviewing Digital Resources
http://www.onlinemag.net/SeptOL/rettig9.html
Article by James Rettig (1996) from Online, 20 (6) (September/October): 52-54, 56, 58-62, 64.

Bibliography on Evaluating Internet Resources (1997 - last updated 2001)
http://www.lib.vt.edu/research/libinst/evalbiblio.html
Sources of information on evaluating online information compiled by Nicole Auer of Virginia Polytechnic Institute and State University, Blacksburg, VA.

Criteria for Evaluation of Internet Information Resources (1997)
http://www.vuw.ac.nz/~agsmith/evaln/index.htm
Checklist by Alastair Smith, Victoria University of Wellington Department of Library and Information Studies, New Zealand.

Evaluate Web Resources (1999 - updated 2000)
http://www.clubi.ie/websearch/resources/index.htm
Suggestions from Eddie Byrne for Web searching.

Evaluating Information Found on the Internet (1996 - last updated 2001)
http://www.library.jhu.edu/elp/useit/evaluate/index.html
Recommendations from Elizabeth E. Kirk, Milton College.

Evaluating Quality on the Net
http://www.hopetillman.com/findqual.html
A paper by Hope N. Tillman, Babson College, presented at Computers in Libraries, Hyatt Regency Crystal City, Arlington, Virginia, Monday, February 26, 1996, last updated 2001.

Evaluation of Information Sources (1997 - last updated 2001)
http://www.vuw.ac.nz/~agsmith/evaln/evaln.htm
Bibliography with links compiled by Alastair G. Smith, who teaches in the Library and Information Studies Group of the School of Communications and Information Management at Victoria University of Wellington, New Zealand.

A Student's Guide to Research with the WWW (1997 - updated 1998)
http://www.slu.edu/departments/english/research/
Tutorial for conducting Web research by the English Department of St. Louis University, St. Louis, MO, with links to search strategies, citation style guides, and search engines.

Teaching Undergrads Web Evaluation: A Guide for Library Instruction
http://144.16.72.189/is213/213-2000-2001/webeval/undwebev.html
Article by Jim Kapoun from College & Research Libraries News 59 (7), July/August 1998, pp. 522-3.

Testing the Surf: Criteria for Evaluating Internet Information Resources
http://info.lib.uh.edu/pr/v8/n3/smit8n3.html
Article by Alastair G. Smith published in the online journal Public-Access Computer Systems Review 8 (3), 1997.

Thinking Critically About World Wide Web Resources (1997, updated 2000)
http://www.library.ucla.edu/libraries/college/help/critical/discipline.htm
Evaluation techniques from Esther Grassian of the University of California, Los Angeles, CA.

Untangling the Web: Critical Thinking in an Online World (1996)
http://www.library.ucsb.edu/untangle/jones.html
Ideas for applying critical thinking to the Web from Debra Jones of Cabrillo College, Aptos, CA.

Identifying Computer Hoaxes

Current Internet Hoaxes, Urban Legends, Rumors, and Other Digital Lies
http://urbanlegends.about.com/science/urbanlegends/cs/nethoaxes/index.htm
Collection of Web resources on hoaxes, urban legends, rumors, and lies, compiled by David Emery for About.com, Inc.

F-Secure: Security Information Center
http://www.datafellows.com/virus-info/
Information on new viruses, hoaxes, and false alerts from F-Secure, a leading provider of data security solutions.

Hoax Pages from ICSAlabs, a division of TruSecure Corporation
http://www.icsalabs.com/html/communities/antivirus/hoaxes.shtml
Hot Hoaxes, Current Hoaxes, and a Hoax Archive from a provider of security assurance services for Internet-connected companies.

Hoaxes (Symantec)
http://www.symantec.com/avcenter/hoax.html
Descriptions of "viruses" that are actually hoaxes uncovered by Symantec, a world leader in Internet security technology.

Jokes (Symantec)
http://www.symantec.com/avcenter/jokes.html
Listing of "joke programs" spread via the Internet, which, while they can be annoying, will do no lasting harm.

Trend Micro Hoax Encyclopedia
http://www.antivirus.com/vinfo/hoaxes/hoax.asp
Listing of hoaxes and advice on what to if you receive one.

Virus Hoaxes (McAfee)
http://vil.mcafee.com/hoax.asp
Information on hoaxes as well as actual viruses with listings of products to help protect your computer from a leading provider of security software.

Virus Info (Sophos)
http://www.us.sophos.com/virusinfo/
Lists and descriptions of viruses, hoaxes, scares, and chain letters from one of the largest anti-virus vendors in the world.

Vmyths
http://www.vmyths.com/
Virus hoaxes from A to Z and more from this independent site not affiliated in any way with antivirus providers.

Back to Top
Proceed to "Exploring Online: Using Information"

DOD Hyperlink Disclaimer: The appearance of hyperlinks does not constitute endorsements by the Department of the Navy of the website or the information, products or services contained therein. For other than authorized activities such as military exchanges and Morale, Welfare, and recreational sites, the Department of the Navy does not exercise any editorial control over the information you may find in these locations. Such links are provided consistent with the stated purpose of this CD-ROM.