Have reference sources eroded in quality in the online era? We're all aware of the challenges facing us and our users in vetting (or not) the authority and credibility of free sources available online. But the quality of contemporary vetted sources is worth examining in its own right.

Editors of the Guide to Reference (guidetoreference.org) maintain a watch on reference sources to evaluate them for inclusion in the Guide. We also monitor changes in the reference publishing landscape and reference source quality in general. We define quality by applying the Rettig/LaGuardia Review Canon from the July/August 1999 issue of Online (James Rettig and Cheryl LaGuardia, "Beyond 'Beyond Cool,'" ONLINE, Vol.23, No. 4, July/August 1999, pp. 51–55) among other criteria. The canon includes evaluating parentage and provenance, authority, audience, content, creation and currency, design, usability, and medium. In writing this article, I would like to offer special thanks for their assistance and contributions to my fellow division editors: Steven W. Sowards, socials sciences, Michigan State University; Sarah G. Wenzel, humanities, the University of Chicago; and Jo Bell Whitlatch, general reference sources, faculty emeritus, San Jose State University.

Issues With Authority

In online products, the author or authority is not always evident. The largest target in the category of "reference source with no authority stated" is, of course, Wikipedia. But users face a more subtle challenge in distinguishing objective sites from those that are merely advertising but masquerading as reference-like sources. On WebMD (webmd.com), the differences between the markings of vetted content and advertisements are extremely subtle. Any designation can help savvy users but they are not always indicated and may be deliberately obscured. Domain extensions (.com, .edu, .gov) are not a reliable indicator of quality or authority either.

Many librarians continue to value traditional authorities, even if users do not. An answer may be found quickly via one's favorite web search engine, but awareness of vetted sources may lead to quick answers as well. Choosing a vetted source first in looking for the answer can offset or reduce time spent carefully evaluating candidate retrieved websites.

In the social media age and the post-modern age, authority is a contested and often despised concept. Yet it is a cornerstone for library collections in general and for reference tools in particular, which are chosen to give the best, most accurate answers. Users may be more inclined to value the opinions of nonauthoritative sources for consumer-related and entertainment topics and social issues. However, fact-related questions such as the load-bearing capacity of a concrete floor require a precise and accurate answer. Accurate and authoritative sources are likely to always be welcomed in library collections.

Not Cool, Publishers

Online sources do not always include all the content of the printed versions. For example, the Oxford Reference Online platform doesn't contain the important introductory essay from the Oxford Dictionary of Quotations. Only the quotations are retrievable. Loss of that front and back matter is a major concern to librarians, many of whom choose to retain old print copies and hesitate to license online products that may not contain 100% of the product. Aggregator services are notorious for omitting some material, such as the college rankings articles from U.S. News & World Report/U.S. News Digital Weekly. Ebsco provides only teaser articles listing the top 10 picks, and ProQuest links out to the usnews.com website where complete data are only available via College/Grad Compass, a fee-based service.

In the early days, some sources were slapped on the web without taking advantage of the medium. Many reduced their value because straight-scanned pages only added keyword searchability while losing the intelligent human indexing, cross-referencing, and creation of tables of contents, for example, that provided such a great added value in the print format. Most ebooks are still PDFs with decent OCR indexing. Many recently published works support hyperlinking and added features, but items scanned in the early days may never be upgraded. Quality of scanning is another issue: De Gruyter (Reference Global, degruyter.com) commands high pricing for poorly scanned print.

Eyeballing—quickly scanning through the entire work—is still valued by many librarians and patrons and is poorly supported on platforms that do not include speedy thumbnailing functions. The "browse" function does not substitute for scannability, and scrolling up and down doesn't substitute for scrolling sideways. Eyeballing also provides context, an ability to view neighboring data, which can be lost when basic searching retrieves only isolated data.

Some publishers currently meld ebooks, including reference books and journals in the same platform, usually to the detriment of all types. Examples include Gale's Artemis (gdc.gale.com/gale-artemis) and Springer (link.springer.com) (Figure 1). Stripping of content from a container can result in a loss of context, either of topic or format type or audience. A single platform also creates larger haystacks, with results resembling those of flawed discovery systems.

Oxford University Press (OUP) currently has chosen a different direction, creating seven different platforms or slightly fewer if only counting reference works: Oxfordhandbooks.com, Grove, ODNB, oxfordbibliographies.com, oxfordreference.com. Multiple platforms per publisher may better serve the unique needs of each type of reference tool. But the platforms are difficult to manage, to juggle, and to educate users about. OUP is trying to solve the juggling and pointing problems through the Oxford Index (oxfordindex.oup.com) (Figure 2), a free discovery tool across Oxford content and all its platforms, which still fails to solve the needle-in-the-haystack issue. Complete cross-platform indexing, advanced searching, and direct linking functionalities are critical for navigation and ease of use. Oh, for the perfect solution!

An annoying but easily solvable problem is that of dueling platforms. For several years, Elsevier offered its customers a choice between having reference books on its Referex or its ScienceDirect platforms, but it would not permit access to a title on both. Mercifully, it has now abandoned the Referex platform and switched those customers to ScienceDirect. Wiley-Blackwell still offers some encyclopedias as scanned-PDF ebooks or on the database platform.

The migration to online reference works has given publishers an opportunity to repackage their offerings, to the frustration of many budget managers. Options may include both subscription and ownership models. Purchases may require an annual maintenance fee in addition to the one-time allocation, and the annual fee may or may not include updates to content. Aggregator platforms are an additional source of confusion, not the least of which is figuring out if a library is paying twice (or more) for the same content available on both a publisher and an aggregator platform or via multiple aggregators. What is a library to do when a valued publisher pulls out of an aggregator but the library remains committed to the aggregator? McGraw Hill learned the hard way that aggregator loyalty can trump individual titles, especially when the publisher only offers titles in inseparable packages. Cambridge Handbooks Online and Oxford Handbooks Online are also bundled. Unaffordable pricing models result in sources not being available and therefore not used by patrons. High-quality but unaffordable sources are as useless as junk sources.

When a library does not license all of a publisher's content, users can feel teased by retrieving results to inaccessible full text. Many platforms offer a subtle button to toggle between "view all results" and "view your/licensed results." On occasion, a library (or a user) can set this default, which is extremely handy for libraries that license very much or very little of a platform's offerings. Exposing users to content they cannot immediately obtain is a delicate choice best left to individual libraries.

To add insult to injury, some publishers still have time lag between the print publication, which appears first, and the online version. This situation is highly likely to cause duplicate purchasing or patron frustration if the library cannot afford both formats. A library purchases the print when available and needed and then purchases the online version later for accessibility or because the title is included in an online bundle.

When use of reference sources requires a personal login, or a website harvests personal information, the simple act of seeking authoritative reference information can expose readers to invasions of privacy. No one wants to consult a reference site, such as WebMD, about a medical condition only to find advertisements related to some illness pop up during other online sessions. This kind of secret surveillance in the name of revenue will alienate users from reference products and services, and it violates basic principles of librarianship in protecting our users' right to privacy and confidentiality with respect to information sought.