SUBSCRIBE NOW!

Vol. 12 No. 7 — July/August 2004

The one thing we have not done so far is to step back and take a look at the big picture — to see where we have been, where we might be going, and to what extent we have accomplished the purposes we set for ourselves when we began this brave new way of doing reference not so very long ago.

Until recently, seeing the "big picture" would have been difficult or even impossible. We just didn't have enough data. But all the articles and reports and tales of individual experience have added up. Now we can see the big picture emerging, while still recognizing that there are lots of missing pieces, things we don't know, and things we could misinterpre t because we don't have all the facts. But we have enough data now at least to open a discussion about where we've been and what we've accomplished over the past few years.

From the Beginning

Even back at the turn of the century (1999 or thereabouts), we knew that traditional library reference was in trouble following the Web's appearance in the early '90s. More and more people were going to the Net to get their information, and our statistics showed fewer and fewer turning up at library reference desks. In fact if you take a look at reference statistics from ARL libraries, you can actually see the "tipping point" when things started to go downhill. ARL has tracked reference statistics since 1995. From 1995 through 1997, the median number of reference questions gradually increased — from 156,414 in 1995 to 162,336 in 1997. However, beginning in 1998, those reference stats began a precipitous decline that continued unabated through 2003 — the latest date for which figures are available. Between 1997 and 2003, median ARL reference statistics dropped from a 1997 peak of 162,336 to a 2003 low of 96,228 — a loss of over 40 percent in the span of 6 years. Figures for public libraries are less definitive and less current, but anecdotal reports indicate that many publics have experienced the same problems. (See the chart below.)

The ARL reference statistics only cover direct transactions between clients and reference librarians, not inquiries answered by library Web sites. These days, patrons who once would have called a librarian to check on a citation or look up an item in the library catalog are probably using the library's Web site to get their answers. Well-developed library sites will have also stocked Web links to answers for frequently asked questions. Such permanent solutions to ongoing problems reflect credit upon the library's performance, even though statistics on library operations may not incorporate them.

Five years ago, other Web developments seemed to threaten library reference services and information brokers. Dozens of new commercial reference services, such as Ask Jeeves and WebHelp, seemed to be popping up everywhere online and appeared wildly popular. Ask Jeeves claimed to handle over 2,000,000 questions per day and to grow at the rate of 46 percent per quarter. WebHelp — a new "real-time" information service offering free, live reference assistance on the Web — was exploding. Although it had just gotten off the ground in November 1999, in its first month, WebHelp logged more than 2.1 million visitors served by a staff of over 900 "Web Wizards." Kerry Adler, WebHelp's CEO, estimated that if things kept growing at that rate, WebHelp would employ over 20,000 "Web Wizards" by the end of its first year in business.

Libraries didn't exactly stand idly by through all of this. By then most of us were offering some sort of e-mail or Web-forms-based reference to try to reach our errant patrons on the Web. But our efforts seemed to meet with little success. E-mail questions were trickling in at the rate of a few a day, while questions at our regular desks were declining at a rate of tens of thousands per year. Not only that, e-mail reference was often a cumbersome process. It lacked the immediacy so many people seemed to seek from the Web. The reference "interview" sometimes required a good deal of back and forth with the patron before the librarian understood the question. Sometimes there would be long lag times between responses; sometimes patrons would simply stop responding at all; and sometimes a librarian would sweat it out getting just the right answer to a question, then fire it off to the patron, only to hear back that the information was no longer needed, because the patron had "broadcast" the question to several libraries and already gotten a response from somebody else.

By the late 1990s, it had become apparent to many that if librarians were ever to successfully move their reference services to the Web, we needed a different solution. Something live, interactive, and real time. Something that allowed us to work with patrons to help them find the information they wanted right when they sought it — not days later in an e-mail. Something that allowed us to provide patrons with a reference service so immediate, so convenient, so ubiquitous, so easy to use, and so 'in your face' (as Anne Lipow famously described it), that patrons would have no choice but to turn to us whenever they needed help on the Web. In fact, they might have a tough time trying to get away.

And for a while there, it looked like we had found just such a solution in the guise of a set of Web-based software applications that allowed people to communicate and work together online, live, and in real time. All kinds of software programs exist that allow people to talk with one another and work together on the Web — ranging from free or low-cost chat and instant-messaging programs in which people type messages back and forth, right up to the most elaborate applications that include features like voiceover IP (you can talk on your computer as if it were a telephone), and remote control (you can take over another person's computer and operate it remotely). Over the past few years, libraries have experimented with many of these programs, but the one type of application that seemed to really catch on for reference applications is something called "Web contact center software" or sometimes "Web collaboration software."

Originally, these applications were designed for companies that wanted to provide live customer service on their Web sites. At a minimum, these programs normally include a chat function of some kind so the CSR (customer service rep) and the customer can communicate with each other while online and some sort of "page-pushing" or Web "co-browsing" capabilities that allow the CSR to send the customer relevant Web pages or even to "escort" them through various sites on the Web. These programs can come with all kinds of other bells and whistles — but chat and Web interaction features, either page-pushing or escorting, were what made them so appealing for doing live online reference.

Although we probably did not realize it at the time, this software was designed to be used in commercial call centers where a number of agents are logged on and handling calls at the same time. Most Web contact center software supports multiple, simultaneously online agents and some sort of routing system that transfers calls to the next available agent. Since all this happens over the Internet, all the agents do not need to be in the same room, as in the case of telephone call centers. With Web contact center software, agents can log on to a system from any computer with Internet access anywhere in the world and the software will route calls to them just as if they sat in the same room. Although this may seem like a small detail, this structural feature has made possible many of the collaborative arrangements that have grown up around virtual reference in the past few years.

In the late '90s, these Web contact center applications seemed well on their way to adoption by major Web retailers and other commercial sites. Lands End and LL Bean both added live online customer service for the Christmas season in 1998, as had a number of major banks and ISPs and the technical support departments of many computer hardware and software manufacturers. Lots of software companies sold products for the Web contact center market, including some well-known names like Cisco and Lucent, and many not so well known — eGain, Netagent, LivePerson, HumanClick, WebAgent, Webline, and others. Many of these firms had high-flying stock prices and were well financed by heady IPOs or heady venture capitalists. WebHelp, an operation that claimed to handle hundreds of thousands of questions each day, had adopted the eGain Web contact center software for their live reference service. So eGain software supported an operation answering questions many of us felt would have once gone to our reference desks.

Of course, it was no great leap to try to adapt this software for the library market and serve our own patrons live and in real time on the Web. What we would do if we actually started getting those hundreds of thousands of questions going to WebHelp and others remained a concern. But we figured that we'd cross that bridge when we came to it — and at least it meant we would be addressing the "right kind" of problem, a New Information World Order problem. LSSI was first to market this new software to the library world when it introduced its Virtual Reference Desk product at the ALA Annual Conference in Chicago in 2000. Other vendors, including 24/7 Reference, Convey Systems, OCLC/Library of Congress' QuestionPoint, and Docutek, followed shortly thereafter. A virtual reference feeding frenzy gripped the profession.

What a Difference a Few Years Make

A lot has changed since those heady early days of the virtual reference "movement" and not always in ways we had expected. Many of the commercial reference services that had us so worried a few years back have either died, gone into other lines of business, or are so gravely wounded as to no longer constitute a threat to anybody.

Ask Jeeves has dropped its live question-answering service, Answer Point, and now focuses on its search technology. In fact, if you type Answerpoint into your browser today you'll either get a "self-help answering community" (don't ask) on msn.groups or the reference service of the Central Rappahannock Regional Library.

WebHelp, the "human search service" that constituted perhaps the most direct competition to library reference services (touting its Web Wizards as the "librarians of cyberspace"), has transformed itself into the "global leader in offshore business process outsourcing." Instead of answering reference questions for free, those WebWizards now handle telephone sales, technical support, warranty management, and other customer service functions for the likes of AOL, Nordic Trak, Norelco, Netscape, Dell, and ClubMed.

WebHelp has also significantly expanded the number of communications channels it supports. The original reference service was limited to chat and Web collaboration based on the eGain platform that many libraries now use. Today, customers can contact WebHelp any way they like, including phone, instant messaging, e-mail, Web self-service, as well as the original chat and Web collaboration technologies. WebHelp still offers its "Live Search Service," at least in theory, but it now costs $9.95 per month for unlimited searching or $9.95 for 10 searches. From all indications, it gets very little use. A search of its question archives showed no references to 9/11, the war in Iraq, or Howard Dean. The service that predicted it would employ 20,000 staff within the first year of operations reportedly now employs less than 200.

Many others have suffered similar fates. Askme.com has gone from a free reference service to "providing the software that manages employee knowledge networks." Allexperts and ExpertCentral have both merged into About.com and continue to operate as a volunteer answer service, but, judging from its archives, at a pretty anemic level. For example, the archive shows only 13 questions on AIDS during all of 2003, and, despite lots of questions on Windows 95, 98, and 2000, nothing on Windows XP — a clear sign that the site no longer gets the traffic it once did. LiveAdvice and Inforocket, two early fee-based online reference services, have both merged into Keen.com and Keen.com has, in turn, morphed into a psychic hotline that offers all kinds of advice online — but none of the kind librarians give. In any case, Keen.com no longer uses chat, it uses the telephone.

At first glance, the one exception to the downward slide in commercial reference services over the past few years appears to be Google Answers. This fee-based reference service lets questioners set the amount they are willing to pay for an answer (ranging from $2.50 to $200, but most running $20 or less). A crew of about 500 of Google's freelance researchers choose the questions they want to answer. Unlike many other commercial reference services, Google has paid attention to quality control with its service. Researchers are carefully screened (at least in comparison with the laissez-faire attitude of other services); there are detailed guidelines for answers; and Google completely guarantees the customer's satisfaction. If you are not happy with the answer, you don't pay. Google Answers debuted in Spring 2002, striking fear and consternation into the hearts of librarians everywhere, who feared that they would now have to face not a run-of-the-mill Internet start-up, but a truly formidable competitor.

However, it now appears that we needn't have worried. Despite the fact that Google is one of the most heavily visited sites on the Web, despite the fact that The Wall Street Journal ranked Google Answers as the best reference service on the Web, better than five competing services including the Library of Congress' QuestionPoint service (though QuestionPoint was only just out when the review appeared), despite the fact that most questions are answered pretty thoroughly and economically — despite all that — Google Answers does not seem to be going anywhere. As of January 2004, it averaged about 60-70 questions per day, down over 50 percent from the 200+ questions per day the service got in the late Spring of 2002, just after it opened. Even using the higher usage figure, you're still only talking about 0.0001 percent of the 200 million searches done every day on the regular Google site.

Nor has Google found it necessary to recruit new researchers. The Google Answers FAQ states that it is not currently accepting applications for new researchers, a statement in place since shortly after the service opened. Yet we know that attrition must have reduced the original crew of 500 quite a bit by this point, and several have been very publicly fired or quit. Searcher magazine has already published the departure tale of one Google Answers' researcher — Jessamyn West (see https://www.infotoday.com/searcher/oct02/west.htm).

More important, the underlying chat technology that powered many live commercial reference services has also failed to find broad acceptance on the Web. The chat and Web collaboration tools that WebHelp, librarians, and others have adapted for online reference were originally designed to help e-commerce companies provide live sales assistance and customer support over the Web. However, evidence shows that, except for a few high-profile retailers such as LandsEnd and L.L.Bean, most companies have been reluctant to adopt it. According to a survey by Benchmark Portal in 2003, only 12 percent of the respondents (generally large corporations with large customer service call centers) offered a 'chat' option on their Web sites, while 88 percent did not ... and when those 88 percent were asked if they intended to implement chat, 99.5 percent responded that they had no plans to do so within the near future [http://www.benchmarkportal.com/
newsite/article_detail.taf?topicid=265].

These companies have good reason to be skeptical about chat communications. First, the general public has yet to accept chat as a means of communications for business dealings and other more formal transactions. We all know that people — and particularly children — are purported to chat back and forth and instant message friends on a nearly continual basis, but it appears that when money or more serious matters are involved, people still prefer to pick up a phone. Secondly, chat "transactions" appear to take at least twice as long as and cost more than the same transaction over the phone. Chat was originally developed in the days of dial-up Internet to give people a way to communicate live online while their phone lines were tied up with the computer. Today, much of that rationale has disappeared as the population moves to cable, DSL, and other forms of broadband that support simultaneous use of voice telephones.

Collectively, these trends have devastated technology companies such as eGain, LivePerson, Kana, WebLine, LiveAssistance, HumanClick, Convey Systems, and others that had hoped to provide tools to help companies do live customer service over the Web. Like commercial reference services, their stock prices too were flying high back in the late '90s during the dot-com boom, but today, many have gone bankrupt or been acquired for pennies on the dollar. Others are in pretty serious trouble. Meanwhile, chat and online Web collaboration remain very much a niche market.

It is difficult for us librarians not to feel a certain sense of smugness watching the demise of those who had sought to replace us on the Web; however, we have serious cause for concern as well. After all, if these commercial reference services armed with venture capital money and marketing clout could not make a go of it online, what does that say for our chances? And if people have so far proved largely unwilling to embrace online chat for banking, insurance, retail, and other sales and customer service applications, what makes us so sure they will embrace it for reference?

So What Happened to Libraries?

Ironically, as commercial reference services closed up, consolidated, went bankrupt, and otherwise dropped off the Web, libraries began scrambling to get on board. In the past few years, libraries have opened up hundreds, perhaps even thousands, of new live online reference services. In 1999, you could count the number of libraries offering live "virtual" reference services over the Web on the fingers of one hand. Today, nobody is really sure exactly how many services there are. Stephen Francoeur and Gerry McKiernan have each tried to maintain a census of virtual reference services; as of February 2004, Francoeur listed approximately 500 services, while McKiernan linked to 132. These registries are valuable sources of information, but tend to underestimate the true numbers of libraries involved — both because keeping up with all the libraries opening new services is tough, and also because many listings actually identify collaborative reference services that may include dozens and sometimes hundreds of individual libraries. Bernie Sloan has tried to track the number of live collaborative services alone; as of November 2003, he listed approximately 1,730 libraries in a total of 62 collaborative services.

Vendors who sell the various virtual reference software provide another source of information. One must use caution with these numbers as well, because vendors always want to show as many customers as possible, so their numbers may need some "interpretation." However, as of January 2004, OCLC claimed its QuestionPoint service was in use in over 1,000 libraries, and Tutor.com (formerly the Reference Division of LSSI) and 24/7 Reference could also claim about 1,000 or so each, especially if you consider the various state-wide projects in which they are involved. Add to that the libraries "powered by" Docutek, LiveAssistance, LivePerson, and the various free options, including Rakim and AOL, Yahoo!, and MSN instant messaging programs, and there could easily be 3,000-4,000 libraries currently offering live online reference assistance using some form of chat and/or Web collaboration software.

But numbers don't tell the whole story, because librarians have also employed the capabilities of the new technology to experiment with all kinds of new and innovative ways to do reference. We've developed collaborative reference services, 24x7 services, and tiered services that offer live access to subject specialists in legal, medical, and other subject areas when the questions warrant. We've experimented with having academic and public libraries work together to answer each others' questions. We've formed international networks where libraries in different parts of the world take turns staffing the service during different parts of the day to provide 24-hour coverage. We've offered Spanish and other foreign-language reference services impossible otherwise. We have tried outsourcing reference services to commercial providers during overnight and weekend hours so the regular staff could get some sleep. We have tested all sorts of other new and novel approaches to doing reference that would have been inconceivable from a traditional reference desk.

And we've spent lots of money. We've bought millions of dollars of software, shelled out thousands more on revamping Web sites and adding authentication software, and spent millions on staff time in development and committee meetings. Nor do these estimates include the training and salary costs of the thousands of librarians who have staffed these systems. Just as no official figures exist for the numbers of libraries that have opened up live online reference services, no official accounting records exist that track all the money spent doing it — but there is no doubt that the bill now runs into the tens of millions of dollars. That sum is even more amazing when you consider that much of this spending took place during one of the most severe funding crises libraries have ever experienced. In fact, in more than a few cases, librarians started up virtual reference services with one hand while slashing book budgets, canceling serial titles, and reducing hours with the other.

One should note that much of the initial funding for many virtual reference projects came from grant monies, sparing library budgets much of the direct initial impact. In the U.S., Library Services and Technology Act (LSTA) grants have provided the bulk of the funding for virtual reference. For example, the 24/7 Reference Project has received several million dollars worth of LSTA money in California alone. Press releases, project proposals, committee minutes, presentations, etc., detail virtual reference projects in dozens of states and hundreds of libraries — all funded, in whole or in part, by LSTA grants. In fact, so much LSTA money has gone into virtual reference projects that at one recent reference symposium, some wag suggested we should change its name to VSTA, for the Virtual Services and Technology Act.

All that grant funding helped libraries get things off the ground quickly, but it comes at a cost. LSTA grants do not last forever. These grants are intended to provide start-up money and funding for pilot projects, but now that many of the initial grants have begun to run out, many libraries will soon face the challenge of figuring out how to pay for a brand-new — and sometimes very expensive — virtual reference service from already overstretched budgets.

But we'll get to those issues in a bit; for right now, it is important to recognize how remarkably successful libraries have been in moving reference services to the Web. In a little less than 4 years, thousands of librarians got out from behind the desk, opened up their shops on the Internet, and made ready to answer patron questions — live and in real time — anytime they were asked.

Where Did All the Patrons Go?

Unfortunately, few patrons have been asking. Like so much else about this field, it is difficult to get good virtual reference usage statistics. These statistics are not regularly separately reported to NCES (National Center for Education Statistics), PLA, ACRL, state libraries, or any other agencies charged with gathering library statistics. Many services have been reticent to share data publicly, which — in and of itself — may indicate a sensitive issue. Probably the most comprehensive survey to date is the Global Census of Digital Reference [http://www.vrd2003.org/proceedings/
presentation.cfm?PID=196] that Joe Janes initiated last November. Janes asked libraries offering any kind of digital reference service (chat, IM, e-mail, Web form, etc.) to report the numbers of questions received during three 'typical' days in November 2003: Monday, 11/3, Thursday, 11/6, and Sunday, 11/9. Janes got responses from 162 services — some individual libraries, others collaborative services representing many libraries. Overall, these 162 services had answered 8,106 questions over the 3-day period, and 5,657 of those questions (or almost 70 percent) were answered live, using chat technology.

In the aggregate, those would seem pretty impressive figures. But when you look at them on a service-by-service basis, the figures tell a different story. Janes found that the median service in the census had answered just 16 questions over the 3-day period — and that the median number of questions answered per day was just a little less than six. These are not the kind of statistics you'd want to run and show your funders, nor the sorts of numbers we might have expected given all the time, money, and effort put into these services. To be fair, although the medians in Janes' census were very low, some services did report much better numbers: the top service reported 765 questions and others reported 630, 440, and 410 questions over the 3-day period. But the problem is, we don't know whether these figures come from individual libraries — in which case, they'd be pretty impressive — or from collaborative services representing dozens, maybe even hundreds, of libraries.

The Association of Research Libraries (ARL) conducted a smaller census of virtual reference services in ARL libraries in October 2002 (ARL Spec Kit 273 — summary at http://www.arl.org/spec/273sum.html). ARL found that 36 of the 124 ARL libraries (over 25 percent) had instituted chat reference service within the past couple of years. However, as Janes discovered, most were seeing very little usage. The University of Florida reported an average of 14 questions per day, although its service had been in existence for over 2 years at that point. George Washington University averaged nine questions per day; the University of Minnesota, five; Louisiana State, three; MIT, UC Santa Barbara, and Syracuse University, just two questions per day; and UC Davis and Michigan State were getting an average of just one question per day. The University of Illinois at Urbana-Champaign (UIUC) was the exception to the rule, averaging 41 questions per day in October 2002. In the few years of the service's existence, it had handled 7,586 questions — no small feat, especially in comparison with the struggles others were having attracting patrons. However, more recently, UIUC's statistics have dropped off rather sharply; some attribute this decline to problems with the virtual reference software being used, others think it may have more to do with changes in patron preferences.

Finally, we conducted an e-mail survey of chat reference services in October 2003. Services were asked to compare the number of chat questions received in October 2003 with the number received in October 2002. Although most libraries responding to the survey reported similar statistics to those found in the Janes and ARL surveys, some did not appear to do quite as well. In particular, a number had seen the amount of questions received in 2003 actually decline from the figures seen in 2002. For example, the St. Charles Public Library in St. Charles, Illinois, got 18 questions in October 2003, down from 32 in October 2002. The Ask the Librarians Live Coalition of Dakota State University, Northern State University, and South Dakota State University handled a total of 32 questions in October 2003, down from 36 in October 2002. Georgia Tech reported three questions in October 2003, down from 27 in the same period in 2002, a decline attributed in part to technical problems with its chat software. The CTW Library Consortium composed of Wesleyan, Connecticut College, Smith, Vassar, and Wellesley collectively reported 142 questions in October 2003, down from 258 in October 2002, but this was attributed to reduced hours.

Others showed some improvement. UC Irvine, for example, reported 115 questions in October 2002, but 141 questions in October 2003, very close to the usage UCLA got in the same period. Others seemed to do even better. York University reported 761 questions in October 2003, up from 298 in 2002 — but even at that rate, virtual reference accounts for less than 6 percent of the 155,000 or so questions that York libraries handle with more traditional means. Overall, though, these are not exactly bragging numbers, nor the kind many originally expected to see given the ready accessibility and convenience of chat reference and the stories heard about the initial popularity of many of the commercial reference services like WebHelp or Ask Jeeves.

While giving us an overall picture of how much virtual reference is being— or not being — used, these surveys don't tell us a lot about the services the statistics represent: how many libraries are involved, how long they've been around, what size population they serve, and how their virtual reference statistics compare with the numbers received through more traditional reference services.

So let's take a closer look at some real-life examples. UCLA was one of the earliest academic libraries to adopt virtual reference. Their AskALibrarian service first opened in Spring 2001 and has run more or less continuously ever since. According to statistics on the UCLA Web site, during the past 2-1/2 years (through the end of Fall Quarter 2003), it had answered a grand total of 2,583 reference questions via chat, or an average of 253 questions per quarter (median = 247), which equals about five questions per day. This is very close to the median figure of six questions per day that Janes found in his census.

Nevertheless, the virtual reference traffic at UCLA has grown over the years; in the Fall Quarter of 2003, it answered 584 questions, or just a little over one question per hour for every hour the service was open. However, that's still less than 2 percent of the total reference questions at UCLA and, even at that rate, chat reference will do very little to make up the losses UCLA and other academic libraries are experiencing in demand for traditional reference services. According to ARL statistics, reference questions at UCLA declined from a peak of 564,973 in 1995 to 198,597 in 2002. That's a loss of more than 366,000 questions or nearly 65 percent of reference traffic over a space of just 7 years. It will take a lot more than 584 chat questions per quarter to have much of an effect on that kind of deficit.

One of the busiest academic library services in the world is Ask A Librarian at North Carolina State University. Like UCLA, NCSU was an early adopter of virtual reference; it first began operations in January 2001 and, although serving a student body only half the size of UCLA's, NCSU's virtual reference statistics have been much higher from the beginning. The service answered 1,995 questions via chat in 2001, 4,119 in 2002, and 4,152 in 2003. That's an average of a little over 11 questions per day 365 days a year. Interestingly, NCSU operates its service 83 hours per week, or around 11 hours per day, whereas UCLA currently operates only 46 hours per week. However, during the hours they are open, both services average one question per hour, suggesting that the two services may be a bit more alike than would appear at first glance. It also suggests that the primary reason NCSU consistently has more questions stems from their running twice as many hours. Still, like UCLA, virtual reference only accounts for a small percentage of the total reference activity at NCSU — 6.4 percent of the 64,620 questions handled last year. And, like academic libraries everywhere, the few questions NCSU picks up from virtual reference does very little to help make up for the 62,000 (down 51 percent) reference questions per year lost from the peak reference statistics of 1996.

Not far down the street from UCLA is the Santa Monica Public Library. Santa Monica serves a population of about 100,000 people in one of the most "wired" communities in the U.S. Santa Monica was another early pioneer in virtual reference. It opened its chat service in July 2000 and has operated it continuously since that time — usually 24 hours a day, 7 days a week — and is available to anybody in the world. Yet despite the hours and availability, and despite the fact that Santa Monica runs a very busy and highly regarded traditional reference service, virtual reference services at Santa Monica Library have never really taken off. The library has averaged only 100 chat questions per month during the 3 years the service has operated. That's less than three questions per day and less than 3/10 of 1 percent of the nearly 40,000 reference questions Santa Monica Public Library answers in a typical month. Even more disturbing, unlike UCLA, chat reference at Santa Monica has been relatively static over the past 3 years. As you can see in the chart above, 2001 — its first year — saw the highest use, with usage declining slightly ever since.

The April 2003 figures for Santa Monica also show an interesting anomaly. During that month, the Main Library moved to a new temporary location, which left the physical reference desk closed down for a period of time and explains the 12,000 or so decline in normal desk reference statistics. During that period, phone reference increased tremendously, growing by over 4,000 questions and raising the percentage of questions asked from 42 percent to over 68 percent, but chat reference increased by only 25 questions during the same period. All of which suggests that when people are forced to use remote reference, they prefer to pick up the phone, rather than open a chat session.

However, a few very busy virtual reference services get thousands of questions each month. The AskUsNow service in Maryland handled over 2,900 questions in October 2003, for example. Australia's AskNow service reported 3,196 questions during the same period. And the QandA NJ service in New Jersey answered 5,800 questions that month. But there's a little catch: All these services come from large consortia made up of many libraries. Individual libraries only receive a small number of questions. Examined individually, most consortia members report numbers very similar to the services we've looked at so far. For example, in November 2003, California's 24/7 statewide reference service reported a total of 3,024 completed questions. However, that consortium comprises over 90 libraries. The largest of these libraries, Los Angeles Public, registered only 599 questions, even though it serves a population of over 3.6 million; Long Beach had 59; Pasadena, 51; San Jose, 64; San Diego, 28; most had far fewer. While the consortia model for doing virtual reference can certainly help reduce costs and share the burdens of operating such services, it appears to have the same problem with low usage that seems to affect many individual libraries.

At least one exception remains to these generally lackluster figures, namely the KnowItNow service in the Cleveland, Ohio, area. KnowItNow comes from a small consortium called Clevnet consisting of libraries primarily serving Cleveland and surrounding cities and a total patron population of less than 1 million. Yet, KnowItNow consistently averages 3,500 to 3,600 questions per month — more than Maryland, California, Australia, or many other consortia serving much larger populations. So far as we can determine, KnowItNow is the busiest public library virtual reference service per capita anywhere in the world, and it is difficult to figure out exactly why. It did get some lucky breaks at the outset, with coverage in the local newspaper and TV, followed by the story reaching both CNN and NPR. KnowItNow has been consistently well managed and well marketed from the beginning and has paid special attention to the needs of school children, who make up a large proportion of the users of chat reference in most public libraries.

It could be any of these factors or perhaps others we haven't considered. One thing is for sure: KnowItNow has set a standard for usage that few virtual reference services can match. However, even with that, the 3,500-3,600 questions per month KnowItNow answers only represent a tiny fraction of the total reference handled by Clevnet libraries. In 2002, Cleveland Public Library alone answered over 1,000,000 reference questions; compared with a rate of 3,500 questions per month, virtual reference would still only account for 4 percent of the total reference workload — and that's not taking into account the reference statistics from other members of the consortia. So, even in the busiest virtual reference library operation, chat remains very much a niche service, and the vast majority of patrons still seem to want to ask their questions face to face or over the phone.

Nobody likes to see services in which we've invested lots of time and money get little use, however, in retrospect, it may be a very good thing that our patrons have not swarmed to our chat services as we had hoped they would. Why? Because, given what we now know about the costs of virtual reference, the possible budget impact could have been unaffordable.

The Costs of Virtual Reference

All reference is not created equal. There are more- and less-efficient ways of doing it. Some methods cost more money than others. Some require more staff. Some take more time. And some require more effort on the part of the librarian, the patron, or both. Unfortunately, by almost any measure, virtual reference turns out to be a pretty expensive way to answer a question.

First, there is the cost of the technology itself. In order to do virtual reference you first need to have virtual reference software that allows patrons and librarians to chat and collaborate over the Web. High-end packages can easily cost $2,000-$6,000 per "seat" and, after you add set-up fees and the like, software costs for a single library can run as high as $10,000-$20,000 depending on the product and the configuration. Larger installations can run considerably more. Of course, there are less-expensive solutions available, like the various free instant messenger programs and some open source software, but many libraries have found that here, as elsewhere in life, you get what you pay for.

Software isn't the only technology cost. Some libraries have found their computers too antiquated to handle the virtual reference software and have had to buy new hardware as well. Additionally, it costs to re-configure library networks, firewalls, and database authentication software so all will work with virtual reference.

Once purchased, the reference staff must be trained to use the software. Few reference librarians have had much experience with chat, IM, Web collaboration, or any other methods of working live online. After the initial classroom training, many libraries allot a considerable amount of time for staff to practice their new-found skills before they feel comfortable taking their first call. Although initial training costs are often included as part of the software package, the cost of staff time to attend those sessions and hundreds of hours of practice time afterwards are often overlooked.

Software and training costs are trivial compared to what it costs to staff a virtual reference service. With few exceptions, most libraries have found it very difficult, if not impossible, to do virtual reference from the regular reference desk. Distractions at the desk interfere with the concentration needed for chat; walk-up patrons interrupt staff. As a result, most libraries dedicate a separate staff member or members to sit in front of a computer and wait for virtual reference questions during every hour their chat service is open. When patrons are few and far between, opening up a virtual reference service can become the equivalent of opening up a brand-new reference desk, with all the attendant staffing issues and costs. After-hour service requires paying staff for that additional time or paying somebody else to do it for you. Some collaborative services work this out by bartering — you cover my early morning hours and I'll cover evenings for you — but whether calculated in dollars or time, either way there is a price to be paid.

Then there's the amount of time it takes to answer the questions themselves. The average chat question takes 10-15 minutes to answer. We lack good comparative data for the average length of phone or desk reference questions, but research in commercial call centers has shown that it takes about twice as long to answer a question in chat as it does over the phone [http://www.benchmarkportal.com/
newsite/article_detail.taf?topicid=265]. This factor has dissuaded many commercial call centers from adopting the technology. Chatting with more than one patron at the same time requires a very experienced operator to still provide good service to both patrons, assuming one works in a rare virtual reference service that gets more than a dribble of questions and therefore would seek such efficiencies.

Add it all up and you can see that virtual reference built on chat technology is a pretty expensive proposition, especially if we ever came close to getting the thousands and thousands of questions many of us expected.

Ok, so what do we do now? Stay tuned for next month's exciting conclusion of "To Chat or Not to Chat."

Linda Arret is an independent consultant specializing in digital reference services. She has extensive experience as a reference librarian in academic and government libraries, including 25 years' experience in leading and implementing systems-based reference services at the Library of Congress, where she developed generations of digital reference services, was the project coordinator for the early and pilot stages of QuestionPoint, and helped coordinate NISO's first steps toward standards in digital reference. Linda can be reached at linda.arret@verizon.net.