SUBSCRIBE NOW!

Vol. 44 No. 10 — December 2024

FEATURE

During the next 5 years, web scale discovery became a standard feature in libraries, but the mergers have only continued. The ability to go your own way on discovery became increasingly less feasible. The explosion of generative AI (gen AI) into the popular consciousness happened at a faster speed than when discovery layers became the norm, but looking back at how those discovery layers were implemented and what choices we wish we had made differently at that prior inflection point can help us make better decisions now. Note that while various large languages models (LLMs) and machine learning (ML) have been around for a long time, this article will focus on the commercial chat-style generative pre-trained transformer (GPT) tools, although without discussing any specific engine or approach.

The Obscurity of Volume

Librarians are trained to understand how information sources are created, described, indexed, and distributed. As the volume of information increased, the tools to process and distribute that information grew more complex. Where prior adoption pathways might have required learning a new controlled vocabulary or indexing codes, systems such as discovery layers and now gen AI tools have made it possible for researchers to find their own information. However, they have made it more challenging for librarians to assist those users or explain search results. Specialized research skills only take you so far with the increased volume of what is available and the obscurity of the tools.

Using computing scale and indexing technologies that had previously been inaccessible to most libraries, discovery at web scale allowed indexing and searching the full text of materials and combined all types of resources into one set of results. While far more convenient in some ways than using multiple systems and learning how different material types are indexed in disparate databases, the convenience obscured the true complexity of what was happening. The algorithms that combine and rank results are obscure even to their creators, which means that even expert librarians are not experts when explaining to researchers why a particular result appears where it does.

The importance of information literacy in this over-rich information environment is not in dispute, but knowing the value of the tools and how best to deploy them requires more technical acumen and flexibility than our older systems did. Librarians could, and did, become expert searchers in older systems that, with experience, would return results for everything on a particular topic the index contained. Piling billions of records with different controlled vocabularies, metadata schema, and obscure indexing on top of each other makes comprehensive retrieval a much harder job, even assuming that this is desirable, given the proliferation of information. Understanding how to approach specific research questions can be overwhelming and leaves researchers susceptible to manipulation—or giving up.

But there is no going back. Even recognizing the downsides, what we have now is a basic expectation. The irony is, of course, that the downsides of a huge volume of largely undifferentiated information is exactly the utility that gen AI tools seek to provide. The scale requires topic modeling and ML to categorize what exists and bring together logical groups. The generative aspect is largely a marketing ploy to encourage comfort with tools that are raw computing but with a veneer of entertainment. Understanding what is really going on is impossible, but we can, just as we continue to do with our previous tools, start to analyze and then make ethical decisions based on that analysis.

Analyzing the Factors of a System

The frameworks in which libraries usually adopt new technologies often involve a department or role that is tasked with evaluating emerging trends along with working groups for cross-departmental collaboration. Such groups may make recommendations for policy, become standing committees, or eventually become new departments. Discovery layer committees required cross-departmental efforts. The exercise of merging and viewing the library’s data across systems eliminated silos that may have existed among cataloging, e-resources management, and user instruction.

Similar exercises are necessary for adopting new AI tools. During the past 2 years, libraries have established AI working groups of various sorts. The first iteration of these groups was largely reactive: understanding how the new popular chatbot gen AI services were going to change how instructors found plagiarism, figuring out ways of rewriting assignments to use these tools rather than avoid them, and educating about the hallucinatory citations these tools produced. The second phase can be more proactive, spending time on careful analysis. Analyzing new systems requires understanding them from both a technical and commercial perspective, as well as within the context of professional ethics and institutional values.

Technical factors include such considerations as data sources, algorithms, and user interfaces. Data sources may be licensed vendor content, historical corpus collections, the open web, or another combination. For tools produced by vendors, ensure the documentation includes (and ask if needed) which LLMs they are using, which specific data sources they are including in the dataset, and what sort of retrieval-augmented generation (RAG) is taking place in the process. Library data that includes patron information should never be used in data that could be used to train models, and care should be taken when testing tools that license agreements are followed. Getting to the point where you can even understand the answer to these questions requires some willingness to learn and experiment, especially if groups include staff for whom this is all new.

Analysis should also include understanding commercial factors such as additional costs that may eventually be passed along to the customer, how the tool fits into the marketplace outside of libraries, incentives for the pathways to profitability for the vendor, and customer voice in the future of the tool. In the same way that discovery layers encouraged homogeneity because of the challenge in obtaining the indexed data, using third-party AI tools on licensed content will likely be a challenge. We want to support a variety of business models and diverse outlooks in making technology purchase decisions just as we do in resource purchases.

While libraries face evaluating AI tools created by vendors on top of their existing products as well as add-ons, avoiding such tools is another choice. This choice must be understood in light of what researchers who want or need to use AI tools will do. Libraries that never adopted a discovery tool may find that their researchers rely on other commercial options such as Google Scholar or perhaps something more nefarious. Libraries who do not adopt an AI tool on top of their licensed data may face issues with researchers using this material against terms of license in other systems. There may be elements of a new system that are worth introducing for educational purposes, even if they are not recommended for serious use. In addition, adopting gen AI tools on top of open data or special collections will require guiding researchers on their best use.

AI tools designed to process library, archives, or museum collections are a slightly different question, but they have the exciting potential to increase discoverability of materials that have remained unprocessed or lack full descriptions. Libraries planning adoptions of AI tools may want to focus on using these on data and processes they control rather than adopting and teaching commercial products.

Making Ethical Choices

The reality is that for libraries within higher education institutions that view the AI explosion as a new pathway to relevance and revenue, there is a strong incentive to adopt and promulgate such tools and to be part of emerging AI research groups. The pressures come from all sides, and with a societal level change of this magnitude, spending time considering the ethical implications of decisions is an important measure to counter the shortsighted following of immediate incentives.

While the move to the internet and digital resources for libraries was a massive technical and logistical change in services, the ethical considerations for the move to gen AI are just as great, if not greater. For some, the current answer to adopting this technology is a clear, “No, not worth it.” The technology is too tainted by the perceived or real misuse of intellectual property, has negative environmental impacts, and does not meet a real need.

For most, avoiding the technology wholesale is not a real option, but it is worth establishing firm parameters for what is acceptable and in which direction development should go. Just as the Open Discovery Initiative (ODI; niso.org/publications/rp-19-2020-odi) helped establish a shared set of principles among discovery vendors, professional communities can work together to establish these for gen AI tools. ODI is currently looking at the state of gen AI in discovery layers. At the same time, larger libraries that are working with AI researchers or working on LLMs to process their own collections need to establish a set of ethics as part of this research.

The Library of Congress Labs Artificial Intelligence Planning Framework (blogs.loc.gov/thesignal/2023/11/introducing-the-lc-labs-artificial-intelligence-planning-framework) is one such model, although it can work for any adoption of new technology. Understanding is the first phase, in which we can evaluate use cases for tools and how these fit into existing systems as well as the risk in the tool, especially considering the complexity of library or archive data. Experimenting is the next phase, in which tools are put to the test using objective, predefined metrics to ensure that the tool meets the need it is intended to fill at a high enough accuracy rate to be useful. Lastly, implementation is the slowest phase, as the library makes clear decisions that are in the best interests of its stakeholders.

A phased adoption approach requires the ability to opt out or adjust as testing uncovers issues, which can be a challenge when working with commercial products. While the discovery market has become increasingly homogenous, the overall pressure to bring gen AI into discovery layers or other search tools means that companies, however mission-driven, will follow the market first. For customers lacking the capacity to be a development partner or early adopter for testing new tools, it is crucial to find other customers in the user community to share results of testing and discuss the implications of adopting the tool, as well as to participate within any formal feedback mechanisms. Consortiums or regional networks can also be important communities from which to draw expertise. Libraries do not have identical ethical frameworks, but they can begin to establish good professional norms.

In adopting an ethical stance, it will be necessary to think about what the understanding phase will mean. Before dismissing or adopting a tool, we should complete the technical assessment above and be able to speak critically about the advantages and disadvantages. Explaining clearly to researchers how a system gets its data and interprets it can be a good avenue for discussing concepts that were always an issue in discovery layers as well, such as algorithmic bias, which is just as prevalent but possibly more obscure in an AI tool. This will, of course, require learning tools and being ready to teach several different systems that researchers may already be using.

Ultimately, when refusing to use a system you believe to be unethical, and therefore not bringing to an implementation phase, you should be able to understand why you made the choice and what is lost, or gained, by that decision. Libraries do not have to fill all technical niches or adopt technology beyond their ability to support it. However, understanding what needs people are filling with AI tools may allow libraries to introduce different methods. For example, given that chat gen AI tools can create false citations, librarians can teach how to search library systems to find the appropriate, correct citations. Open data and open source technology may likewise fill specific needs. Open source LLMs and GPTs are available; many are already in use in academic research or library projects. The shift to AI at a societal level will not allow anyone to remain ignorant, however. Learning the tools and being able to recommend the appropriate approach for each research problem with the most ethical approach are going to be the most realistic ways that libraries move forward.

The Next Few Years

Libraries will be inundated with new tools in the coming years, and assessing all of the new trends and adapting to the massive changes that will inevitably occur are going to take planning and commitment. Ongoing analysis and integration across the library will be important, and those AI task forces or working groups springing up in many libraries will need to create flexible structures for ongoing assessment. In the short term, those who were and are skeptical of web scale discovery will now have another layer of obscurity on top of that, but increasingly, these will be available in what we might think of as a traditional abstracting and indexing database. The more technically minded among library staff will have to continue to build their own knowledge and evince empathy for their colleagues to help them meet new challenges.

It may well be that before much longer, societal barriers will be in place for people without access to AI technology, just as people without access to a computer or internet currently face difficulties for their basic life needs. Libraries could fill this gap in the same way that they provide public computers and digital literacy training. OA initiatives created and funded by libraries have gone a long way to making information more widely accessible. Using some of the same shared infrastructure that have made other large technology collective projects possible may be possible for providing more equitable access to this emerging technology.

Inflection points in our professional lives should invite reflection. Knowing what we know now about how discovery layers changed our relationship with library data, what choices do we wish we had made at that time? We can project ourselves in the future 15 years and imagine a future where AI works for us, not against us, and the obscurity of the technology does not cloud judgment.