SUBSCRIBE NOW!

Vol. 43 No. 6 — Jul/Aug 2023

TECHNOLOGY & POWER

An LLM is a type of ML model that performs various natural language processing tasks—such as recognizing, summarizing, translating, and generating text; answering questions; and carrying on a conversation. An LLM is developed by deep learning techniques, and training its artificial neural networks requires a massive amount of data. Deep learning is a type of ML, and ML is a subfield of AI. Since ChatGPT outputs new content as a response to a user’s inquiry, it is considered a tool in the realm of generative AI.

Generative AI for Content Creation

ChatGPT was launched by OpenAI on Nov. 30, 2022, and it quickly became a phenomenon. In 5 days, more than a million people signed up to try this product. Shortly thereafter, on Feb. 7, 2023, Google unveiled Bard, its own ChatGPT-like chatbot. Bard is built on top of Google’s natural language processing model, called LaMDA, which stands for Language Model for Dialogue Applications. Around the same time, Microsoft debuted a new Bing chatbot as a competitor to ChatGPT and Bard.

In addition to generating a human-like conversation, ChatGPT and other similar AI-powered chatbots can write essays, computer code, recipes, grocery lists, and even poems. While some inaccuracies and incoherence were soon found in the responses of various chatbots (Bobby Allyn, “Microsoft’s New AI Chatbot Has Been Saying Some ‘Crazy and Unhinged Things,’ ” NPR, March 2, 2023; npr.org/2023/03/02/1159895892/ai-microsoft-bing-chatbot), there is general agreement that these AI- powered chatbots perform noticeably better than any past, non-AI chatbots.

While ChatGPT, Bard, and Bing chatbots generate texts as a response, DALL-E, Midjourney, and Imagen create images as an output upon receiving a user input in text. Make-A-Video generates a video that fits the description given in text, and MusicLM generates a piece of music. GitHub Copilot outputs computer code and is used as a pair programming tool. These AI tools are introducing generative AI to the public at an increasing speed.

What Is Generative AI?

Generative AI refers to deep-learning algorithms that generate novel content in a variety of forms—such as text, image, video, audio, and computer code. New content thus generated can be an answer to a reference question, a step-by-step solution to a problem posed, or a machine-generated artwork, just to name a few possibilities.

As with any deep-learning model, developing a generative AI model requires a large volume of data for training, a large number of parameters, and a large amount of processing power. The largest model of GPT-3 was trained on 499 billion tokens of data that came from approximately 45 terabytes of com pressed plain text, which is equivalent to about 1 million feet of bookshelf space or a quarter of the entire collection of the Library of Congress (Tom B. Brown et al, “Language Models Are Few-Shot Learners,” arXiv, July 22, 2020; doi.org/10.48550/arXiv.2005.14165). The largest model of GPT-3 has 175 billion parameters and would require 355 years and $4.6 million to train, even with the lowest-priced GPU cloud on the market and if you used a single GPU for it (Chuan Li, “OpenAI’s GPT-3 Language Model: A Technical Overview,” The Lambda Deep Learning Blog, June 3, 2020; lambdalabs.com/blog/demystifying-gpt-3). These examples show that developing a generative AI model such as GPT-3 is resource-intensive and costly.

AI for Scientific Research

AI and ML led to the rise of very powerful general-purpose content-creation tools. Another field that has actively adopt ed ML is scientific research. A great example in this area is AlphaFold, an AI program developed by DeepMind. DeepMind is the company that built AlphaGo, which made headlines back in 2016 by winning a game of Go in its match with the 18-time world champion Lee Sedol. (Google has owned DeepMind since 2014.)

AlphaFold takes a protein’s genetic sequence as an input and outputs the prediction of its 3D protein structure with impressive accuracy. In July 2021, DeepMind announced that it had used AlphaFold to predict the structure of nearly every protein made by humans, as well as the entire “proteomes” of 20 other widely studied organisms, such as mice and the bacterium E. coli (Ewen Callaway, “DeepMind’s AI Predicts Structures for a Vast Trove of Proteins,” Nature vol. 595, no. 7869, July 22, 2021; doi.org/10.1038/d41586-021-02025-4). A proteome refers to the complete set of proteins made by an organism, such as a species or a particular organ.

Working in partnership with European Molecular Biology Lab oratory’s European Bioinformatics Institute (EMBL-EBI; ebi.ac.uk), DeepMind released more than 200 million protein structure-predictions by AlphaFold and made them freely available to the scientific community. These included nearly all cataloged proteins known to science. What AlphaFold does is significant because proteins often fold into elaborate 3D structures and even form complexes with each other to perform certain functions in a cell. For this reason, being able to predict the 3D shape of proteomes is quite valuable in life science research, even more so in drug discovery.

As another example of applying AI and ML to scientific re search, evolutionary biologists employed facial recognition, one of the most widely used ML techniques, to track chimpanzees in the wild, which is not easily achievable by other means (Daniel Schofield et al., “Chimpanzee Face Recognition From Videos in the Wild Using Deep Learning,” Science Advances vol. 5, no. 9, Sept. 4, 2019; doi.org/10.1126/sciadv.aaw0736). Beyond life sciences and evolutional biology, ML techniques are also being ap plied in many other disciplines such as anthropology, astronomy, astrophysics, chemistry, evolutionary biology, engineering, and meteorology.

Generative AI for Library Users

What do these developments in ML and generative AI mean to libraries and library users? For one, requests for non-existing citations are finding their way to librarians at my library as students try to obtain articles cited by ChatGPT without realizing that they were simply made up. However, students are not the only ones who are falling prey to ChatGPT’s so-called “hallucinations” (Karen Weise and Cade Metz, “When A.I. Chatbots Hallucinate,” The New York Times , May 1, 2023; nytimes.com/2023/05/01/business/ai-chatbots-hallucination.html). In the recent court case Roberto Mata v. Avianca Inc., a lawyer cited and quoted nonexistent cases, which he had gotten from ChatGPT, in his legal brief (Benjamin Weiser, “Here’s What Happens When Your Lawyer Uses ChatGPT,” The New York Times, May 27, 2023; nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html).

As a generative AI model, ChatGPT composes its responses based upon statistical probability from the data on which it is trained. In that sense, ChatGPT is basically an autocomplete program, albeit a highly sophisticated one. What this means is that ChatGPT cannot differentiate what looks real from what is real. Nevertheless, students already seem to be using ChatGPT for their academic assignments, and instructors, such as Siva Vaidhyanathan, are well aware of this trend (“My Students Are Using AI to Cheat. Here’s Why It’s a Teachable Moment,” The Guardian, May 19, 2023; theguardian.com/technology/2023/may/18/ai-cheating-teaching-chatgpt-students-college-university).

In the recent poll about generative AI conducted by Educause, which received 1,070 complete responses, 54% of respondents indicated that generative AI is impacting higher education and selected teaching and instructional support as the most impacted areas (Nicole Muscanell and Jenay Robert, “EDUCAUSE QuickPoll Results: Did ChatGPT Write This Report?” EDUCAUSE Review On line , Feb. 14, 2023; er.educause.edu/articles/2023/2/educause-quickpoll-results-did-chatgpt-write-this-report). Since the verification of sources and original and independent thinking are essential to academic and research integrity, the uncritical use of ChatGPT and other similar generative AI tools for learning, teaching, and research purposes should be carefully considered.

Perhaps as an answer to spurious sources conjured up by generative AI chatbots, the U.K.’s CORE (Connecting Repositories) project (core.ac.uk) released CORE-GPT, which answers a user’s question based upon information from CORE’s corpus of 34 million OA scientific articles along with their citations. Like CORE-GPT, scite (scite.ai) also provides an answer with explicit references to published scientific research papers.

Other examples of AI products that aim to facilitate learning and research include Consensus, Elicit, and Librari Consensus (consensus.app) is an AI-powered search tool that takes in research questions and finds relevant answers by extracting and distilling findings from scientific research papers in the Semantic Scholar database. Elicit (elicit.org) is an AI research assistance tool that aims to expedite the literature review process by providing a list of relevant articles and the summaries of their abstracts specific to a user’s query. Librari (librari.app) promises the answering of factual questions, helping with schoolwork, providing reader advisory services, and performing creative tasks based upon its answers curated with more than 300,000 human-engineered prompts.

While generative AI products have achieved some remarkable feats in their performance so far, they are hardly mature yet. As Elicit’s FAQ page (elicit.org/faq) explicitly states, an LLM may miss the nuance of a paper or misunderstand what a number refers to. Nevertheless, the overwhelming amount of interest in these tools suggests that they will be rather quickly adopted and utilized for a wide variety of scholarly and research activities. We are likely to learn more about what these generative AI tools are well-suited for and what we humans are better at.