SUBSCRIBE NOW!

Vol. 43 No. 9 — November 2023

FEATURE

In order to overcome its hallucinatory deficiencies and the fact that the large language model on which it was trained ends in 2021, OpenAI began rolling out plug-ins to its ChatGPT Plus users in March. These were designed to be the eyes and ears for ChatGPT, allowing it to access more recent information and perform a number of other very specific tasks, such as searching databases, booking a flight, or making a restaurant reservation.

The plugins allow users to move beyond the limitations of ChatGPT and to extend its functionality without interfering with its code. OpenAI hopes that not only will this allow ChatGPT to regain some of the ground and confidence it lost, but also by limiting access to plugins to paying subscribers, give potential customers another reason to subscribe.

When the plugins were first announced, there were fewer than 100 of them. There are now around 650—probably more by the time you read this. They have been created by third-party developers to suit their particular needs and requirements, hopefully giving them access to more customers as well as raising their profile. Early plugins were produced by Expedia, Shopify, Wolfram|Alpha, and Zapier. Open-AI reviews all plugins before they are added to its collection, but it’s up to users to decide if they are prepared to trust them or not. A recent WIRED article (“ChatGPT Has a Plugin Problem”) suggests there may be some concerns over their potential security (wired.co.uk/article/chatgpt-plugins-security-privacy-risk). However, I haven’t seen any stories yet indicating that any plugins have acted maliciously and done any damage, so I think it’s more of a theoretical concern than a practical one.

The only real issue that OpenAI has had with plugins is one of its own making. It produced a plugin that integrated the Bing search engine directly into ChatGPT, making it possible to search the internet in real time. However, this had the unintended result of giving people access to material behind paywalls, so it was withdrawn a few days later. There’s no information on when—or if—it will return.

Using Plugins

Once you’re a ChatGPT Plus subscriber, you’ll have access to the plugin library at no extra cost. First of all, you need to go into your settings and enable plugins. After that, you can click on the GPT-4 button and scroll down the options to Plugins. Next, click on the plug-in store to gain access to the entire collection. You can choose to view them by Popular, New, All, or Installed. Finally, there is a search option, which is becoming more useful given the increasing numbers of plugins. Choose a plug-in, and click to install it. You can install as many as you want, but you are limited to having a total of three in operation at any one time. If you want to use a different one, you have to swap it out with one currently in use. You may also need to register with the organization that produced the plugin before you can use it, which is an irritating extra step, but it only needs to be done once.

ChatGPT generally chooses which plugin it uses to accomplish a particular task. I usually include the name of the plugin I want it to use in the prompt, just to be on the safe side. Users will then see an icon to say that ChatGPT is using the particular plugin. When it’s finished, you’ll see the response on the screen. You can click to see the dialogue that the chatbot had with the plugin, where it went, and what instructions were given and executed. For example, I used the ScholarAI plugin to find scholarly articles on using Tixagevimab plus cilgavimab for preventing COVID-19. The request that was sent to the database was this: “keywords”: “Tixagevimab, cilgavimab, COVID-19, prevention”, “query”: “Tixagevimab plus cilgavimab for preventing COVID-19.” In return, I got the result, nicely formatted as an answer, but could also go and look at the raw response that had been returned, which said the same thing. It was much more difficult to read, so you’ll probably only do it once for curiosity’s sake.

I could just as easily have gone to the website to search OA Springer Nature articles. However, once ChatGPT has access to the results of a search, I can then ask it to manipulate that information. I can ask for a summary of a particular article or ask for it to be rewritten. I can even ask for it to be reproduced in the form of a poem, if I were so inclined. Plugins do more than simply reach out to a third-party resource; they give ChatGPT the ability to transcend the September 2021 cutoff date and access material on the web that is current to whatever content the plugin provides access to.

Another benefit of using plugins is that they guide ChatGPT to specific data, reducing the chances of generating inaccurate information. If I asked the free, 3.5 version for academic articles on the extent to which those drugs prevented COVID-19, I simply couldn’t trust the results, due to the very great possibility that it would confidently make up papers, references, and authors that simply don’t exist. By restricting it to a trusted body of knowledge, it doesn’t have room to hallucinate—meaning that it’s not necessary to double-check the information being returned, saving a considerable amount of time and headaches.

What Plugins Are Available?

There are a lot of plugins to choose from. A quick glance at the Popular list shows several that illustrate their breadth and range. There are plugins that allow users to link ChatGPT to an online PDF file, which can then be interrogated. Other plugins will allow ChatGPT to browse dozens of webpages in a single query, generate PDF files, generate CSV files, or build charts, graphs, and diagrams. Users can install plugins to search Expedia or Kayak to help plan a holiday. These search flights and give recommendations on where to stay and what to see.

There are tools to help you search the internet as well. Let me be clear: ChatGPT isn’t a search engine—that’s not what it was designed as—but some plugins overcome this limitation. YouTube transcripts can be searched. The Wikipedia plugin allows users to ask questions about its content, search for public information from the U.K.’s Companies House, get answers from millions of scientific papers, search for jobs, images, social networks, and the U.S. census—the list is almost endless. A search for “search” in the list of plugins produced more than 110 tools that can be used for searching various databases or types of knowledge.

Advanced Data Analytics

OpenAI hasn’t stopped at providing access to plugins. It also produces a tool (again, subscriber only) called Code Interpreter. To be fair, it is one of the most boring and ineffectual product names that I have ever seen. The company obviously agreed with me, because OpenAI renamed it Advanced Data Analytics at the end of August 2023. To be honest, I think the new name isn’t much of an improvement, although it’s slightly more descriptive as to what the plugin does. It’s an amazing tool—basically, a staff member who is trained in using analytics tools to interpret any data that you give it.

When you want to use it, you simply call it up from the same pull-down menu used for plugins. You can then click on the little plus sign in the search box and upload your file—such as a CSV, Excel spreadsheet, or SQL database—and ask ChatGPT to give you insights into what it finds. ChatGPT is excellent at explaining what exactly it’s doing and how it’s interpreting the data it finds.

I gave it an Excel spreadsheet on a year’s worth of data on how far I walked each day in terms of steps, miles, and time. It was able to work out exactly what the data was, figure out how big the dataset was, and describe the type of cells. It immediately did some calculations to show approximately how far I walked every day in both steps and miles, as well as how long it took. I was then able to see which days I walked the most, average steps to a mile, average time taken to walk a mile, and so on. Additionally, I had it produce a graph based on dates and steps, dates, and miles, as well as pie charts to show the total steps taken in each month as a percentage of the total number of steps taken. If I’d also listed the different places that I had walked, it could have produced a map for me, as well color-coding locations to distances walked.

This was all painfully simple data, but even so, it would have taken me a reasonable amount of time to extract that information myself. I’m sure you can come up with your own examples, but I asked ChatGPT for some suggestions:

• Text analysis—Analyzing customer reviews to determine overall sentiment about library services
• Data management—Removing duplicates or correcting inconsistences in library databases
• Statistical analysis—Using statistical models to predict which books or materials should be stocked based on past usage data
• Date and time information—Understanding the busiest and slowest hours of the library to optimize staffing

Other examples I’ve seen include taking data on world heritage sites and creating maps that produce a PNG file I could download, detailing sites per country. I could give it information on pound-to-dollar exchange rates and ask it to plot daily prices in a chart.

It doesn’t end there. It can run and test code, interpret it, and then explain it to you. It can produce Python code to solve mathematical problems. You can upload equations and get the full answer, including all of the workings, which is going to give math teachers headaches, I’m sure. It can identify insights from the datasets it’s given, predicting patterns and trends, and provide recommendations or action items based on what it has discovered from the dataset. It can even draft a report for you based on the analysis of the data you have given it.

I would go so far as to say that, for many people, this is going to be the single most useful tool that OpenAI has produced for use with ChatGPT, and we haven’t even scratched the surface of its potential. Of course, it’s far from perfect—the file size limit for uploading data to be processed by the tool is 5MB as of this writing. As a result, you may need to preprocess it locally before uploading a smaller subset. If this is an issue for you, it may be worthwhile to look at one of ChatGPT’s closest competitors, Claude. It allows users to upload five files, 10MB each, and it accepts PDF, TXT, CSV, and so on. The disadvantage is that it can’t produce graphs or visualize trends—but it is free. If you’re not satisfied with that, you could always visit Perplexity and upload a file for it to analyze for you, but you’re limited to a single file up to 25MB, with a total of four uploads a day.

The Future

It would take a braver—or perhaps more foolish—person than me to predict exactly where we are going next. However, that would make a pretty poor ending to an article, so as long as you don’t hold me to any of my predictions in this area, I’ll give it a go.

We are already seeing competition for ChatGPT by Bard and the aforementioned Claude and Perplexity. It’s only a matter of time before they start to offer their own range of plugins, and if they can provide them for free, it’s going to put pressure on ChatGPT to make its offering available to everyone, not just the paying subscribers. By the end of the year, there will be thousands of plugins available, and they will be able to work seamlessly with their associated chatbots. While all of them will have some limitations in terms of the currency of data and the tendency to hallucinate, a balanced use of plugins will alleviate those problems.

Plugins will increase in functionality. They will be able to translate conversations in real time, convert text to speech, and add sign language avatars. Users will be able to customize their look and feel. ChatGPT recently added in a feature to customize itself, so it’s only logical that this will also happen to plugins as well.

At the moment, plugins are basic, since they act as the filling in the sandwich of data and chatbot. I can expect them to start to work together. So if I want to go on holiday, I can use one plugin to find a good location for me, then pass that information onto a second that can suggest good hotels, before handing off to another to book reservations in restaurants that serve the food I like.

Alternatively, we will end up with all-encompassing plug-ins that will offer to do everything for us in one go. We will see domain-specific features, so I’ll be able to use a plugin to talk to my fitness device, identify my physical activity, talk directly to my physician to set up appointments, or get repeat prescriptions while simultaneously searching for interesting articles that are pertinent to my conditions. A hypochondriac’s dream, I’m sure, but you get the point.

We will have plugins that are able to work across teams. If a number of people are working on a project, the plugin will be able to search for information and channel it through to the appropriate team member in detail while sending a brief overview to others to keep them in the loop. The plug-in will be able to handle discussions, create to-do lists, arrange meetings by accessing team members’ calendars, do real-time searching during the meeting, translate conversations into different languages (in real time), and display charts and graphs to the entire group based on what data is uploaded.

If we thought that ChatGPT was going to be a revolution in the world of information, it’s only going to be the tip of a very large AI iceberg, most of which will be plugins and data analyzers working beneath the surface. We will be going from chats to charts to connections, seamlessly bridging the gap between conversation and data-driven insights. This is just the beginning of a journey that is going to redefine how we interact with, analyze, and understand information.

[A version of this article originally appeared in Information Today Europe/ILI365, Sept. 5, 2023. —Ed.]