5 ways OpenAI’s ChatGPT plugins could change the AI game | The AI Beat

Last week felt like a lifetime in AI. Sure, the week before was fast-paced and so was the week before that, but this one — really, a lifetime. And somehow I just couldn’t let it go. While the rest of the world went about its business, I noodled about the implications of OpenAI’s latest ChatGPT chess move.

You’ve likely heard by now that OpenAI unveiled plugins that link its conversational AI ChatGPT to the real world. Developers can connect their apps and data to ChatGPT, letting it summon information (stock ticks, headlines), regurgitate docs, or act on your behalf (book travel, place takeout orders).

The news immediately got Twitter buzzing, with many seeing the initial list of available ChatGPT plugins, including Expedia, Instacart, Zapier and OpenTable, as a signal of OpenAI‘s ambitions to further its dominance by turning ChatGPT into a developer platform. Others marveled at the simple yet powerful fact that the plugins make it possible for ChatGPT to browse the internet for real-time information.

So how was my weekend? Well, I considered some of the other big ways ChatGPT plugins will shift the AI landscape. Here are five possibilities:

1. Help minimize ChatGPT hallucinations

According to OpenAI, plugins offer the potential to tackle various challenges associated with large language models, including “hallucinations,” keeping up with recent events, and accessing (with permission) proprietary information. By integrating explicit access to external data — such as up-to-date information online, code-based calculations, or custom plugin-retrieved information — language models can strengthen their responses with evidence-based references.

But one software developer and AI researcher, Chomba Bupe, pointed out on Twitter that while the plugin API itself may be factual, ChatGPT can “still hallucinate when formulating a prompt from the user & it can further introduce unwanted details when reinterpreting results.”

2. Share data to gather data

I had an interesting talk today with Tim Hwang, founder and CEO of FiscalNote, whose plugin is one of the initial batch that OpenAI announced on Thursday. FiscalNote aggregates legislation, regulations and government filings from thousands of federal, state and local agencies; uses AI models to structure it and normalize it; and delivers personalized data feeds to company clients.

FiscalNote partnered with OpenAI on its plugin partly as a way to combat misinformation with solid data about government filings. “I do feel like the data that we’re contributing is very important to making sure that the technology is able to deliver factual outcomes,” said Hwang.

But, he added, the plugin also adds value to FiscalNote, because the company receives data from OpenAI on ChatGPT queries, which it can then add to its own AI models. “We serve up a portion of our data to OpenAI. We don’t open up the entire firewall,” he said. “We can see what people are querying at any given time, which enables us to go back and refine our own data collection efforts and dig deeper into areas that people want.”

3. Put the future of websites in peril

What does it mean if ChatGPT can crawl websites for their content for free? Will it kill websites and website traffic, for example, if fewer people browse websites since they can get a direct answer right from ChatGPT?

As Riad Benguella, an Automattic engineer, blogged a couple of weeks ago: “Why would I open a browser if I can just talk to a bot and get an instant customized reply?”

4. Create security issues for sensitive data

A bug in one of ChatGPT’s new plugins leaked personal information as well as credit card details of several of its premium, paying customers to other users of the service. OpenAI quickly fixed the bug and revealed that only a small portion of users had been affected.

However, at the same time, the service encountered a few security issues. First, an ethical hacker revealed a list of unreleased plugins, exposing a potential flaw, and a bug may have exposed user’s credit card details. Rightly, the ChatGPT community is concerned that ChatGPT’s vulnerabilities might open it up to future attacks.

5. Hello, vector databases

OpenAI released three of its own ChatGPT plugins, including an open-source knowledge-retrieval plugin “to be self-hosted by any developer with information with which they’d like to augment ChatGPT.” It allows users to “obtain the most relevant document snippets from their data sources, such as files, notes, emails or public documentation, by asking questions or expressing needs in natural language.”

For plugins retrieving information, developers need to access a vector database that indexes and searches documents and acts as a “long-term memory” for the application. The ChatGPT plugin allows developers to choose one of several vector database options, including Pinecone. “Everyone is rushing to build [a ChatGPT plugin] now,” said Pinecone VP of marketing Greg Kogan. “Companies who build plugins will need this vector database component, this long-term memory.”