OpenAI’s Custom Chatbots Are Leaking Their Secrets

You don’t need to know how to code to create your own AI chatbot. Since the start of November—shortly before the chaos at the company unfolded—OpenAI has let anyone build and publish their own custom versions of ChatGPT, known as “GPTs”. Thousands have been created: A “nomad” GPT gives advice about working and living remotely, another claims to search 200 million academic papers to answer your questions, and yet another will turn you into a Pixar character.

However, these custom GPTs can also be forced into leaking their secrets. Security researchers and technologists probing the custom chatbots have made them spill the initial instructions they were given when they were created, and have also discovered and downloaded the files used to customize the chatbots. People’s personal information or proprietary data can be put at risk, experts say.

“The privacy concerns of file leakage should be taken seriously,” says Jiahao Yu, a computer science researcher at Northwestern University. “Even if they do not contain sensitive information, they may contain some knowledge that the designer does not want to share with others, and [that serves] as the core part of the custom GPT.”

Along with other researchers at Northwestern, Yu has tested more than 200 custom GPTs, and found it “surprisingly straightforward” to reveal information from them. “Our success rate was 100 percent for file leakage and 97 percent for system prompt extraction, achievable with simple prompts that don’t require specialized knowledge in prompt engineering or red-teaming,” Yu says.

Custom GPTs are, by their very design, easy to make. People with an OpenAI subscription are able to create the GPTs, which are also known as AI agents. OpenAI says the GPTs can be built for personal use or published to the web. The company plans for developers to eventually be able to earn money depending on how many people use the GPTs.

To create a custom GPT, all you need to do is message ChatGPT and say what you want the custom bot to do. You need to give it instructions about what the bot should or should not do. A bot that can answer questions about US tax laws may be given instructions not to answer unrelated questions or answers about other countries’ laws, for example. You can upload documents with specific information to give the chatbot greater expertise, such as feeding the US tax-bot files about how the law works. Connecting third-party APIs to a custom GPT can also help increase the data it is able to access and the kind of tasks it can complete.