OpenAI today announced a key update that finally delivers the top-requested feature developers have been asking for.
With the launch of its Structured Outputs in the API feature, the company is providing developers with a reliable way to ensure that the outputs of their generative artificial intelligence models match the data stored within their JavaScript Object Notation Schema files.
JSON Schema files are essential for AI development, because they’re used to describe the content, structure and type of data within the JSON documents that developers feed into their large language models. They also describe the constraints on the outputs based on that data.
JSON documents are files that use the JSON format for storing and exchanging text-based data. They’re popular because the form is simple to use, flexible and compatible with multiple programming languages. In addition, it’s easily readable by both humans and machines, and for that reason it has become an industry standard.
Sadly, LLMs don’t seem to quite be able to match the ability of humans and regular machines when it comes to reading JSON files. They struggle to do so, so LLMs that use JSON data will often “hallucinate” and make up responses or output toxic content. That’s because they’re only able to follow the instructions partially within the JSON files. To get around this, developers have been forced to use an assortment of clunky open-source files and test their models with numerous prompts and repeated requests to ensure their outputs fall within safe boundaries.
This is why Structured Outputs in the API has become the most in-demand feature from developers that work with OpenAI’s LLMs. By ensuring that LLM outputs match the contents of their JSON files, developers can build more consistent and safer generative AI applications.
In a tweet, OpenAI Chief Executive Sam Altman said the feature was being released due to “popular demand,” and in a blog post, the company said that evaluations of Structured Outputs on its most powerful LLM, GPT-4o, achieved a perfect 100% score.
by very popular demand, structured outputs in the API:https://t.co/AbfzTp4LMF
— Sam Altman (@sama) August 6, 2024
OpenAI first added support for JSON documents with the debut of its JSON Mode last year, but the feature received mixed reviews because of the problem of ensuring reliable outputs. With today’s update, OpenAI said, developers will now find it much easier to constrain its LLMs and ensure they match their JSON Schemas.
As a side benefit, Structured Outputs in the API also helps LLMs to better understand more complex JSON Schemas, the company said. “While both (JSON Mode and Structured Outputs) ensure valid JSON is produced, only Structured Outputs ensures schema adherence,” the company wrote in its blog post.
According to the company, this means developers will no longer have to worry that the LLMs they’re using might omit a required key or hallucinate on invalid enumeration values. With Structured Outputs, developers will be able to request a “step-by-step” answer that guides them through to the intended output.
“Safety is a top priority for OpenAI,” the company insisted. “The new Structured Outputs functionality will abide by our existing safety practices and allow models to refuse an unsafe request.”
Structured Outputs in the API is available now with GPT-4o, GPT-40-mini and multiple fine-tuned versions of those two LLMs. It can be used with various application programming interfaces, including the Assistants API, Batch API and Chat Completions API, and it is compatible with vision inputs as well as text.
Yet more controversy
Today’s update comes at yet another tumultuous time for OpenAI, which has been hit by the departures of three key executives in the last week. Today Jon Schulman, who played a key role in its product development efforts, said he is leaving the company to join rival AI firm Anthropic PBC. Schulman announced his departure on X, becoming the eighth of OpenAI’s 11 co-founders to quit the company.
Within hours, it followed that another co-founder, Greg Brockman – who serves as the company’s president – was stepping back for four months in order to take a sabbatical and “relax for the first time” since helping to launch the company nine years ago.
Brockman, Altman and chief researcher Wojciech Zaremba are the only members of the original founding team still working at the company. Schulman’s and Brockman’s announcements came on the heels of last week’s news that OpenAI product manager Peter Deng had left the company, having only joined it last year.
Also this week, another of OpenAI’s co-founders, tech entrepreneur Elon Musk, revealed that he has filed another lawsuit against the company. This time around, he is accusing the company of misleading him about the goals of its AI development efforts.
In the lawsuit, Musk says that OpenAI positioned itself as a nonprofit entity that was “committed to making its AI technology available under an open-source license.” He argues that the launch of its for-profit arm later and its subsequent partnership with Microsoft Corp. breached this agreement.
Musk added that OpenAI was guilty of committing a “Shakespearian” betrayal when it abandoned its open-source mission.