OpenAI is moving to strengthen its technology stack. The company today announced it has acquired Rockset, a vendor known for providing a real-time analytics database to power intelligent enterprise applications.
While the terms of the deal remain undisclosed, OpenAI has confirmed that it will integrate Rockset’s “word-class indexing and querying” capabilities to strengthen its retrieval infrastructure across products. The company also added that Rockset’s entire team will transition to OpenAI as part of the acquisition.
The move marks the second major acquisition publicly announced by OpenAI following its acquisition of New York City startup Global Illumination, Inc. last year.
The move also comes as the competition in the gen AI domain continues to intensify. Just yesterday, Anthropic, one of OpenAI’s biggest competitors, released Claude 3.5 Sonnet, a powerful LLM that convincingly outperforms OpenAI’s recently released GPT-4o on benchmarks.
Countdown to VB Transform 2024
Join enterprise leaders in San Francisco from July 9 to 11 for our flagship AI event. Connect with peers, explore the opportunities and challenges of Generative AI, and learn how to integrate AI applications into your industry. Register Now
Even Ilya Sutskever, OpenAI co-founder and former chief scientist, has launched his own new AI startup, Safe SuperIntelligence.
How will Rockset power OpenAI?
Founded in 2016, Rockset provides enterprises with a cloud-based real-time analytics database that allows developers to build data-intensive applications, like those for personalization and IT automation, at scale.
The product continuously ingests and indexes data from sources like Kafka, MongoDB, DynamoDB, and S3, enabling real-time information availability and querying. It can run sub-second SQL queries on semi-structured data, without requiring a predefined schema.
At the core, Rockset uses the open-source RocksDB persistent key-value store created at Meta (formerly Facebook) as a foundation. It acts as an external secondary index for OLTP databases, data lakes, and streaming platforms, accelerating real-time analytic queries and providing performance isolation for primary transactional systems. Throughout 2023, the company improved its product with capabilities aimed at powering AI use cases.
Now, with the acquisition by OpenAI, the tech stack built by Rockset, especially the “world-class” indexing and querying capabilities, will be powering the retrieval stack of OpenAI.
The companies are yet to share the exact specifics on how this integration will take shape, but one thing is pretty much clear: Rockset will enable OpenAI products to answer customer questions with the freshest and most relevant information, faster than ever before. Imagine an enterprise GPT being able to better address user questions with the most accurate data.
“Rockset’s infrastructure empowers companies to transform their data into actionable intelligence. We’re excited to bring these benefits to our customers by integrating Rockset’s foundation into OpenAI products,” Brad Lightcap, COO at OpenAI, said in a statement.
Enterprises and startups using AI tools such as OpenAI’s models have already been attempting to leverage a technique known as “retrieval augmented generation,” coined in a 2020 paper by researchers from Meta, University College London and New York University, which links generative AI with external knowledge bases, ideally improving the model’s ability to handle specific queries and reducing the risk of incorrect answers — such as answering an employee’s question about expense policy guidelines by pulling on an enterprise’s documentation, for example.
As for Rockset, it remains unclear how the business of the company with existing customers will change. It worked with some notable players in the industry, including Klarna, Meta, Whatnot and Windward. Questions sent by VentureBeat remained unanswered at the time of writing.
“We’re excited to be joining OpenAI to empower users, enterprises and developers to fully leverage their data by bringing powerful retrieval to AI,” said Venkat Venkataramani, CEO of Rockset, in a statement.
