Dremio, the open data lakehouse vendor that combines the capabilities of a data lake and warehouse on a unified layer, is going all-in on generative AI.
The company today announced two new gen AI capabilities for its platform: a text-to-SQL experience for conversational querying of data, and an autonomous semantic layer to help with data cataloging and processing.
>>Follow VentureBeat’s ongoing generative AI coverage<<
The offerings will simplify working with data for Dremio users, enabling them to explore, discover and analyze their data assets quickly and easily. Similar efforts have been made by other leading players in the data ecosystem (including Snowflake and Informatica), signaling the rise of AI-driven data handling.
Unlocking value from data has long been dependent on a number of manual and time-consuming steps. With the latest capabilities, Dremio is using generative AI to address some of these gaps.
For instance, with the new Text-to-SQL experience, instead of spending time writing complex SQL (structured query language) queries, users can simply put in natural language inputs to get insights from their data. The offering uses a semantic understanding of metadata and data, and automatically converts the plain-language query into SQL, providing the desired results.
Similarly, the autonomous semantic layer uses generative AI to eliminate the hassle of manual data cataloging. It automatically learns the intricate details of users’ data and produces descriptions of datasets, columns and relationships to establish taxonomies for easy discovery and exploration of data.
According to Dremio, this layer also learns from users’ workloads and creates reflections (optimized materialization of source data or a query) to accelerate data processing.
“By integrating generative AI capabilities into our platform, we are accelerating data workflows and eliminating much of the manual work involved in SQL development, data catalog creation and curation, and more,” Tomer Shiran, cofounder and CPO of Dremio, said.
“Generative AI will transform data engineering, data science and analytics over the coming years, and we are excited to provide our users with the industry’s most powerful tools to uncover the true potential of their data,” he added.
In addition to the generative AI tooling, Dremio is integrating vector database capabilities directly into its lakehouse, enabling companies to build AI-powered applications without creating additional data silos.
With this feature, users will be able to add a column of type “vector” to store and search embeddings for various data elements.
For instance, if a user has a table of Amazon reviews, they will be able to store the embeddings that encode the meaning of each review alongside other attributes. Then, when required, they could use Dremio’s indexes and SQL functions to retrieve similar or related reviews based on their meanings.
The text-to-SQL experience is now available for Dremio users, while the autonomous semantic layer and vector database capabilities will be rolling out at a later stage.
>>Don’t miss our special issue: Building the foundation for customer data quality.<<
Join top executives in San Francisco on July 11-12, to hear how leaders are integrating and optimizing AI investments for success. Learn More
Dremio, the open data lakehouse vendor that combines the capabilities of a data lake and warehouse on a unified layer, is going all-in on generative AI.
The company today announced two new gen AI capabilities for its platform: a text-to-SQL experience for conversational querying of data, and an autonomous semantic layer to help with data cataloging and processing.
>>Follow VentureBeat’s ongoing generative AI coverage<<
The offerings will simplify working with data for Dremio users, enabling them to explore, discover and analyze their data assets quickly and easily. Similar efforts have been made by other leading players in the data ecosystem (including Snowflake and Informatica), signaling the rise of AI-driven data handling.
Event
Transform 2023
Join us in San Francisco on July 11-12, where top executives will share how they have integrated and optimized AI investments for success and avoided common pitfalls.
How will the new features help?
Unlocking value from data has long been dependent on a number of manual and time-consuming steps. With the latest capabilities, Dremio is using generative AI to address some of these gaps.
For instance, with the new Text-to-SQL experience, instead of spending time writing complex SQL (structured query language) queries, users can simply put in natural language inputs to get insights from their data. The offering uses a semantic understanding of metadata and data, and automatically converts the plain-language query into SQL, providing the desired results.
Similarly, the autonomous semantic layer uses generative AI to eliminate the hassle of manual data cataloging. It automatically learns the intricate details of users’ data and produces descriptions of datasets, columns and relationships to establish taxonomies for easy discovery and exploration of data.
According to Dremio, this layer also learns from users’ workloads and creates reflections (optimized materialization of source data or a query) to accelerate data processing.
“By integrating generative AI capabilities into our platform, we are accelerating data workflows and eliminating much of the manual work involved in SQL development, data catalog creation and curation, and more,” Tomer Shiran, cofounder and CPO of Dremio, said.
“Generative AI will transform data engineering, data science and analytics over the coming years, and we are excited to provide our users with the industry’s most powerful tools to uncover the true potential of their data,” he added.
Vector database capabilities
In addition to the generative AI tooling, Dremio is integrating vector database capabilities directly into its lakehouse, enabling companies to build AI-powered applications without creating additional data silos.
With this feature, users will be able to add a column of type “vector” to store and search embeddings for various data elements.
For instance, if a user has a table of Amazon reviews, they will be able to store the embeddings that encode the meaning of each review alongside other attributes. Then, when required, they could use Dremio’s indexes and SQL functions to retrieve similar or related reviews based on their meanings.
The text-to-SQL experience is now available for Dremio users, while the autonomous semantic layer and vector database capabilities will be rolling out at a later stage.
>>Don’t miss our special issue: Building the foundation for customer data quality.<<
VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.
Author: Shubham Sharma
Source: Venturebeat
