Big-data company dbt Labs Inc. kicked off its annual user conference Coalesce 2024 in Las Vegas this week with a host of updates to its flagship dbt Cloud product, saying they will help cement its status as a “data control plane” for enterprise analytics.
The company said the new services announced at the event support users across every stage of the analytics development lifecycle, introducing what the company says is unrivaled cross-platform flexibility. They include a new artificial intelligence-powered copilot that’s embedded throughout the dbt Cloud platform, enabling more people to contribute to analytics workflows by simplifying various data preparation tasks.
Dbt Labs is the creator of a cloud-based data transformation tool, which can be used by companies to alter data and make it easier to process and analyze. It’s a comprehensive data platform that does everything from consolidating multiple spreadsheets into a single file, filtering inaccuracies in a dataset, and changing the way data is formatted across multiple database systems.
The company pitches dbt Cloud as a kind of “data control plane” that’s designed to assist with every stage of the analytics development lifecycle. It’s compatible with various data warehouse platforms, including Snowflake, Databricks and Google BigQuery.
At Coalesce 2024, the company said the new updates are focused on eliminating problems around data quality and data ownership, as well as data literacy, thereby making its platform more accessible to users without sophisticated technical skills.
The most important new capability is the dbt Copilot, which is described as an “AI engine” that can help workers to accelerate analytics processes by automating numerous tasks that previously always had to be done manually. For instance, the dbt Copilot has the ability to auto-generate tests, documentation and semantic models, the company said.
In addition, there’s a chat interface that allows users to ask questions of their data in natural language, available through the dbt native app in Snowflake, and the ability to bring your own OpenAI application programming interface key. The new capabilities are all available in beta from today, with the exception of the latter, which is generally available now.
Dbt Labs is also adding support for Apache Iceberg, which is an open-source high-performance format for huge analytic tables. Iceberg enables the use of Structured Query Language tables for big data while enabling engines like Spark, Trino, Athena, Databricks, Starburst and Dremio to work safely with the same tables.
In addition, the dbt Mesh tool is gaining support for cross-platform references using the Iceberg table format as the underlying transport layer, the company said. Essentially, this means customers will be able to use dbt Cloud to centrally define and maintain data governance standards across various big data platforms.
For end users, one of the most visible new updates is the new “visual editing experience” that’s being introduced in beta. It’s a low-code drag-and-drop environment that allows users to explore various dbt models for integrating data, part of the company’s effort to democratize the analytics development lifecycle process by making it more accessible.
The new interface will enable downstream users, who generally possess the most business context, to accessibly and safely author analytics code, the company said. Meanwhile, those users who are familiar with SQL can also use the visual editing experience to check their code and visualize the way their dbt models work.
More sophisticated users will likely appreciate the introduction of a more advanced command line interface, generally available now, which is designed to help catch unexpected behavior before new code is moved into production. This will achieve two things, the company said – namely improving code quality and helping companies optimize their compute spending by only materializing correct models.
Other new features include data health tiles, generally available now, which can be embedded into any downstream app to provide real-time context into key trust signals like data freshness and data quality, and auto-exposures with Tableau Software, which incorporates Tableau dashboards directly into dbt lineage. With this, users can more easily orchestrate the creation of end-to-end data pipelines, and be sure that their analysis only uses the most up to date data.
The company also revealed an upcoming integration with Microsoft Corp.’s Power BI tools, allowing customers who have standardized on that company’s ecosystem, to query and analyze their data with more consistent metrics. Finally, it said Teradata Corp. and Amazon Athena have become the latest databases to integrate with dbt Cloud.
Dbt Labs founder and Chief Executive Tristan Handy said the data industry has already made significant progress towards maturity in recent years, but problems still remain around siloed data and a lack of trust.
“There’s still too much duct tape in our operational systems,” he pointed out. “Our announcements this week go a long way toward fixing these gaps with a ‘One dbt experience’ that’s cross-platform, multi-persona, trusted and infused with AI.”