Those of you who have been following the Modern Data Stack movement will know that Fivetran and dbt have been a combination similar to PB&J. Fivetran is a best-in-class ingestion tool, specializing in the “E”xtract and “L”oad of ELT. Whereas dbt has been a leader in the remaining “T”ransform. They’re complementary.
However, not long ago Fivetran had acquired Tobiko Labs, makers of SQLMesh, the number one competitor in the Analytics Engineering space after dbt acquired SDF Labs. Many thought this was a strategic direction to compete and expand into new markets by purchasing an up-and-coming competitor. Now with dbt, SDF labs, and Tobiko consolidated under one umbrella, the Fivetran + dbt merger has brought together the leading toolsets, apart from Dataform, Google’s equivalent that primarily serves BigQuery environments.
This leads to a larger conversation around acquisitions in the data space. Over the past several years, many companies have been expanding their footprint and have aimed to become THE all-in-one enterprise-grade data platform.
Some examples:
- Google acquired Looker
- Salesforce acquired Tableau and Informatica
- Rivery acquired by Boomi
In addition to pure growth through acquisitions, there are moves by larger players to extend or repackage their platform:
- Microsoft consolidated their separate Azure services rolling out Data Fabric as an all-in-one SKU
- SAP released Business Data Cloud (including SAP Databricks)
- Snowflake released OpenFlow
So, what does this mean?
Ultimately, we’re seeing a rise in consolidated platform offerings, but does that change how customers decide to architect their internal data platform? From my perspective, it does. There are times that customers pick and choose the best-in-class solutions for each part of their architecture, or, in some cases, the most affordable ones to meet their immediate requirements. However, going all in with integrated platforms can have a meaningful payoff. .
Pros
- Better integration
- More negotiating power with vendors
- Additional features / add-ons
- Simplified billing
However, that doesn’t mean it is all roses either.
Cons
- Vendor lock-in
- Restricted to available solutions
- Tied to vendor’s roadmap
That’s not to say there still aren’t bespoke or specialized solutions, but with an ever-increasing focus in enabling AI and reducing costs, companies are looking to platforms that can help them save while advancing toward their goals.
Full Circle
Coming back to the merger, how does the Fivetran and dbt evolution affect the landscape? Well: It’s not just that they’re able to offer end-to-end data solutions with their powers combined. Now, they are leading the charge in open compute.
What this means is you can build out your data infrastructure on their tooling while having the opportunity to choose which hyperscaler (AWS, GCP, and Azure) or data platform (e.g., Snowflake and Databricks) it runs on.
This ability to choose is a unique advantage because it allows you to keep your data pipelines and models as they are, and to migrate if necessary. You have the choice to control your cloud infrastructure and data warehousing, which is a major determinant of costs (compute power, as most people know, is the biggest lever driving costs up or down.)
Into The Unknown of Open Source Tooling
One of the biggest questions by the community is what this means for the future of the open source tooling.
dbt, having grown through the proliferation of dbt Core, their open source solution, has made several decisions which raise concerns for the open-source community around its commitment to open source. The main concern here is the release of DBT Fusion and the halt of future updates to sdf cli after their SDF Labs acquisition.
Meanwhile, the open-source-forward SQLMesh has been acquired by Fivetran, which does not have a history of open-source tooling. At this time, the community is waiting to see how this merger will continue to support or diverge from the dbt and SQLMesh open-source roots.
Conclusion
Fivetran and dbt have been joined at the hip in countless architectures that DI Squared has seen and implemented, and so we understand the bigger implications and benefits of these changes. Personally, I’m excited to see the companies partner together to better align their roadmaps and serve their customers.
Having experience and partnerships with these’s tools, we’re eagerly waiting to see the waves they make as they build off each other’s strengths.
At the same time, while platforms are consolidating, new market challengers are emerging. DI Squared’s goal is to always provide an independent strategic direction that best serves our customers can trust. Whether you are excited or nervous about the new changes, we’d love to talk how this may impact you and help you realize your data platform vision! Sign up for your free consultation to discuss your data challenges and goals.
