Tuesday, February 18, 2025
HomeBig DataDatabricks Follows Cloudera by Adopting Iceberg, Whereas Snowflake Mulls Open Supply Strategy

Databricks Follows Cloudera by Adopting Iceberg, Whereas Snowflake Mulls Open Supply Strategy


A relentless movement of breaking information from the info lakehouse house is making notable tech headlines this week.

On Tuesday, Databricks introduced that it’ll purchase Tabular, a knowledge administration firm based by the creators of Apache Iceberg, Ryan Blue, Daniel Weeks, and Jason Reidfor. The deal was for an unconfirmed sum, however some studies counsel that quantity to be between $1B and $2B (and allegedly outbidding Snowflake). The transfer goals to unify the 2 hottest open-souce lakehouse codecs — Apache Iceberg and Linux Basis Delta Lake — to boost information compatibility throughout completely different codecs.

The prior day, Snowflake – nonetheless coping with the aftermath of final week’s information breach – introduced Polaris Catalog, a vendor-neutral, open catalog for Apache Iceberg. The corporate additionally introduced at its annual person convention that Polaris Catalog can be open sourced within the subsequent 90 days.

So, how do you make sense of all these bulletins and what does this imply to you? 

Iceberg is the Champion within the Desk Format Battle

Databricks placing this a lot worth in Iceberg is proof that Delta Lake has misplaced the desk format conflict, and Iceberg is the clear winner. Iceberg will additional turn into, and can stay, the de facto customary for large-scale information and analytics deployments for the long term. 

Cloudera was a first mover in adopting Iceberg as central and native to our information, analytics, and AI platform – reinforcing our credibility as the most effective vendor to work with once you need managed Iceberg information estates, at scale, throughout all clouds and on-premises. 

How Open is Your Open Supply?

Regardless of its claims because the open information lakehouse firm, Databricks is NOT well-known for being true to open supply. In contrast to Tabular, Databricks has made industrial variations as proprietary implementations of open supply expertise in a bid to retain buyer lock-in, and it’ll stay to be seen if this transfer adjustments that method. 

Cloudera is a impartial get together that manages Iceberg with out vendor lock-in and at scale – in all clouds and on-premises. Cloudera additionally counts as clients lots of the different massive organizations that immediately contribute to the undertaking.  That’s really open supply.

Tabular Does Not Personal Iceberg

Tabular was based by the originators of the Iceberg undertaking. The corporate has about 20% of the Iceberg contributors and committers on workers (corporations like AWS, Google, Dremio, Starburst, Adobe, Apple, Netflix, and extra), which make up the majority of the contributions. It has a wholesome group, in contrast to Delta Lake, and a number of large tech corporations who’re invested in protecting it open supply and vendor impartial.

It is a dangerous and expensive acquisition by Databricks, significantly if the 80% of the committers determine that different committer affiliations weaken the mission to stay open supply for all.

Welcome to the Get together

Cloudera has been forward of this sport for years. Our 2022 open lakehouse place weblog publish was basically the blueprint for the Databricks acquisition announcement. 

Iceberg has, and continues to be, central to Cloudera’s open information lakehouse structure throughout hybrid clouds – not simply one thing for use on the facet. Databricks failed to achieve adoption for Delta Lake from communities and third-party distributors, and now should make this BIG and expensive wager. On the similar time, Snowflake’s Polaris catalog timing exhibits that they’ve been compelled into this house because the market and clients have moved Iceberg because the central desk format for his or her information two years after Cloudera.

They’re each not solely late to affix the get together, however will miss the enjoyable–and alternative–as they play catch as much as these of us who’ve been right here from the beginning. 

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments