TechReaderDaily.com
TechReaderDaily
Live
Software · Data Infrastructure

Duck Typing Redefines Data Warehouses: DuckDB and Lakehouse Converge

The convergence of DuckDB, open table formats, and the lakehouse model is collapsing the old boundary between OLAP engines and data warehouses, proving that if it queries like a warehouse, it is one.

Architecture diagram showing DuckDB's native Delta Lake integration, with query engine, catalog, and object storage layers connected through the Delta protocol. duckdb.org

In June 2024, the DuckDB project shipped native support for reading Delta Lake tables, complementing extensions that already spoke Parquet, Iceberg, and CSV. The change arrived in a minor release, tracked by a blog post on the project's own site. It did not make headlines. What it made was a category problem. The software that added Delta Lake support was not a data warehouse, not a lakehouse engine, and certainly not a cloud platform. It was a single 40-megabyte binary that a data scientist could install with pip install duckdb and point at a directory of Parquet files on a laptop. And yet, with that release, DuckDB could do something that had been the exclusive territory of warehouse-grade systems: participate in the open table format ecosystem as a first-class query engine, reading from and writing to Delta tables through the same protocol that Databricks and its partners had spent years hardening.

Two years on, that category problem has spread across the entire data infrastructure industry. The boundaries that once separated the online analytical processing (OLAP) engine from the data warehouse, the data warehouse from the lakehouse, and the lakehouse from the data lake have become fluid to the point of near-irrelevance. The trigger for this convergence was not a single product launch. It was the simultaneous maturation of three trends: the open table format, led by Apache Iceberg and Delta Lake; the embeddable OLAP engine, of which DuckDB became the exemplar; and the serverless cloud model, which made it plausible to run an analytical query engine without owning a single server. Together, they have produced a situation that database architects are beginning to describe with a borrowed phrase from programming language theory: duck typing. If it can query your data, return aggregates, and serve a BI tool, it is a warehouse, regardless of what the vendor called it at launch.

To understand why this matters, it helps to trace the separation that is now dissolving. For most of the past fifteen years, the data infrastructure industry organized itself around a clean division of labor. On one side sat the data warehouse, an appliance-like system optimized for structured queries against curated tables, with strong governance, ACID transactions, and a price tag that reflected the engineering required to make those guarantees hold at scale. Teradata, Vertica, and eventually Snowflake occupied this tier. On the other side sat the data lake, a low-cost dumping ground for raw files in open formats, running on commodity object storage, queried by engines such as Presto and Apache Spark that were fast but offered none of the transactional guarantees of a warehouse. The lake was cheap and flexible; the warehouse was expensive and reliable. The two were complementary, and the industry's operational playbook involved moving data from one to the other in a ritual known as ETL.

The lakehouse, popularized by Databricks beginning around 2020, proposed to collapse that division by layering warehouse-grade guarantees on top of open-format data in the lake. The key insight was transactional: if you could add ACID semantics to Parquet files through a metadata layer, then the same storage could serve both the raw-data use cases of a lake and the governed-query use cases of a warehouse. Apache Iceberg, Apache Hudi, and Delta Lake became the three competing implementations of that metadata layer, and by 2025, Iceberg had effectively won the standards war. Snowflake, which had built its business on a proprietary storage format, announced broad Iceberg support. Oracle, as Forbes reported in October 2025, shipped its Autonomous AI Lakehouse with native Iceberg integration, a move that analyst Steve McDowell described as Oracle recognizing enterprises' need for flexibility across platforms. The lakehouse had become the default architecture for new analytical deployments, and the open table format was its ratification as an industry standard.

What the lakehouse did not do, however, was change the fundamental model of how queries reached the data. A lakehouse still required a query engine: something that could parse SQL, plan an execution DAG, push filters down to the storage layer, and return results. For years, that engine was a cluster. Spark, Trino, and Databricks' Photon all ran as distributed systems, requiring capacity planning, compute provisioning, and a nonzero amount of operational attention. Then DuckDB appeared, and the operational model turned inside out. Here was an engine that could run analytical queries against Parquet and Iceberg files on a single machine, with no cluster, no daemon, and no configuration file longer than a connection string. It was OLAP that installed like a library.

DuckDB's architectural wager was not that distributed systems were unnecessary. It was that single-machine performance had been radically underestimated. A modern cloud VM with 64 vCPUs and a terabyte of memory could hold most organizations' analytical working sets in RAM, and a columnar engine optimized for that machine could saturate its I/O bandwidth before it exhausted its CPU budget. For workloads below the petabyte threshold, the argument went, distribution was overhead, not necessity. The project's developers called this philosophy "small data" in the same way that a microsurgeon might use the word "small": precise, powerful, and aimed at the 95th percentile of actual analytical queries, which scans millions of rows, not trillions. By mid-2025, DuckDB had become one of the fastest-growing database projects on GitHub, and engineers who had once provisioned Redshift clusters for ad-hoc analytics were running the same workloads on their laptops during a flight.

The duck typing of warehouses began in earnest when MotherDuck, a Seattle-based startup founded by Google BigQuery veterans, raised a $52.5 million Series B and launched a serverless cloud service built on DuckDB. In September 2025, MotherDuck announced a European cloud region, extending its reach into Frankfurt and positioning the service as a direct competitor to Snowflake and BigQuery for workloads that did not require the full machinery of a traditional warehouse. The pitch was audacious in its simplicity: write SQL, point it at your S3 buckets, and pay only for the bytes scanned during query execution. No clusters, no warehouses, no concurrency scaling policies. The underlying engine was the same DuckDB binary that runs on a laptop, now paired with a cloud control plane that handled authentication, caching, and multi-tenancy.

MotherDuck's existence forced a definitional question that the industry is still answering. If a service built on an embedded OLAP engine can accept SQL, query open-format tables in object storage, return results to a BI tool, and bill by the query, in what sense is it not a data warehouse? The traditional answer would point to missing features: materialized views, workload management, row-level security, and the kind of governance catalog that Snowflake had spent years building. But that answer grew weaker with every release. By June 2026, MotherDuck had added agentic data ingestion, letting AI agents construct pipelines that loaded and transformed data without human-authored ETL scripts, according to SiliconANGLE. The service was acquiring features faster than the category definitions could keep up.

The established warehouse vendors were not standing still. At its 2026 Summit in San Francisco, Snowflake presented what it called the "agentic enterprise," a vision in which AI agents interact with governed, cataloged data through the same SQL interfaces that BI tools have used for decades. Snowflake's Horizon Catalog, announced in November 2025, positioned the company's metadata layer as a universal AI catalog spanning both Snowflake-managed data and external Iceberg tables. The message was unmistakable: Snowflake was no longer a proprietary warehouse that occasionally read external data. It was an open lakehouse that happened to have a world-class query optimizer attached.

Databricks, for its part, made an even more dramatic move. At the Data + AI Summit in June 2026, CEO Ali Ghodsi unveiled LTAP, a new architecture that, as Forbes reported, aims to collapse the 40-year-old separation between OLTP and OLAP databases. The acronym stands for Live Table Access Protocol, and the idea is to allow analytical queries to run against transactional data without the traditional extract-and-load pipeline that has defined data engineering for two generations. If successful, LTAP would erase the category boundary not just between lake and warehouse, but between the operational database that records a sale and the analytical database that later counts it.

What connects DuckDB, MotherDuck, Snowflake's Horizon Catalog, and Databricks' LTAP is not a shared codebase. It is a shared assumption that the storage format is now table stakes and the query engine is the product. In 2015, a data warehouse vendor's competitive moat was its proprietary file format, which locked customers in while enabling performance optimizations that open formats could not match. By 2026, that moat has drained. Iceberg and Delta Lake provide parity features, including time travel, schema evolution, and partition evolution, and every major query engine supports them. The differentiation has shifted upward, to the optimizer, the catalog, the governance layer, and the experience of writing and running queries. And because DuckDB proved that an excellent optimizer could fit in a laptop binary, the barrier to entry for a new query engine has dropped to roughly the cost of a single talented engineering team's attention span.

The result is a market in which the definition of a data warehouse has become operational rather than architectural. A warehouse is whatever answers the question "What were our top-selling SKUs by region last quarter?" within a latency budget that the business finds acceptable, with governance controls that the compliance team finds sufficient, at a cost that the finance team does not flag. If a MotherDuck deployment, a Snowflake instance, and a Databricks SQL warehouse all return the same answer from the same Iceberg table in the same S3 bucket, the only remaining question is which one costs less and which one is easier to manage. The category has been duck-typed: the label "warehouse" now describes a behavior, not a design.

This shift has consequences for how organizations think about data architecture. The traditional playbook assumed that a data warehouse was a destination. You built pipelines to load data into it, you tuned schemas for its particular optimizer, and you accepted that migrating away would be a multi-quarter project with significant switching costs. The new playbook, enabled by open table formats and portable query engines, treats the warehouse as a transient shell around persistent data. You store your data in Iceberg on S3 or Azure Data Lake Storage, you point a query engine at it, and if the engine's performance or pricing displeases you, you point a different engine at the same data next week. The data has tenure; the warehouse does not.

The CRN list of the hottest big data tools of 2025 captured this moment in product taxonomy. The list included both Databricks Lakebase, which is a lakehouse platform, and Snowflake Intelligence, which is an AI layer on top of a warehouse that now behaves as a lakehouse. It also included Amazon Aurora DSQL, a serverless distributed SQL database that borrows architectural ideas from both the transactional and analytical worlds. The categories on the list were blurring because the underlying technologies were blurring. When every tool can query open-format data, the question stops being "What kind of database is this?" and becomes "What kind of workload does it serve best?"

There are, to be clear, workloads for which the duck-typing analogy breaks down. A regulated financial institution running month-end close on a trillion-row general ledger is not going to replace its Teradata appliance with a Python library. The high end of the analytical market still demands features that small embeddable engines do not offer: workload isolation, predictable concurrency under hundreds of simultaneous users, fine-grained access controls that integrate with enterprise identity systems, and support contracts that guarantee response times measured in minutes, not days. DuckDB and MotherDuck are not competing for that business, and the vendors that hold it are not losing sleep over a 40-megabyte binary. Yet the high end is a shrinking fraction of the total analytical workload, and the middle of the market, where most organizations live, has been dramatically underserved by the traditional warehouse pricing model.

The pricing model is worth examining because it explains why the duck-typing phenomenon has gained so much velocity. Traditional cloud warehouses charge for a combination of storage and compute, with compute billed by the second or by the credit, and minimum cluster sizes that ensure the meter is always running. Snowflake's virtual warehouses and Databricks' SQL warehouses both follow this model, and for continuous, predictable workloads it can be efficient. For sporadic, bursty, or ad-hoc analytical workloads, however, a per-query pricing model, of the kind that MotherDuck and BigQuery offer, is often cheaper by an order of magnitude. When the data already lives in object storage and the query engine is stateless, there is no reason to pay for idle compute. The duck-typed warehouse is, in economic terms, a bet that the analytical workload of the typical enterprise is bursty enough to make always-on pricing irrational.

One way to measure the reality of this convergence is to watch the integration roadmaps of the major BI tools. Tableau, Power BI, Looker, and Metabase all now speak to multiple query engines, and their product teams track DuckDB and MotherDuck support requests with the same attention they once reserved for new Snowflake features. A BI tool does not care whether the engine behind its SQL endpoint is a distributed cluster or an embedded library; it cares about latency, cost, and the breadth of the SQL dialect. As those metrics converge across engine types, the BI tool's connector list becomes the most honest taxonomy of the market, and that taxonomy is flattening fast.

The open source dimension adds another layer. DuckDB's permissive MIT license means that it can be embedded inside other products without triggering the reciprocal obligations that come with copyleft licenses. This has produced an ecosystem of tools that bundle DuckDB as a query runtime: dashboarding frameworks that run aggregations in the browser via WebAssembly, CLI tools that replace awk for CSV analysis, and ETL platforms that use DuckDB as an intermediate transformation engine. In each case, the tool itself is not a warehouse, but the query capability it provides is warehouse-grade, and the distinction between tool and platform erodes. A dashboard that can run a GROUP BY across 100 million rows does not need a separate warehouse tier; it needs a fast query engine and data stored in a format that engine can read. DuckDB and Iceberg together supply both halves of that equation.

What happens next depends on how the incumbents respond to the lowering of the engine barrier. Snowflake's Horizon Catalog strategy suggests that the company sees governance, not query execution, as its long-term moat. If every engine can query Iceberg tables, the lock-in shifts to the catalog that tells those engines where the data lives, who is allowed to see it, and what transformations have been applied. Databricks' LTAP play suggests a different bet: that collapsing OLTP and OLAP eliminates the pipeline latency that is the primary friction in the current architecture, and that the company that solves that problem owns the next decade of enterprise data. MotherDuck's agentic ingestion suggests a third bet: that AI will make pipeline construction so cheap that the bottleneck shifts to query execution cost, where per-query pricing wins.

All three bets can be right because they address different points on the same spectrum. A multinational bank and a twelve-person startup both run analytical queries, but their constraints differ by orders of magnitude on every axis that matters: data volume, concurrency, regulatory burden, budget. The duck-typing of warehouses does not mean that every engine is interchangeable. It means that interchangeability is now a design choice rather than a technical impossibility, and that the choice of engine is becoming a tactical decision rather than a strategic commitment. The organization that picks a warehouse today can pick a different one tomorrow without moving its data, and that fact changes the negotiating position of every buyer in the market.

The most significant checkpoint to watch in the next twelve months is not a product launch. It is the steady, unglamorous work of the Iceberg and Delta Lake communities on catalog interoperability. If the catalogs become as portable as the data they describe, then the warehouse becomes a truly disposable layer, and the industry will have completed a transformation that began the moment an engineer first ran SELECT COUNT(*) FROM read_parquet('s3://bucket/sales/') inside a DuckDB session and realized that the answer arrived before the coffee finished brewing. The warehouse is no longer a place. It is a verb, and the verb is "query."

Read next

Progress 0% ≈ 10 min left
Subscribe Daily Brief

Get the Daily Brief
before your first meeting.

Five stories. Four minutes. Zero hot takes. Sent at 7:00 a.m. local time, every weekday.

No spam. Unsubscribe anytime · Privacy.