dbt v2 Ships DuckDB. Your pip Adapter Is Optional

Geertjan Wielenga’s DuckDB post on September 22 is eight minutes of reading and one sentence of product: dbt v2 is the first dbt that ships a DuckDB adapter in the box. dbt 2.0.0 landed on September 14. The blog is the DuckDB-shaped tour of that release.

If you already pip install dbt-duckdb and point a profile at ./warehouse.duckdb, that workflow is not dead. Python dbt Core is still installable. What changed is the default path: a Rust binary, an ADBC driver that dbt downloads on first run, and a DuckDB build dbt pins instead of whatever pip resolved last Tuesday.

We already argued that DuckDB is the in-process analytics engine and that the catalog can be SQL. dbt is the missing transform client for that file. The installer is what changed this week.

The adapter you already had

dbt-duckdb’s first pull request is dated August 27, 2021. The DuckDB post cites 1.4k GitHub stars. For five years the pitch was rude in a good way: one Python package, one local file, no warehouse invoice. You wrote SELECTs. dbt ordered them. DuckDB built tables into a file on disk.

dbt Labs announced the Fusion engine on May 28, 2025. Two days later a user named ran-codes opened an issue asking for DuckDB. The quote in the DuckDB post is the whole local-dev case: they learned dbt because the DuckDB setup was light. At writing time that issue had 146 heart reactions and 21 thumbs up. Fusion launched without DuckDB in the box.

June 1, 2026 is the first alpha of dbt Core 2.0. A large part of Fusion moved into the dbt-core repository under Apache 2.0. The dbt-fusion repo was archived. Two distributions, same engine, both free to install locally: after September 14 the CLIs are called dbt (proprietary) and dbt-oss (open source). “Fusion” is now mostly the engine name.

The rename is branding. Where the adapter lives is the actual change.

pip was a package. v2 is a driver

In v1 an adapter was a standalone Python package. In v2, adapters live in a Rust monorepo and talk through ADBC. dbt downloads and caches the DuckDB driver the first time you run it. After install dbt there is no second pip line for DuckDB.

ADBC is Arrow’s database connectivity API. The short version for a Python person: the adapter is not import duckdb inside a site-packages tree. It is a driver dbt fetched, talking Arrow buffers to a pinned engine. That is closer to how a warehouse adapter talks than how a notebook talks. If you debug with duckdb.connect('warehouse.duckdb') in IPython while dbt v2 has the same file open, you still have one-writer rules. File locks did not go away because the installer is Rust.

The profile looks like the one you already typed:

my_project:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: ./warehouse.duckdb

That YAML is the continuity. The process behind it is not. v2 is a compiled Rust binary. There is no Python dependency tree to resolve before a run. If your mental model of dbt is “a bunch of packages in a venv,” that model is the v1 story.

dbt’s own DuckDB setup page is clearer than the launch copy about what you lose. The bundled driver does not load DuckDB extensions. httpfs, parquet, spatial: not on the bundled path. To load extensions you install a DuckDB driver with dbc. v2 checks for a system-installed driver first and falls back to the bundle if it finds none.

That is the first Python-shaped gotcha. A notebook that INSTALL httpfs; LOAD httpfs; against a normal DuckDB will not map 1:1 onto the driver dbt stuffed in the binary. If your models read S3, you are in extension land. Install the system driver. Do not assume the bundle grew httpfs because the blog said “built in.”

The docs also say some v1 dbt-duckdb features are not in v2 yet. Feature parity is dbt-fusion#1593. SQL analysis gaps are dbt-fusion#1464. I have not audited either issue tracker for your project. If you rely on an adapter quirk, parse before you cut over.

catalogs.yml is v2 only

v2 adds catalog support the Python adapter does not have. dbt’s DuckDB docs mark it “dbt v2 only.” The legacy Python adapter attached DuckLake through the profile attach block. v2 wants catalogs.yml, catalog-aware materializations, and use_catalogs_v2: true in dbt_project.yml. dbt writes the ATTACH statements.

A DuckLake catalog in the blog’s example:

catalogs:
  - name: local_lake
    type: ducklake
    table_format: default
    config:
      duckdb:
        metadata_path: metadata.ducklake
        data_path: s3://my-bucket/lake

Then a model config with catalog_name = 'local_lake'. Iceberg REST is in the same docs family. Pinning a DuckDB version is what makes read-write Iceberg REST possible, according to the post. pip’s “whatever version resolved” was never a catalog contract.

Iceberg REST in the same catalogs.yml family is the part that needed the pin. The Python adapter could attach what your pip DuckDB happened to support. A REST catalog that writes is a feature on a specific build. If your Iceberg write path worked on dbt-duckdb last month, check the v2 pin before you call that a regression. It may be a different DuckDB.

This is why I keep the DuckLake article in the same week of thinking. DuckLake puts the catalog in SQL. dbt v2 is the transform tool that will emit ATTACH for that catalog instead of asking you to paste it into profiles.yml. If you are still on the Python adapter, you are on attach. That is fine. It is also the old door.

Do not migrate a Glue catalog because a YAML snippet compiled. Register one table. SELECT count(*). Diff it. We already said that about DuckLake. It is still true when dbt is the one running ATTACH.

The manifest as another Parquet dataset

v2 can write project metadata as Parquet instead of only the giant JSON manifest. dbt parse --generate-info-schema drops files under target/info_schema/v1/. The post’s exhibit is a SELECT against a fixture parquet in the dbt repo: model name, materialization, schema. Three views in main.

The reason is size. On a large project manifest.json can be hundreds of megabytes. You load the whole file to ask which models are tables. Parquet is columnar, so DuckDB reads the columns you named. That is an audit script, not a keynote. CI can ask “which models have no tests” without standing up the warehouse.

You can also point DuckDB at those files after a parse and treat the project as a dataset. That is the same habit as picking DuckDB over pandas when the question is SQL over files. The files happen to be your dbt graph.

Column-level lineage is the other parquet. dbt compile with --generate-info-schema --static-analysis strict writes dbt.column_lineage. Local. No dbt platform account. The engine claims a native reading of SQL across dialects, so invalid column references and type mismatches can fail in the editor instead of in DuckDB at run time. The VS Code extension rides that: autocomplete, hover, inline errors. It is on the Marketplace. I have not timed it against your 400-model repo. The architecture is “parse in a native binary, don’t round-trip the warehouse to find a typo.”

Static analysis without executing the query is also a limit. dbt’s setup page flags that v2 infers types and lineage without running SQL, and that local flat files make that analysis incomplete. If a model is read_parquet('data/*.parquet') and the files are not there, the analyzer is guessing. Run the model when the files exist. Do not trust a red squiggle on a glob.

Pin the engine. Then check the functions

v2 ships a pinned DuckDB, not the version pip would have picked for dbt-duckdb. Pinning is how Iceberg REST read-write landed. Pinning also lets dbt push work into the database. Some adapter logic that used to be a SQL macro is a native DuckDB extension function. The example is array_except, exposed as sf_array_except.

If you have macros that assumed DuckDB 1.x behavior, the pin is a compatibility matrix, not a gift. Print the version dbt actually opened. Run one model that uses list functions. Then go home. “Native function” means the SQL you copy from an old dbt-duckdb gist may be the wrong name.

Faster local development, in the post, is the Rust binary plus that SQL comprehension. Parsing and compiling happen inside one executable. That is a real change from resolving a Python tree every time a junior adds a package. It is not a reason to delete the venv you use for notebooks. dbt v2 and your analysis kernel are allowed to be different processes. Keep them that way until you have a reason not to.

A migration that does not start with rm -rf

dbt v1.12 ships an opt-in v2 parser. dbt parse --use-v2-parser is the low-risk step while you are still on v1. If it parses, follow the install guide. dbt-autofix handles a pile of the mechanical edits. There is an upgrade guide. Python dbt Core remains if you would rather not move.

My order, if you already run dbt-duckdb on a laptop:

Monday: dbt parse --use-v2-parser on v1.12. Save the log. Do not install v2 yet.

Tuesday: list every INSTALL/LOAD and every attach in the profile. Those are the extension and catalog edges. Cross them against the bundled-driver limit on the DuckDB setup page.

Wednesday: install v2 in a throwaway directory. Same path: ./warehouse.duckdb copy, not the production file. dbt parse --generate-info-schema. Open the parquet in DuckDB. Confirm model counts match what you already believe.

Thursday: one model that hits S3 or a DuckLake path. System DuckDB driver via dbc if you need httpfs. If it fails, you are still on v1 for that project. That is allowed. The DuckDB post says so.

Friday: only then point catalogs.yml at a lake you can rebuild. Not the one your CFO thinks is the source of truth.

If dbt-autofix rewrites YAML you do not understand, stop and read the diff. Mechanical upgrades that touch profiles.yml and dbt_project.yml are how people lose the attach block they still need. Keep a branch. Keep the v1 lockfile. The post is explicit that Python dbt Core remains. Use that sentence if someone in chat says v2 is mandatory this sprint.

What this is not

It is not a Snowflake killer essay. You can still pay a warehouse. You can still run dbt there. The DuckDB path is develop, test, and publish on your machine. That was true in 2021 with a pip package. It is true in 2026 with a Rust binary and a pin.

It is not “delete pandas.” Dataframes still exist. dbt still emits SQL. The three-tool chooser still holds. Add a question: which dbt is compiling that SQL, the Python one or the binary that pinned DuckDB?

It is not DuckLake 1.0 all over again. DuckLake is the catalog. dbt v2 is a client that knows how to ATTACH it. If you do not have a lake, path: ./warehouse.duckdb is still the whole product. A file is a warehouse if your team fits in the file.

The bundled driver will not load httpfs because someone on a podcast said local-first. Read the setup page. Install dbc when you need extensions. Keep Python dbt Core if a v1 adapter feature is the reason your job runs. Issue 1593 is not a vibe. It is a list.

dbt v2 shipping DuckDB is the adapter moving into the engine. Your models are still SELECT. The pip package is optional, which is different from gone.