dbt project structure and profiles

A dbt project has three files you cannot avoid: dbt_project.yml (project name, model paths, materializations), profiles.yml (database connections by target), and one schema.yml per model directory (sources, tests, docs). Get those right and the rest is just SQL.

How the three core dbt files relate

dbt_project.yml names the project, profiles.yml supplies the warehouse connection, schema.yml documents and tests each model directory.

cloud_run_dbt/dbt/profiles.yml
yaml
pipeline_ecommerce:
  target: dev
  outputs:
    dev:
      type: bigquery
      method: oauth
      project: "{{ env_var('GCP_PROJECT_ID') }}"
      dataset: "{{ env_var('BQ_DATASET', 'ecommerce_marts') }}"
      location: "{{ env_var('BQ_LOCATION', 'US') }}"
      threads: 4
    ci:
      type: duckdb
      path: /tmp/lwp_dbt_ci.duckdb
      threads: 4

Two targets: dev (BigQuery, env-var driven) and ci (DuckDB at /tmp). The CI target lets the smoke test run dbt parse without GCP credentials.

The dbt-duckdb shim profile lets dbt parse --target ci run without GCP. The smoke test uses it to validate the project graph, macros, and variable resolution. Most type errors and ref() typos surface here, before any BigQuery dollars are spent.

Yes. profiles.yml supports unlimited targets. Each environment is a different GCP project or different BigQuery dataset. Switch via dbt run --target prod. Combined with environment-specific service accounts, this is how you keep dev experiments out of production data.

Quiz: Quiz

Loading practice…