Terraform for GCS, IAM, and service accounts

Terraform owns the GCP foundation: the GCS raw bucket, the BigQuery datasets, the service accounts, and the IAM bindings between them. ClickOps in the Cloud Console is fine for one-time exploration. For production, every resource lives in HCL.

terraform/gcs.tf
hcl
resource "google_storage_bucket" "raw" {
  name     = var.gcs_raw_bucket
  project  = var.gcp_project_id
  location = var.region
  uniform_bucket_level_access = true
  versioning { enabled = true }
  lifecycle_rule {
    action { type = "Delete" }
    condition { age = 90 }
  }
}

The raw bucket with versioning and lifecycle rules. uniform_bucket_level_access disables ACLs in favor of pure IAM, which is the modern default.

terraform/iam_service_account.tf
hcl
resource "google_service_account" "dbt_runner" {
  account_id   = "dbt-runner-sa"
  display_name = "dbt runner Cloud Run SA"
  project      = var.gcp_project_id
}

resource "google_project_iam_member" "dbt_bq_user" {
  project = var.gcp_project_id
  role    = "roles/bigquery.user"
  member  = "serviceAccount:${google_service_account.dbt_runner.email}"
}

resource "google_project_iam_member" "dbt_bq_dataeditor" {
  project = var.gcp_project_id
  role    = "roles/bigquery.dataEditor"
  member  = "serviceAccount:${google_service_account.dbt_runner.email}"
}

Two service accounts: dbt-runner (used by Cloud Run) and airflow-trigger (used by Airflow to call Cloud Run). Each gets exactly the roles it needs.

roles/bigquery.user lets the SA run queries. roles/bigquery.dataEditor lets it write to datasets. Together they are enough for dbt builds. Avoid roles/bigquery.admin (overprivileged) and roles/owner (always too broad).