The Flask /run_transformation endpoint

Cloud Run runs containers that respond to HTTP. The dbt runner becomes a thin Flask app: GET / returns service info, GET /health returns ok, POST /run_transformation runs dbt. That is enough to make the runner triggerable from anywhere with the right credentials.

Three endpoints, three responsibilities

Root status for humans, /health for Cloud Run probes, /run_transformation for the actual dbt build.

cloud_run_dbt/app.py
python
from flask import Flask, jsonify, request

if os.environ.get("K_SERVICE"):
    try:
        import google.cloud.logging
        google.cloud.logging.Client().setup_logging()
    except Exception:
        logging.basicConfig(level=logging.INFO)
else:
    logging.basicConfig(level=logging.INFO)

app = Flask(__name__)

@app.route("/health", methods=["GET"])
def health() -> object:
    return jsonify(status="ok")

The Flask app. Note the lazy-import of google.cloud.logging so local runs do not need GCP credentials. K_SERVICE is set automatically by Cloud Run, which is how we detect we are running in the cloud.

cloud_run_dbt/app.py
python
@app.route("/run_transformation", methods=["POST"])
def run_transformation() -> object:
    from dbt.cli.main import dbtRunner, dbtRunnerResult

    try:
        request.get_json(force=True, silent=True)
        runner = dbtRunner()
        cli_args = ["--project-dir", "dbt", "--profiles-dir", "dbt"]

        runner.invoke(["source", "freshness"] + cli_args)
        result: dbtRunnerResult = runner.invoke(["build"] + cli_args)

        if result.success:
            return jsonify(status="ok", message="dbt build succeeded")
        return jsonify(status="error", message="dbt build failed"), 500
    except Exception as exc:
        logging.exception(exc)
        return jsonify(status="error", message=str(exc)), 500

The run_transformation endpoint invokes dbt source freshness, then dbt build. Returns 200 on success, 500 on failure. Logs every step.

Cloud Run defaults to a short HTTP request timeout. For dbt builds, bump it toward the Cloud Run hard ceiling via gcloud run deploy --timeout 60m. Long-running endpoints are a Cloud Run-specific pattern. The alternative is async (return a job ID, client polls), which the orchestration phase covers.

Quiz: Quiz

Loading practice…