Free to read

Data engineering foundations with the medallion pattern

Stop stitching pandas scripts. Rehearse the bronze, silver, gold pattern on a real dataset in DuckDB and pandas before you touch Airflow, Spark, or a cloud warehouse.

Start readingPreview curriculum

Still deciding? Ask first.

Message a mentor about fit, prerequisites, or where to start. Replies come on WhatsApp, usually within a day.

  • Curriculum fit, prerequisites, or where to start
  • Honest answer, no pressure to enroll

Taught by an engineer who has shipped this

  • ISO 27001

    Led the engineering work behind the certification of a regulated EU platform.

  • Series A platform

    Architected the case-management product that became the business a €11.6M round was raised on.

  • 3x faster deploys

    Cut time-to-deploy by migrating to Kubernetes on GCP with deploy-on-merge.

Take a real dataset from raw dump to BI-ready marts using the bronze, silver, gold pattern. All in DuckDB and pandas, all in a single notebook, no cluster or orchestrator required. The foundation every data engineer needs before touching Airflow or Spark.

Bronze, silver, gold warehouse patterns with DuckDB and pandas, rehearsed on a real dataset before you touch Airflow or Spark.

What you'll ship

Real projects, not toy demos.

  • A notebook that ingests a public dataset into a DuckDB bronze table without touching a single value
  • A silver layer that dedupes, types, renames, and standardizes into something a dashboard can trust
  • A gold star schema with fact, dimension, and surrogate-key tables that a BI tool can navigate
  • A quality gate that flags broken joins and domain drift before the dashboard sees them
  • Business marts aggregated from gold that a product manager can open without asking you first
  • Charts rendered from the same marts so engineering and BI never argue about numbers again

What you'll learn

You finish able to:

  • Explain the medallion pattern in terms a backend engineer understands, without the vendor marketing
  • Ingest a messy public dataset into a bronze DuckDB table without altering a byte
  • Clean the bronze layer into silver with typed columns, deduped rows, and standardized names
  • Design a gold star schema with fact and dimension tables and real surrogate keys
  • Gate your pipeline with quality checks that catch bad joins before they reach a chart
  • Publish BI-ready marts and render matching charts so business teams stop duplicating numbers

Curriculum

From raw dataset to BI-ready marts using DuckDB, pandas, and the medallion pattern.

  1. 01

    The medallion pattern and why it exists

    See why stitching pandas scripts fails at the first hard question, and frame the bronze, silver, gold contract you will build in the rest of the course

  2. 02

    Build bronze and silver with DuckDB

    Ingest a real dataset into a bronze table without mutating a byte, then clean it into a silver layer a human can trust on sight

  3. 03

    Gold star schema, marts, and charts

    Model silver into a dimensional schema with surrogate keys, slice it into marts, and render charts so BI and engineering agree on the numbers

Who it's for

Is this for you?

Backend engineers

dragged into "fix the data pipeline" when the pipeline is actually a folder of unmarked scripts

Analysts who code

tired of re-running the same pandas chain every week because nothing is reusable

AI engineers

trying to build RAG or features on top of data that was never modeled properly

Data scientists

who inherit tribal knowledge and want a pattern that survives the next person leaving

FAQ

Common questions.

  • Do I need Spark or a cloud warehouse for this?

    No. The point is to rehearse the pattern on your laptop with DuckDB and pandas before you pay for a cluster. The same pattern scales to Spark, Snowflake, or BigQuery later. The next course in the track shows that jump.

  • Is this different from dbt?

    dbt is one tool that happens to implement medallion-style staging and marts. The pattern is older and tool-agnostic. Once you understand the layered contract, dbt, DuckDB, Spark, and Snowflake all become implementation details.

  • Why DuckDB? Why not just pandas?

    Pandas gives you dataframes. DuckDB gives you SQL on top of those dataframes. Real warehouses speak SQL, so rehearsing on DuckDB means your gold layer code transfers when you move to Snowflake or Postgres without rewriting.

  • Will this help me prep for a data engineering interview?

    Yes. The layered model, star schema, surrogate keys, and quality gates are standard interview territory. The notebook is the practical side of what the interviewer is probing when they ask "how would you model this?".