Refresh patterns and keeping Snowflake current

Three refresh modes. Auto on the Catalog Integration interval. Manual via ALTER ICEBERG TABLE ... REFRESH. Triggered after Glue commits via a Snowflake task or external scheduler. Pick by how fresh dashboards need to be.

snowflake-refresh.sql (illustrative)

sql

ALTER ICEBERG TABLE WAREHOUSE_WEATHER_DATA_DB.PUBLIC.weather_actual_data_timeseries REFRESH;

Force a refresh from Airflow at the end of a Glue run. Snowflake re-reads the latest snapshot from Glue and starts serving it immediately.

Expose freshness as a column. Add a loaded_at timestamp to every Iceberg row, written at Glue commit time. Dashboards then SELECT MAX(loaded_at) and surface it. Users see whether they are looking at fresh data without your help.

Compare MAX(loaded_at) from Snowflake to the last Glue commit time. The difference should be under one refresh interval. Bigger gap means a refresh failed or the Catalog Integration is paused. Wire this into a CloudWatch alarm or a Snowflake task.