S3 buckets and IAM for Glue

Terraform owns the AWS state. Buckets, roles, Glue catalog, Snowflake trust policy, CodeBuild project. Every resource has a clear name, a clear scope, and a clear destruction path. The course ships a working Terraform configuration that we walk through file by file.

terraform/s3.tf
hcl
resource "aws_s3_bucket" "datalake" {
  bucket = var.datalake_bucket
  force_destroy = false
}

resource "aws_s3_bucket" "scripts" {
  bucket = var.scripts_bucket
  force_destroy = true
}

resource "aws_s3_bucket_versioning" "datalake" {
  bucket = aws_s3_bucket.datalake.id
  versioning_configuration { status = "Enabled" }
}

Three buckets: data, scripts, and CodeBuild artifacts. force_destroy=false on the data bucket prevents an accidental terraform destroy from wiping production data.

terraform/iam.tf (excerpt)
hcl
resource "aws_iam_role" "glue_etl" {
  name = "${var.pipeline_name}-glue-etl"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Effect = "Allow",
      Principal = { Service = "glue.amazonaws.com" },
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "glue_service" {
  role       = aws_iam_role.glue_etl.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}

The Glue role assumes a trust policy that allows the Glue service. The attached policy grants only what the job needs: read raw, write warehouse, talk to the catalog.

AWSGlueServiceRole is the AWS-managed default. For production, attach a custom policy that scopes S3 access to the specific buckets and prefixes. The course shows the inline policy that locks Glue to read/write only what it needs.