Graceful shutdown: watch + Semaphore

Before the code: what graceful shutdown actually means

Picture a restaurant at closing time. The good waiter does not yell stop and dump plates on the floor. They stop seating new guests, let the people already eating finish, then turn the lights off. That is graceful shutdown for a server. When the operating system sends a stop signal, you want the server to stop accepting new connections, let the writes already in flight finish, then quit. In Rust on tokio you wire this with tokio::signal, a watch channel that fans the signal to every task, and a Semaphore that counts in-flight work.

Stop accepting, drain, exit

A clean shutdown has three phases: stop accepting new work, let in-flight work finish, then exit the process.

Quiz: Quiz

Loading practice…

Why does graceful shutdown unlock the production-ready path? Because tokio\u2019s task model already gives us what Python had to engineer with selectors. The remaining production gap is shutdown: when SIGTERM arrives, drain cleanly, persist state, then exit.

09-graceful-shutdown/src/main.rs
rust
let (shutdown_tx, shutdown_rx) = watch::channel(false);

// inside main, after the accept loop spawns:
tokio::signal::ctrl_c().await?;
println!("Ctrl+C received, draining...");
let _ = shutdown_tx.send(true);

watch::channel(false) holds a single bool. Every connection clones the Receiver. ctrl_c sets it to true.

09-graceful-shutdown/src/main.rs
rust
loop {
    tokio::select! {
        res = socket.read(&mut tmp) => { /* dispatch commands */ }
        _ = shutdown.changed() => {
            if *shutdown.borrow() {
                let _ = socket.write_all(&encode_error("SHUTDOWN server is shutting down")).await;
                return Ok(());
            }
        }
    }
}

Connection task watches shutdown. On signal, write a -SHUTDOWN error and exit. Existing in-flight reads complete their current frame.

Same idea, different crate. watch is in core tokio and we already use sync primitives from there. CancellationToken from tokio_util is the idiomatic choice once you have other reasons to pull tokio_util in. Both work; watch keeps the dependency tree smaller for this workshop.

09-graceful-shutdown/src/main.rs
rust
let max_permits = usize::MAX >> 3;
loop {
    let in_flight = max_permits - tracker.available_permits();
    if in_flight == 0 { break; }
    tokio::select! {
        _ = &mut drain_deadline => {
            println!("timeout, {in_flight} connection(s) still open");
            break;
        }
        _ = tokio::time::sleep(Duration::from_millis(100)) => {}
    }
}

Semaphore tracks in-flight tasks. After signalling shutdown, the main task loops on available_permits with a 10s deadline.

Quiz: Quiz

Loading practice…