AOF: tokio::fs and the Option<&AofFile> pattern

Before the code: what crash-safety actually means

Your store is fast because it lives in memory. The price of that speed is that a single power cut wipes everything. This module fixes that. The trick is simple: keep a flight recorder. Every write the server accepts is appended to a file on disk. After a crash, you replay the file from the top and the server is back. That is the entire append-only log idea, used inside Redis, every SQL database, Kafka, and even git.

Write to memory, log to disk

Every write goes two places at once: the in-memory HashMap for fast reads, and the log file so the map can be rebuilt after a crash.

Quiz: Quiz

Loading practice…

Every mutating command appends the RESP-encoded frame to data.aof via tokio::fs. On startup, the file is replayed. Crash-safe: partial trailing frames are detected and the byte count is reported.

05-aof-persistence/src/main.rs
rust
type Store = Arc<Mutex<StoreInner>>;
type AofFile = Arc<Mutex<File>>;

AofFile is just an alias. Arc<Mutex<File>> serialises writes across tasks.

05-aof-persistence/src/main.rs
rust
async fn handle_args(args: &[String], store: &Store, aof: Option<&AofFile>) -> Vec<u8> {
    // ... handle the command, compute reply ...
    if let Some(aof) = aof {
        if mutating && !reply.starts_with(b"-") {
            let mut f = aof.lock().await;
            let payload = encode_array_strings(args);
            let _ = f.write_all(&payload).await;
        }
    }
    reply
}

handle_args takes Option<&AofFile>. Some(aof) during normal serving; None during replay so we do not re-append.

Option<&AofFile> is cleaner than a boolean flag because the Option carries the resource. There is no way to pass true without also passing the file. The type system makes the contract: either you have an AOF and you log, or you do not and you do not.

05-aof-persistence/src/main.rs
rust
async fn aof_replay(path: &PathBuf, store: &Store) -> io::Result<(u64, usize)> {
    if !tokio::fs::try_exists(path).await? { return Ok((0, 0)); }
    let mut f = File::open(path).await?;
    let mut buf = Vec::new();
    f.read_to_end(&mut buf).await?;
    let mut cur = 0;
    let mut replayed = 0u64;
    let mut dropped = 0usize;
    while cur < buf.len() {
        match parse_resp(&buf[cur..]) {
            Ok((value, consumed)) => {
                cur += consumed;
                if let RespValue::Array(arr) = value {
                    let args = args_as_strings(&arr);
                    let _ = handle_args(&args, store, None).await;
                    replayed += 1;
                }
            }
            Err(_) => { dropped = buf.len() - cur; break; }
        }
    }
    Ok((replayed, dropped))
}

Replay reads the file once, parses RESP frames, calls handle_args with None. Trailing partial bytes are reported, not crashed on.

tokio::fs::File::write_all returns when the bytes reach the OS page cache. For full durability, add f.sync_all().await for each write (appendfsync always) or run a periodic ticker that calls sync_all (appendfsync everysec). The exercises walk through both.