AOF: Append-only with serialised writes

Before the code: what crash-safety actually means

Your store is fast because it lives in memory. The price of that speed is that a single power cut wipes everything. This module fixes that. The trick is simple: keep a flight recorder. Every write the server accepts is appended to a file on disk. After a crash, you replay the file from the top and the server is back. That is the entire append-only log idea, used inside Redis, every SQL database, Kafka, and even git.

Write to memory, log to disk

Every write goes two places at once: the in-memory map for fast reads, and the log file so the map can be rebuilt after a crash.

Quiz: Quiz

Loading practice…

Every mutating command appends the RESP-encoded frame to data.aof. On startup, the file is replayed. Crash-safe: partial trailing frames are detected and skipped.

05-aof-persistence/main.go
go
var aofMu sync.Mutex

func aofAppend(frame []string) error {
    aofMu.Lock(); defer aofMu.Unlock()
    f, err := os.OpenFile(aofPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil { return err }
    defer f.Close()
    _, err = f.Write(encodeRESPArray(frame))
    return err
}

aofMu serialises writes. POSIX O_APPEND is atomic per syscall on Linux/macOS, but we keep the lock for portability and to bracket the encode + write pair.

05-aof-persistence/main.go
go
func aofReplay() (int, int) {
    f, err := os.Open(aofPath)
    if err != nil {
        if os.IsNotExist(err) { return 0, 0 }
        return 0, 0
    }
    defer f.Close()
    r := bufio.NewReader(f)
    replayed, dropped := 0, 0
    for {
        frame, err := parseRESP(r)
        if err == io.EOF { break }
        if err != nil { dropped++; break }
        if arr, ok := frame.([]any); ok {
            args := toStrings(arr)
            _ = handleArgs(args, false)
            replayed++
        }
    }
    return replayed, dropped
}

Replay reads the file with the same parseRESP we use for incoming network frames. Partial trailing data is detected and skipped.

Our aofAppend opens, writes, closes per command. That gives durability through the OS page cache but does not fsync to disk. Add f.Sync() before close for appendfsync always semantics. Or run a ticker every second for appendfsync everysec.