Write-Ahead Log
How Tachyon guarantees every acknowledged write survives a crash — the append-only log every write hits before it's confirmed.
A write-ahead log (WAL) is the standard durability mechanism behind databases and search engines alike: before a write is acknowledged to the caller, it's first appended to an on-disk log. If the process crashes before that write makes it into the main data structure, the log is replayed on restart to reconstruct it.
Tachyon's WAL format
Every write (upsert or delete) is appended as a checksummed frame: a record header, a payload length, a CRC32 checksum, then the JSON-encoded operation itself. The checksum is what makes safe recovery from a mid-write crash possible — see Recovery below.
When it's written
Every document upsert or delete hits the WAL before the memtable, and before the request is acknowledged. This is what makes "acknowledged" mean something: once a write returns success, it will survive a crash, because it's already durable on disk independent of whatever's still only in memory.
Fsync policy
Appending to the WAL and fsyncing it are different things — a write can
be appended to the OS's page cache without yet being forced to physical
disk. How often that fsync happens is --sync-interval-ms/
TACHYON_SYNC_INTERVAL_MS:
0(default) — fsync before acknowledging every single write. The safest setting: nothing is ever acknowledged that isn't already durable. Slower under heavy write load, since every write pays a disk-sync round trip.- A positive value — fsync at most once per that many milliseconds instead of on every write, trading a bounded window of potential data loss (writes acknowledged but not yet fsynced, lost only if the OS itself crashes or loses power before the next sync) for materially higher ingest throughput.
See Configuration for the full flag reference.
Recovery
On startup, Tachyon replays the WAL to reconstruct any writes that were
durable but hadn't yet been flushed into a segment. If the log ends
mid-record — the exact shape a crash during a write leaves behind — replay
stops at the first incomplete or checksum-failing frame and truncates the
log there, rather than refusing to start. The result: a crash loses at most
the writes that hadn't yet been fsynced under the configured
--sync-interval-ms, never a partially corrupted index.
Relationship to segments
The WAL is the durability mechanism; segments are the queryable index. New writes land in an in-memory memtable (backed by the WAL) and are only periodically flushed into a new immutable, memory-mapped segment on disk — see Segment Merging for what happens to segments after that, and Persistence for how the two mechanisms fit together end to end.