# Persistence
URL: /docs/persistence

How Tachyon keeps data durable and recovers from a crash.



Tachyon persists everything under a single `--data-dir` (`TACHYON_DATA_DIR`,
default `./data`). Two mechanisms work together: a write-ahead log for
durability, and immutable, memory-mapped segments for the actual index.

## Write-ahead log [#write-ahead-log]

Every write (upsert or delete) is appended to a write-ahead log before it's
acknowledged. On startup, Tachyon replays the WAL to reconstruct any data
that hadn't yet been flushed into a segment. If the log ends mid-record —
say, a crash during a write — replay stops at the first incomplete or
checksum-failing record and truncates there, rather than failing to start.

How often the WAL is fsynced to disk is controlled by
`--sync-interval-ms`/`TACHYON_SYNC_INTERVAL_MS`:

* `0` (default) — fsync before acknowledging every write. Safest, slower
  under heavy write load.
* a positive value — fsync at most once per that many milliseconds, trading
  a bounded window of potential data loss for higher ingest throughput.

## Segments [#segments]

New documents accumulate in an in-memory memtable. Once it holds
`--max-memtable-docs`/`TACHYON_MAX_MEMTABLE_DOCS` documents (default
`100,000`), it's flushed into a new immutable, memory-mapped segment on
disk — postings, columnar field values, and document bodies are all read
lazily and decoded only for what a given query actually touches, which is
what keeps memory usage bounded by how much of the corpus is "hot" rather
than by total corpus size.

A search fans out across the current memtable plus every committed segment,
so segment count drives query latency up over time independent of corpus
size. To bound that, Tachyon merges segments in the background: once a
collection holds more than `--merge-trigger-segments`/`TACHYON_MERGE_TRIGGER_SEGMENTS`
(default `8`), the smallest `--merge-fan-in`/`TACHYON_MERGE_FAN_IN` (default
`4`) segments by document count are folded into one, right after the flush
that crossed the threshold.

<Callout type="warn">
  A merge holds everything it's folding together in memory at once while it
  runs. On a memory-constrained deployment, `--merge-fan-in` is the knob that
  trades merge memory usage against how many segments accumulate between
  merges.
</Callout>

## Recovery [#recovery]

Because the WAL is replayed on startup and segments are immutable once
written, a crash mid-write loses at most the writes that hadn't yet been
fsynced under the configured `--sync-interval-ms` — never a partially
corrupted index.
