Benchmarks
Numbers only get published here once they can be reproduced by someone other than us, following the methodology below.
We publish a number here only once it can be reproduced by someone other than us — hardware, dataset, and query mix fully disclosed, following the methodology below. See the Roadmap for where the benchmark suite stands.
Planned dataset sizes
Where technically feasible, each benchmark will be run at:
Methodology
The exact instance type, CPU, memory, and disk used for every run will be published alongside the numbers — not just described in prose.
Corpus size, average document size, field cardinality, and where the dataset itself comes from (synthetic vs. real-world) will be documented per benchmark.
Separate numbers for single-term, multi-term, filtered, faceted, and sorted queries — a single blended average hides too much.
Latency and throughput reported at a range of concurrent client counts, not just a single-client best case.
Every non-default flag used for a run (memtable size, merge thresholds, sync interval) will be listed — configuration changes performance.
Each result set will be pinned to a specific Tachyon commit or release, so numbers don't silently drift out of date.
Comparisons
Once Tachyon's own numbers are published, comparative benchmarks will follow the same rule: same hardware, same dataset, equivalent queries, documented configuration for every system under test, no cherry-picked results.