Performance history

Every change made to speed Timelake up, with what it measured before and after. The Benchmarks page shows the latest numbers; this page keeps the history. Raw measurements are in bench/results/PERFORMANCE.md and the .jsonl files next to it.

All numbers are from a laptop (M1 Pro) over the internet to AWS S3 us-east-2 or Tigris, unless noted. A “first query” runs in a fresh process, so nothing is cached. Single runs vary by 10–20%; where it matters, numbers are medians of many runs.

Summary

Each version is a tag on GitHub: check it out to run the same benchmark on it.

From 0.2.0 to 0.3.0, the first latest query on S3 went from about 510 ms to 240 ms, and on Tigris from about 810 ms to 210 ms. 0.1.0 is the baseline for 0.2.0’s commit numbers.

0.7.0: rollups answer coarser queries and are computed from finer ones

• What changed: a query whose bucket is a multiple of a rollup's (daily from a 1-minute rollup), or that groups by fewer of its columns, aggregates the rollup's buckets again instead of reading the raw data. A table's rollups are refreshed from the narrowest to the widest, each computed from a finer one where that is up to date.
• Measured on 30 days of IIoT readings (1,000 tags) with a 1-minute rollup: daily statistics per tag 2.07 → 0.25 s, 6-hourly totals over all tags 1.10 → 0.11 s, hourly values of 20 tags 144 → 66 ms; creating a daily rollup 5.2 → 1.8 s. Answers equal the raw data's, up to the rounding of sums added in another order.

0.7.0: faster merge of files that alternate series

• What changed: compaction's merge (also used for appends larger than 1 GiB) gathers each output batch in one pass and finds where each run of rows ends by probing ahead (1, 2, 4… rows) before searching. Files that hold every series, such as 5-minute batches, alternate series by series, so most runs are one row.
• Measured: two inputs of 10M rows that alternate: 13.9 → 2.6 s; of 400,000 rows: 532 → 124 ms. Compaction of hourly files with long runs is unchanged.

0.7.0: newest-first scans

• What changed: a query sorted by time, newest first, with a limit reads the newest time partition first, and older partitions only while they can still hold one of the rows it asks for. Setting newest-first.
• Measured on a laptop: on 30 days of IIoT readings, the 100 newest before a past time 298 → 38 ms; on TSBS IoT (3 days), the 10 newest 102 → 80 ms on local disk and 392 → 320 ms on RustFS with 20 ms per request. Queries for one series' newest rows were already fast and stay the same.

0.7.0: pruning with long IN lists

• What changed: DataFusion skips row groups and pages with IN lists of up to 20 values only. For longer lists of literals, Timelake works out from the file footers which row groups and pages hold one of the values and has the scan read only those. Setting in-list-pruning, on by default.
• Measured on the IIoT benchmark's readings, one day, local disk, best of 7: with compacted files, 100 tags 54 → 31 ms and 21 tags 53 → 26 ms; with the small files inline maintenance leaves, 21 tags 74 → 36 ms and 100 tags 93 → 88 ms (their pages each span hundreds of tags).
• Joins: a join's runtime filter, the list of keys of the dimension rows that matched, is used the same way on object storage. IIoT energy (readings joined to 100 tags) on RustFS with 20 ms per request: 54 → 26 ms repeated, 204 → 183 ms first run. On local disk, where filters run while pages are decoded, the extra checks cost more than they saved on other queries (daily-stats 469 → 712 ms), so joins aren't pruned this way there.

0.7.0: block cache requests of at most 1 MiB

• What changed: the block cache splits a long run of missing blocks into GETs of at most timelake_cache_request_bytes (default 1 MiB), fetched at once. Before, a run was one GET however long.
• Measured on RustFS with 20 ms per request and each response limited to 50 MiB/s, as S3 streams one request at roughly 50–90 MiB/s: first queries 126 → 115 ms (geomean), the heaviest up to 16% faster, with 247 → 364 requests across the 17 queries. Without the rate limit the part size made no difference. Parts of 2 MiB gained 2%, 4 and 8 MiB nothing; 512 KiB gained another 4%, but each extra request may need a new connection, which costs more on S3 than here.

0.7.0: block cache fetches separate runs at once

• What changed: a read that needs several runs of uncached blocks, such as three column chunks of a row group, sends their GETs together. Before, it waited for each before sending the next.
• Measured on RustFS on the same laptop with 20 ms per request and 50 MiB/s per response, TSBS IoT standard, four runs each: first queries 162 → 125 ms (geomean), one truck's history 233 → 109 ms, its hourly averages 170 → 89 ms. Repeat queries, served from the cache, are unchanged.

0.7.0: filter pushdown on local files

• What changed: the setting filter-pushdown (auto, true, false). With it on, a scan decodes a row group's filter columns, evaluates the filters, and decodes the other columns only for rows that pass. auto turns it on for local files and off on object storage. Before, it was always off on plain scans.
• Measured on a laptop, TSBS standard scale, each pair run twice. Local disk, repeat queries (geomean): IoT 10.7 → 10.2 ms, DevOps 16.4 → 15.6 ms. Selective group-bys gained 15–40%; queries whose filters keep most rows lost up to 20%.
• Object storage (RustFS with 20 ms per request): each row group takes one more round trip, the filter columns first. First queries took 12–17% longer (geomean 125 → 147 ms at 50 MiB/s per request), and repeats from the cache 5–8% faster. So auto leaves it off there, and true turns it on for readers that mostly repeat queries.

0.6.0: columns encoded in parallel

• What changed: Timelake writes Parquet with its own writer, shared with the WebAssembly writer, instead of iceberg-rust's. Each row group's columns are encoded on threads of their own. The files are byte for byte the same, with the same statistics in Iceberg metadata (checked by a test).
• Measured on the same laptop and data as streaming compaction below, three runs each: compaction 2.37 → 1.17 s; importing the 11.7M rows in one append 6.0 → 4.7 s (two runs each).

0.6.0: streaming compaction

• What changed: compaction merges its input files as sorted streams instead of loading a whole partition, sorting it, and removing copies in memory. See Compaction.
• Measured on a laptop (M1 Pro, local disk): the TSBS IoT readings (11.7M rows, 3 days) appended in 72 hourly files, then timelake maintain rewrote each day into one file. Three runs each:

• The files written are identical: the same rows in the same order.
• Memory now depends on the files merged, about 10 MB per open file here (its row group, held compressed while it is decoded), plus the row group being written, not on the partition's size.
• Of the 2.37 s, reading and merging take 0.44 s; the rest is encoding the output on one thread.
• Tried: read batches of 1,024 to 16,384 rows and reading 1 to 4 batches ahead: the same time, within noise in memory. Data pages of 32 KiB or 128 KiB instead of 1 MiB: barely less memory (347 and 365 MB for one day against 362 MB), so pages stay at 1 MiB.

0.6.0: version check interval

• What changed: a reader that already has a table version checks for a newer one only after the setting version-check-interval (default 1 second, as before). Set it longer, per table or session, to skip the check on queries in between.
• Effect: with the block cache warm, a repeat query on S3 made one request, the check; within the interval it makes none. In the unit test with 100 ms per request, a warm latest went from one round trip to none. Not measured on S3 yet: the benchmark’s repeat runs follow each other within a second, where the check was already skipped.

0.5.0: block cache

• What changed: files a table never changes (data files, manifests, the last-value index) are cached in 1 MiB blocks: in memory by default, and on local disk, shared by processes, with timelake_cache=disk. See Block cache.
• Measured on the benchmark instance (c8gd.2xlarge, us-east-2) on S3, one run per setting (geometric means over the queries):

• Repeat runs send no requests. DevOps at 16 clients stays at 21 queries/s with or without the cache: those queries are limited by computation.
• Tried: 256 KiB blocks. Repeats were the same, and first runs sent more requests (one DevOps query 10 → 83), so 1 MiB is the default.

0.3.0: footers in the head, cold loads in one round trip

• What changed: the head also carries the Parquet footers of the newest files. A cold load lists the table’s metadata files while it reads the head, so it knows at once whether the head is current. A process that already has a table only checks for a newer version.
• Round trips (unit test): a first latest query 2 → 1, a first filtered scan 3 → 2.
• Measured (median of 15 first runs, ms):

“Before” here is the same table without a head, read by the same code.

• Tried and turned off: opening a few connections ahead of the data reads. It saved at most about 20 ms on scans and nothing on point queries, and it costs requests, so it is off by default (timelake_warm_connections).
• Cost option: timelake_version_lookup=hint reads the version hint instead of listing. A first query takes one more round trip (about 60 ms on S3: latest 305 ms) and avoids the LIST. See Requests and cost.

0.3.0: table state in one object

• What changed: after each commit the writer also writes metadata/timelake-head: the table metadata, the live files with their statistics, and the last-value index. A reader gets them in one request instead of following the version hint, metadata, manifest list, and manifests. See Architecture.
• Round trips (unit test): a first latest query 3 → 2, a first filtered scan 6 → 3.
• Measured (first query, ms; S3 mean of two runs, Tigris one run):

0.2.0: two-step commit

• What changed: an append encodes its data files, manifests, manifest list, and last-value index first, uploads them all at once, then makes the one conditional write that commits. The version hint is written in the background.
• Round trips (unit test): an append 5 → 2.
• Measured (the full v0 ingest on S3, 1,728 commits):

Measure it yourself

# 100 commits and 4 queries against any S3-compatible store (a new, empty prefix)
BENCH_URL=s3://bucket/run-1 BENCH_OPTIONS='{"aws_region": "us-east-2"}' \
  uv run python bench/v0/small.py --label s3
 
# Only the queries, 5 first runs each, against what an earlier run wrote (reads only)
BENCH_URL=s3://bucket/run-1 uv run python bench/v0/small.py --label s3 --queries-only --cold-runs 5

--head-format none writes tables without a head, for a “before” on the same code.