Benchmarks

Does Timelake beat what a team would otherwise build for time series on object storage? We stream the same IoT data into Timelake, naive Iceberg (the S3 Tables write pattern) and Lance, then run the same queries against each.

Workload

TSBS IoT: readings and diagnostics, 11.7M rows each over 3 days, 4,234 trucks, plus a tags table.

Ingest: 864 five-minute windows × 2 tables = 1,728 commits of about 13.5k rows (Firehose's default buffer interval). Every system gets identical rows.

Queries and states

The 12 TSBS IoT queries plus 5 single-truck queries. Each runs in a fresh process: one first run, then repeats.

Measured right after ingest (what readers see until a compaction job runs) and after each system's standard maintenance. All row counts matched across engines.

Environment

One M1 Pro laptop (10 cores). “S3” is RustFS behind a proxy that adds 20 ms before each response, roughly S3 in-region; it is not AWS.

Naive Iceberg is PyIceberg appends read by DuckDB; its maintenance simulates S3 Tables' default binpack compaction. Lance is read by DuckDB and by DataFusion with Lance's scanner.

Caveats

  • One dataset and one machine. Real S3 adds bandwidth limits and latency variance that the proxy does not.
  • S3 Tables' managed compaction runs asynchronously and can sort if a sort order is configured; here it is binpack only, with all snapshots kept.
  • Lance compaction on RustFS needed upload concurrency 1 (concurrent multipart uploads failed), so Lance's S3 maintenance time is not comparable.
  • Timelake keeps replaced files for 1 hour (snapshot retention), so storage on disk exceeds the live table size until then.
  • Timelake's reader reuses a table version loaded in the last second for planning; its own commits are always visible.