TSBS IoT: methodology
A truck fleet's location and diagnostics, streamed in and queried like a fleet dashboard.
What we ran
We use TSBS's data generator, unchanged, and its 12 query types, ported to SQL every system runs. We don't use TSBS's loaders or query runners: timebench loads the data and times the queries, so the numbers don't compare with published TSBS results.
The original
The Time Series Benchmark Suite (TSBS), by Timescale, generates data for a use case, loads it into a database, and runs a fixed set of query types. Its IoT use case models a fleet of trucks: a readings table (location, speed, fuel use) and a diagnostics table (fuel level, load, status), each truck described by tags.
TSBS measures bulk-load throughput and query throughput with many parallel workers against a database server. We keep its data and queries, and change how data arrives and what we time, because a lake receives data in small batches and is read by short-lived processes.
Data
- TSBS's own generator,
tsbs_generate_data --use-case=iot, seed 123, one reading every 10 seconds per truck from 2026-01-01 to 2026-01-04. - Standard scale: 500 trucks, 11.7M readings and 11.7M diagnostics. The generator leaves about 10% of points missing and delivers some late, on purpose.
- Converted once to Parquet with the same schema TSBS loads into TimescaleDB:
readings,diagnostics, and atagstable.
How the data arrives
Rows are streamed in 5-minute windows, the default buffer interval of Amazon Data Firehose: 864 windows, one commit per table per window, 1,728 commits of about 13,500 rows. Every system gets identical rows in the same order. The tags table is loaded once first.
Queries
- The 12 TSBS IoT query types: last location, low fuel, high load, stationary trucks, long driving sessions, long daily sessions, average versus projected fuel use, average daily driving duration and session, average load, daily activity, breakdown frequency.
- Ported once from TSBS's TimescaleDB SQL to SQL every engine runs. Random parameters are fixed (fleet South, windows near the end of the data). TimescaleDB's
LATERAL … LIMIT 1becomesROW_NUMBER() … = 1. Time buckets are placeholders filled per engine. Breakdown frequency counts the intended fraction of rows with status 0: TSBS's version always returns 1. - 5 more queries about one truck (its last hour, all its history, its latest row, its hourly averages) and the fleet's last hour: what a dashboard asks most.
Systems
| System | Writes | Maintenance | Read by |
|---|---|---|---|
| Timelake | Table.append per commit | inline, plus Table.maintain() | Timelake; DuckDB |
| Iceberg (PyIceberg) | append per commit, partitioned by day | rewrite into larger files, as S3 Tables does by default | DuckDB |
| S3 Tables | the same appends through its Iceberg endpoint | its own managed compaction | DuckDB |
| Lance | write_dataset in append mode | compact_files, cleanup_old_versions | DuckDB; DataFusion |
| Parquet files | a new file per commit, folders by day | one sorted file per day | DuckDB; DataFusion |
| TimescaleDB | COPY per commit into hypertables | convert chunks to the columnstore | PostgreSQL |
Where it runs
- AWS S3: tables in an S3 Standard bucket in us-east-2. The weekly runs read and write from an instance in the same region. The runs of 2026-10-13 came from a laptop about 45 ms away, so their times include that trip for every system.
- Local disk: tables on the machine's own SSD. The operating system's file cache stays warm between queries.
- Tiger Cloud: TimescaleDB on Tiger Cloud's smallest tier (shared CPU), loaded in one go rather than streamed, queried from a laptop with a 30 ms round trip. On the other storage, TimescaleDB runs in a container on the benchmark machine.
- S3 Tables runs only on AWS, in its own table bucket. Its compaction runs on AWS's schedule, hours after ingest; the run waits for it before measuring after maintenance.
What the results show
- Query time, right after ingest and after maintenance: the geometric mean of first runs over the queries every system completed.
- Commit time: the median and 95th percentile of the 1,728 commits.
- While ingesting: every 15 seconds a fresh reader runs the 5 single-truck and fleet queries (the dashboard) and checks the newest row it sees.
- Concurrent dashboards: 1, 4, and 16 clients running the dashboard queries for 30 seconds each, after maintenance.
- Cost on AWS: ingest requests per month at this data rate, storage per month, and one query, at S3 list prices.
- Storage after maintenance: files and bytes.
- On S3: requests and bytes per query and per phase.
Compared with published TSBS numbers
Published TSBS results measure a database server loading data as fast as it can and answering many queries in parallel. Ours measure single queries from a fresh process after streaming commits, so the numbers can't be set side by side.
Planned: a run with TSBS's own loader and query runner, once Timelake has a server mode that speaks the PostgreSQL protocol. Those numbers can sit next to published ones.