Goal
Establish reproducible before/after persistence benchmarks so that later
refactors (subissues #1525-05 through #1525-07) can be evaluated against a
concrete performance baseline.
This is subissue 03 of the EPIC #1525 — Overhaul
Persistence.
Subissues #1525-01 (DB compatibility matrix) and #1525-02 (qBittorrent E2E
test) are already merged into develop.
Revised approach: benchmark at the DB driver level
The original spec
proposed benchmarking through the HTTP API with Docker Compose stacks, a
multi-image before/after runner, and concurrent HTTP worker threads. That adds
significant complexity while measuring the wrong thing: each subsequent subissue
changes the persistence layer, not the HTTP layer. HTTP roundtrip, JSON
serialisation, and business logic are all constant across those refactors.
The revised approach benchmarks the Database trait methods directly, reusing
the same driver instantiation already present in
packages/tracker-core/src/databases/driver/.
What gets measured
Every method on the Database trait, grouped by category:
| Category |
Methods |
| Torrent metrics |
save_torrent_downloads, load_torrent_downloads, load_all_torrents_downloads, increase_downloads_for_torrent |
| Aggregate metrics |
save_global_downloads, load_global_downloads, increase_global_downloads |
| Whitelist |
add_info_hash_to_whitelist, get_info_hash_from_whitelist, load_whitelist, remove_info_hash_from_whitelist |
| Auth keys |
add_key_to_keys, get_key_from_keys, load_keys, remove_key_from_keys |
Each method is called --ops N times (default 10) and the resulting
Vec<Duration> is sorted to extract: count, best, median, worst.
The default of 10 is small enough that a full local run stays well under
3 minutes. Pass a larger --ops value when tighter statistics are needed.
What is NOT measured (and why)
- Startup time — not a persistence-layer concern; constant across refactors.
- Concurrent throughput — the existing drivers are synchronous (
r2d2); a
single-threaded loop gives stable, comparable numbers.
- HTTP roundtrip latency — noise relative to what is being refactored.
- Before/after image swapping — unnecessary; the benchmark runs once per
branch and the committed report is the baseline.
Module layout
The benchmark lives inside packages/tracker-core — persistence is entirely
a tracker-core concern, and Cargo workspace packages can expose their own
binaries.
packages/tracker-core/src/bin/persistence_benchmark_runner.rs ← thin entry point
packages/tracker-core/src/bench/
mod.rs ← module doc, re-exports
runner.rs ← CLI args (clap), orchestration, tracing init
driver_bench.rs ← driver setup, measurement loops, RawResults
metrics.rs ← Vec<Duration> → OperationStats (count, best, median, worst)
report.rs ← OperationStats → JSON (serde_json)
types.rs ← newtype wrappers (BenchDriver, Ops, …)
No new Docker Compose files. No image tags. No port discovery.
CLI
cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
--driver sqlite3|mysql # exactly one driver per run
--db-version 8.4 # DB image tag; ignored for sqlite3; default "8.4" for mysql
--ops 10 # samples per operation; default 10
--json-output <path> # default: bench-results.json
Committed baseline
Run once per driver/version combination on the current develop HEAD and
commit the JSON files. Each subsequent subissue reruns the same commands and
commits updated reports alongside the code change — the diff is the comparison.
cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
--driver sqlite3 \
--json-output docs/benchmarks/baseline-sqlite3.json
cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
--driver mysql --db-version 8.0 \
--json-output docs/benchmarks/baseline-mysql-8.0.json
cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
--driver mysql --db-version 8.4 \
--json-output docs/benchmarks/baseline-mysql-8.4.json
Trade-offs vs. the original spec
| Dimension |
Original spec |
This proposal |
| Measures HTTP layer |
Yes |
No |
| Measures DB layer directly |
No |
Yes |
| Signal-to-noise for persistence refactors |
Low |
High |
| Docker Compose files needed |
2 |
0 |
| Docker image building/tagging |
Required |
Not required |
| Before/after comparison |
Built-in CLI flag |
Git diff of committed reports |
| Binary placement |
workspace root |
packages/tracker-core |
| New source modules |
~8 |
~6 |
| PostgreSQL extension path |
Compose + image tag |
Add Driver::PostgreSQL arm |
| Complexity to review |
High |
Low |
Extension to PostgreSQL (subissue #1525-08)
Adding PostgreSQL follows the same pattern as MySQL: a testcontainers image, a
BenchDriver::PostgreSQL variant, and --driver postgresql. The --db-version
flag already accepts any string tag, so --db-version 16 works without CLI
changes.
Acceptance criteria
Goal
Establish reproducible before/after persistence benchmarks so that later
refactors (subissues #1525-05 through #1525-07) can be evaluated against a
concrete performance baseline.
This is subissue 03 of the EPIC #1525 — Overhaul
Persistence.
Subissues #1525-01 (DB compatibility matrix) and #1525-02 (qBittorrent E2E
test) are already merged into
develop.Revised approach: benchmark at the DB driver level
The original spec
proposed benchmarking through the HTTP API with Docker Compose stacks, a
multi-image before/after runner, and concurrent HTTP worker threads. That adds
significant complexity while measuring the wrong thing: each subsequent subissue
changes the persistence layer, not the HTTP layer. HTTP roundtrip, JSON
serialisation, and business logic are all constant across those refactors.
The revised approach benchmarks the
Databasetrait methods directly, reusingthe same driver instantiation already present in
packages/tracker-core/src/databases/driver/.What gets measured
Every method on the
Databasetrait, grouped by category:save_torrent_downloads,load_torrent_downloads,load_all_torrents_downloads,increase_downloads_for_torrentsave_global_downloads,load_global_downloads,increase_global_downloadsadd_info_hash_to_whitelist,get_info_hash_from_whitelist,load_whitelist,remove_info_hash_from_whitelistadd_key_to_keys,get_key_from_keys,load_keys,remove_key_from_keysEach method is called
--ops Ntimes (default10) and the resultingVec<Duration>is sorted to extract:count,best,median,worst.The default of
10is small enough that a full local run stays well under3 minutes. Pass a larger
--opsvalue when tighter statistics are needed.What is NOT measured (and why)
r2d2); asingle-threaded loop gives stable, comparable numbers.
branch and the committed report is the baseline.
Module layout
The benchmark lives inside
packages/tracker-core— persistence is entirelya tracker-core concern, and Cargo workspace packages can expose their own
binaries.
No new Docker Compose files. No image tags. No port discovery.
CLI
Committed baseline
Run once per driver/version combination on the current
developHEAD andcommit the JSON files. Each subsequent subissue reruns the same commands and
commits updated reports alongside the code change — the diff is the comparison.
cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \ --driver sqlite3 \ --json-output docs/benchmarks/baseline-sqlite3.json cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \ --driver mysql --db-version 8.0 \ --json-output docs/benchmarks/baseline-mysql-8.0.json cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \ --driver mysql --db-version 8.4 \ --json-output docs/benchmarks/baseline-mysql-8.4.jsonTrade-offs vs. the original spec
packages/tracker-coreDriver::PostgreSQLarmExtension to PostgreSQL (subissue #1525-08)
Adding PostgreSQL follows the same pattern as MySQL: a testcontainers image, a
BenchDriver::PostgreSQLvariant, and--driver postgresql. The--db-versionflag already accepts any string tag, so
--db-version 16works without CLIchanges.
Acceptance criteria
cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- --driver sqlite3runs to completion and prints a summary.cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- --driver mysql --db-version 8.4runs to completion and prints a summary.docs/benchmarks/.count,best,median, andworstfields.docs/benchmarking.md.cargo test --workspace --all-targetspasses.linter allexits with code0.