Skip to content

feat(tracker-core): add persistence benchmarking (#1525-03) #1710

@josecelano

Description

@josecelano

Goal

Establish reproducible before/after persistence benchmarks so that later
refactors (subissues #1525-05 through #1525-07) can be evaluated against a
concrete performance baseline.

This is subissue 03 of the EPIC #1525 — Overhaul
Persistence
.
Subissues #1525-01 (DB compatibility matrix) and #1525-02 (qBittorrent E2E
test) are already merged into develop.


Revised approach: benchmark at the DB driver level

The original spec
proposed benchmarking through the HTTP API with Docker Compose stacks, a
multi-image before/after runner, and concurrent HTTP worker threads. That adds
significant complexity while measuring the wrong thing: each subsequent subissue
changes the persistence layer, not the HTTP layer. HTTP roundtrip, JSON
serialisation, and business logic are all constant across those refactors.

The revised approach benchmarks the Database trait methods directly, reusing
the same driver instantiation already present in
packages/tracker-core/src/databases/driver/.

What gets measured

Every method on the Database trait, grouped by category:

Category Methods
Torrent metrics save_torrent_downloads, load_torrent_downloads, load_all_torrents_downloads, increase_downloads_for_torrent
Aggregate metrics save_global_downloads, load_global_downloads, increase_global_downloads
Whitelist add_info_hash_to_whitelist, get_info_hash_from_whitelist, load_whitelist, remove_info_hash_from_whitelist
Auth keys add_key_to_keys, get_key_from_keys, load_keys, remove_key_from_keys

Each method is called --ops N times (default 10) and the resulting
Vec<Duration> is sorted to extract: count, best, median, worst.

The default of 10 is small enough that a full local run stays well under
3 minutes. Pass a larger --ops value when tighter statistics are needed.

What is NOT measured (and why)

  • Startup time — not a persistence-layer concern; constant across refactors.
  • Concurrent throughput — the existing drivers are synchronous (r2d2); a
    single-threaded loop gives stable, comparable numbers.
  • HTTP roundtrip latency — noise relative to what is being refactored.
  • Before/after image swapping — unnecessary; the benchmark runs once per
    branch and the committed report is the baseline.

Module layout

The benchmark lives inside packages/tracker-core — persistence is entirely
a tracker-core concern, and Cargo workspace packages can expose their own
binaries.

packages/tracker-core/src/bin/persistence_benchmark_runner.rs   ← thin entry point
packages/tracker-core/src/bench/
  mod.rs           ← module doc, re-exports
  runner.rs        ← CLI args (clap), orchestration, tracing init
  driver_bench.rs  ← driver setup, measurement loops, RawResults
  metrics.rs       ← Vec<Duration> → OperationStats (count, best, median, worst)
  report.rs        ← OperationStats → JSON (serde_json)
  types.rs         ← newtype wrappers (BenchDriver, Ops, …)

No new Docker Compose files. No image tags. No port discovery.


CLI

cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
    --driver sqlite3|mysql      # exactly one driver per run
    --db-version 8.4            # DB image tag; ignored for sqlite3; default "8.4" for mysql
    --ops 10                    # samples per operation; default 10
    --json-output <path>        # default: bench-results.json

Committed baseline

Run once per driver/version combination on the current develop HEAD and
commit the JSON files. Each subsequent subissue reruns the same commands and
commits updated reports alongside the code change — the diff is the comparison.

cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
    --driver sqlite3 \
    --json-output docs/benchmarks/baseline-sqlite3.json

cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
    --driver mysql --db-version 8.0 \
    --json-output docs/benchmarks/baseline-mysql-8.0.json

cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- \
    --driver mysql --db-version 8.4 \
    --json-output docs/benchmarks/baseline-mysql-8.4.json

Trade-offs vs. the original spec

Dimension Original spec This proposal
Measures HTTP layer Yes No
Measures DB layer directly No Yes
Signal-to-noise for persistence refactors Low High
Docker Compose files needed 2 0
Docker image building/tagging Required Not required
Before/after comparison Built-in CLI flag Git diff of committed reports
Binary placement workspace root packages/tracker-core
New source modules ~8 ~6
PostgreSQL extension path Compose + image tag Add Driver::PostgreSQL arm
Complexity to review High Low

Extension to PostgreSQL (subissue #1525-08)

Adding PostgreSQL follows the same pattern as MySQL: a testcontainers image, a
BenchDriver::PostgreSQL variant, and --driver postgresql. The --db-version
flag already accepts any string tag, so --db-version 16 works without CLI
changes.


Acceptance criteria

  • cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- --driver sqlite3 runs to completion and prints a summary.
  • cargo run -p bittorrent-tracker-core --bin persistence_benchmark_runner -- --driver mysql --db-version 8.4 runs to completion and prints a summary.
  • One JSON report per driver/version combination is committed under docs/benchmarks/.
  • Each JSON report records: git revision, driver name, db-version, ops count, timestamp; each operation entry has count, best, median, and worst fields.
  • Workflow documentation (how to run, how to interpret results) is added to docs/benchmarking.md.
  • cargo test --workspace --all-targets passes.
  • linter all exits with code 0.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions