feat(tracker-core): introduce sqlx schema migrations (subissue 1525-06)

docs/issues/1525-07-align-rust-and-db-types.md

@@ -84,7 +84,7 @@ ALTER TABLE torrent_aggregate_metrics
 
 PostgreSQL migration files are not created here. They will be added in subissue `1525-08` when
 the PostgreSQL driver is introduced. Following the
-[history-alignment pattern](1525-06-introduce-schema-migrations.md#history-alignment-pattern)
+[history-alignment pattern](1719-1525-06-introduce-schema-migrations.md#history-alignment-pattern)
 established in `1525-06`, subissue `1525-08` creates **all four** migration files for
 PostgreSQL starting from migration 1. PostgreSQL's migration 1 creates the columns as
 `INTEGER` (matching the original schema from the other backends), and migration 4 widens them
@@ -212,7 +212,7 @@ These tests extend the existing driver `#[cfg(test)]` modules.
 ## References
 
 - EPIC: `#1525`
-- Subissue `1525-06`: `docs/issues/1525-06-introduce-schema-migrations.md` — must be completed
+- Subissue `1525-06`: `docs/issues/1719-1525-06-introduce-schema-migrations.md` — must be completed
   first (provides the migration framework)
 - Subissue `1525-08`: `docs/issues/1525-08-add-postgresql-driver.md` — adds PostgreSQL
   migration files including the history-aligned no-op for this migration

docs/issues/1525-08-add-postgresql-driver.md

@@ -771,7 +771,7 @@ Acceptance criteria:
   deferred here)
 - Subissue `1525-03`: `docs/issues/1525-03-persistence-benchmarking.md` — benchmark runner
   (PostgreSQL deferred here)
-- Subissue `1525-06`: `docs/issues/1525-06-introduce-schema-migrations.md` — migration
+- Subissue `1525-06`: `docs/issues/1719-1525-06-introduce-schema-migrations.md` — migration
   framework and history-alignment pattern
 - Subissue `1525-07`: `docs/issues/1525-07-align-rust-and-db-types.md` — fourth migration
   and `NumberOfDownloads = u64`

docs/issues/1525-overhaul-persistence.md

@@ -119,7 +119,7 @@ You can then browse or search it while working in the main repository.
 
 ### 6) Introduce schema migrations
 
-- Spec file: `docs/issues/1525-06-introduce-schema-migrations.md`
+- Spec file: `docs/issues/1719-1525-06-introduce-schema-migrations.md`
 - Outcome: schema changes become explicit, versioned, and testable
 
 ### 7) Align persisted counters and Rust/SQL type boundaries

packages/tracker-core/Cargo.toml

@@ -29,7 +29,7 @@ mockall = "0"
 rand = "0"
 serde = { version = "1", features = [ "derive" ] }
 serde_json = { version = "1", features = [ "preserve_order" ] }
-sqlx = { version = "0.8", features = [ "mysql", "runtime-tokio-native-tls", "sqlite" ] }
+sqlx = { version = "0.8", features = [ "macros", "mysql", "runtime-tokio-native-tls", "sqlite" ] }
 thiserror = "2"
 tokio = { version = "1", features = [ "macros", "net", "rt-multi-thread", "signal", "sync" ] }
 tokio-util = "0.7.15"

packages/tracker-core/migrations/README.md

@@ -1,5 +1,52 @@
 # Database Migrations
 
-We don't support automatic migrations yet. The tracker creates all the needed tables when it starts. The SQL sentences are hardcoded in each database driver.
+The tracker applies schema migrations automatically on startup using
+[`sqlx::migrate!`][sqlx-migrate]. Each backend has its own migration folder:
 
-The migrations in this folder were introduced to add some new changes (permanent keys) and to allow users to migrate to the new version. In the future, we will remove the hardcoded SQL and start using a Rust crate for database migrations. For the time being, if you are using the initial schema described in the migration `20240730183000_torrust_tracker_create_all_tables.sql` you will need to run all the subsequent migrations manually.
+- `migrations/sqlite/` — applied to SQLite databases
+- `migrations/mysql/` — applied to MySQL databases
+
+Migration files are embedded into the binary at compile time and applied in
+timestamp order. The `_sqlx_migrations` table (created automatically on the
+target database) records which migrations have already run, so each migration
+is applied exactly once per database.
+
+## Adding a new migration
+
+1. Pick a UTC timestamp prefix higher than every existing file and **strictly
+   greater than `20250527093000`** (the last legacy migration; see
+   [Upgrading from older versions](#upgrading-from-older-versions)). Use the
+   pattern `YYYYMMDDhhmmss_short_description.sql`. You can either create the
+   file by hand or, if you have [`sqlx-cli`][sqlx-cli] installed
+   (`cargo install sqlx-cli`), run `sqlx migrate add <name>` inside the target
+   backend folder — it only generates the empty file with the right timestamp
+   and has no runtime role.
+2. Create the file under **every** backend folder where the change applies, so
+   the `_sqlx_migrations` history stays aligned across backends.
+3. This project uses the simple, forward-only migration style. Do **not** add
+   `.up.sql` / `.down.sql` pairs — `sqlx` does not allow mixing the two styles
+   in the same folder.
+4. Use SQL syntax supported by `sqlx`'s statement splitter — separate
+   statements with `;` and use `--` for line comments (this applies to both
+   the SQLite and MySQL backends; `#`-style comments are not accepted).
+5. Run the test suite: `cargo test -p bittorrent-tracker-core`. A rebuild is
+   required for the new migration to be embedded into the binary.
+
+## Migration file immutability
+
+Once a migration file has been deployed it must never be modified. `sqlx`
+records each migration's checksum in `_sqlx_migrations`; editing a committed
+migration file causes a checksum-mismatch error on the next startup for any
+database that has already applied that migration. To fix or extend an existing
+schema, add a new migration with a later timestamp.
+
+## Upgrading from older versions
+
+Users of pre-v4 trackers must have applied all three legacy migrations
+(`20240730183000_*`, `20240730183500_*`, and `20250527093000_*`) before
+upgrading. The legacy bootstrap path of `create_database_tables()` detects
+existing schemas without a `_sqlx_migrations` table and seeds the migration
+history so the embedded migrator skips them on subsequent runs.
+
+[sqlx-migrate]: https://docs.rs/sqlx/latest/sqlx/macro.migrate.html
+[sqlx-cli]: https://github.com/launchbadge/sqlx/tree/main/sqlx-cli

packages/tracker-core/migrations/sqlite/20240730183000_torrust_tracker_create_all_tables.sql

@@ -4,7 +4,7 @@ CREATE TABLE
         info_hash TEXT NOT NULL UNIQUE
     );
 
-# todo: rename to `torrent_metrics`
+-- todo: rename to `torrent_metrics`
 CREATE TABLE
     IF NOT EXISTS torrents (
         id INTEGER PRIMARY KEY AUTOINCREMENT,

packages/tracker-core/src/databases/driver/mysql/mod.rs

@@ -1,6 +1,7 @@
 //! The `MySQL` database driver.
 use std::str::FromStr;
 
+use ::sqlx::migrate::Migrator;
 use ::sqlx::mysql::{MySqlConnectOptions, MySqlPoolOptions};
 use ::sqlx::{MySqlPool, Row};
 use torrust_tracker_primitives::NumberOfDownloads;
@@ -14,6 +15,12 @@ mod whitelist_store;
 
 const DRIVER: Driver = Driver::MySQL;
 
+/// Embedded `sqlx` migrator for the `MySQL` backend.
+///
+/// All `.sql` files under `migrations/mysql/` are compiled into the binary at
+/// build time and applied in timestamp order by `MIGRATOR.run(&pool)`.
+pub(super) static MIGRATOR: Migrator = ::sqlx::migrate!("migrations/mysql");
+
 /// `MySQL` driver implementation.
 ///
 /// This struct encapsulates an async `sqlx` connection pool for `MySQL`.
@@ -202,8 +209,92 @@ mod tests {
 
         run_tests(&driver).await;
 
+        // Idempotency: a second `create_database_tables()` call must be a
+        // no-op (embedded `sqlx` migrator skips migrations already recorded
+        // in `_sqlx_migrations`).
+        driver
+            .create_database_tables()
+            .await
+            .expect("second migration run should be a no-op");
+
+        // Legacy bootstrap: simulate a pre-v4 database (no `_sqlx_migrations`
+        // table, all four legacy tables present) and verify
+        // `create_database_tables()` seeds the migration history without
+        // re-running the embedded migrations.
+        driver
+            .drop_database_tables()
+            .await
+            .expect("drop tables before legacy bootstrap test");
+
+        let raw_pool = ::sqlx::mysql::MySqlPoolOptions::new()
+            .connect(&config.database.path)
+            .await
+            .expect("connect to mysql for raw DDL");
+        create_legacy_pre_v4_schema(&raw_pool).await;
+
+        driver
+            .create_database_tables()
+            .await
+            .expect("legacy bootstrap should succeed");
+
+        let recorded: i64 = ::sqlx::query_scalar("SELECT COUNT(*) FROM `_sqlx_migrations`")
+            .fetch_one(&raw_pool)
+            .await
+            .expect("count _sqlx_migrations");
+        assert_eq!(recorded, 3, "all three legacy migrations should be fake-applied");
+
+        // Partial-state rejection: only two of four legacy tables present.
+        driver
+            .drop_database_tables()
+            .await
+            .expect("drop tables before partial-state test");
+        for stmt in [
+            "CREATE TABLE whitelist (id INTEGER PRIMARY KEY AUTO_INCREMENT)",
+            "CREATE TABLE torrents (id INTEGER PRIMARY KEY AUTO_INCREMENT)",
+        ] {
+            ::sqlx::query(stmt).execute(&raw_pool).await.expect("partial DDL");
+        }
+
+        let err = driver
+            .create_database_tables()
+            .await
+            .expect_err("partial legacy state must be rejected");
+        match err {
+            crate::databases::error::Error::LegacyDatabaseNotMigrated { reason, .. } => {
+                assert!(reason.contains("apply every pre-v4 migration"));
+            }
+            other => panic!("unexpected error: {other:?}"),
+        }
+        drop(raw_pool);
+
         mysql_container.stop().await;
 
         Ok(())
     }
+
+    /// Recreate the schema produced by the three pre-v4 manual migrations.
+    ///
+    /// This raw DDL mirrors the cumulative state of
+    /// `migrations/mysql/2024073018*.sql` and
+    /// `migrations/mysql/20250527093000_*.sql` after they have been applied
+    /// in order. We build it by hand so the legacy-bootstrap test path
+    /// can build a database that looks exactly like a pre-v4 tracker on disk
+    /// (legacy tables present, no `_sqlx_migrations` row).
+    ///
+    /// # Legacy compatibility
+    ///
+    /// Drop this helper at the same time as the
+    /// `bootstrap_legacy_schema` function in
+    /// `mysql/schema_migrator.rs` — see the legacy-compatibility note on
+    /// that function.
+    async fn create_legacy_pre_v4_schema(pool: &::sqlx::MySqlPool) {
+        for stmt in [
+            "CREATE TABLE whitelist (id INTEGER PRIMARY KEY AUTO_INCREMENT, info_hash VARCHAR(40) NOT NULL UNIQUE)",
+            "CREATE TABLE torrents (id INTEGER PRIMARY KEY AUTO_INCREMENT, info_hash VARCHAR(40) NOT NULL UNIQUE, completed INTEGER DEFAULT 0 NOT NULL)",
+            "CREATE TABLE `keys` (`id` INT NOT NULL AUTO_INCREMENT, `key` VARCHAR(32) NOT NULL, `valid_until` INT(10), PRIMARY KEY (`id`), UNIQUE (`key`))",
+            "CREATE TABLE torrent_aggregate_metrics (id INTEGER PRIMARY KEY AUTO_INCREMENT, metric_name VARCHAR(50) NOT NULL UNIQUE, value INTEGER DEFAULT 0 NOT NULL)",
+        ] {
+            ::sqlx::query(stmt).execute(pool).await.expect("legacy DDL");
+        }
+    }
 }