Fix shutdown message and improve it

[josecelano]

Relates to: #1476 (comment)

1 Problem: graceful shutdown message shown even when the server has finished

That message is only for axum servers. The application continues to display the message even if all Axum servers are successfully shut down, but the main tracker cannot stop because there are other pending jobs, such as event listeners.

We should:

• Show the socket address.
• Stop the loop after 90 seconds.
• Stop the loop when there are no connections.

How to reproduce

Change the main.rs file to this:

use tokio::time::{sleep, Duration};
use torrust_tracker_lib::app;

#[tokio::main]
async fn main() {
    let (_app_container, mut jobs) = app::run().await;

    // Add a task that just waits for 120 seconds to allow other tasks to finish.
    let sleep_job = tokio::spawn(async {
        sleep(Duration::from_secs(120)).await;
    });

    jobs.push(sleep_job);

    // handle the signals
    tokio::select! {
        _ = tokio::signal::ctrl_c() => {
            tracing::info!("Torrust tracker shutting down ...");

            // Await for all jobs to shutdown
            futures::future::join_all(jobs).await;

            tracing::info!("Torrust tracker successfully shutdown.");
        }
    }
}

That will add an extra job that waits 2 minutes to give other jobs time to finish. You will see this message forever:

2025-04-25T09:36:27.870333Z  INFO graceful_shutdown: torrust_axum_server::signals: remaining alive connections: 0
2025-04-25T09:36:27.870362Z  INFO graceful_shutdown: torrust_axum_server::signals: remaining alive connections: 0
2025-04-25T09:36:27.870388Z  INFO graceful_shutdown: torrust_axum_server::signals: remaining alive connections: 0

2 Improvements

We should show the job that is preventing the tracker from shutting down.

Torrust tracker shutting down ...

Waiting for all jobs to finish ...

Waiting for the HTTP core event listener to finish ...
Waiting for the HTTP core event listener to finish ...
Waiting for the HTTP core event listener to finish ...

We can implement it by giving the jobs a name.

This is a draft, non-tested implementation from ChatGPT 4o:

use torrust_tracker_lib::app;
use tokio::time::{sleep, Duration};
use tokio::task::JoinHandle;
use std::sync::{Arc};
use std::sync::atomic::{AtomicBool, Ordering};

#[tokio::main]
async fn main() {
    let (_app_container, raw_jobs) = app::run().await;

    // Create flags for each job
    let mut job_flags = Vec::new();
    let mut wrapped_jobs = Vec::new();

    for (i, job) in raw_jobs.into_iter().enumerate() {
        let running = Arc::new(AtomicBool::new(true));
        let running_clone = running.clone();
        job_flags.push((format!("Job {}", i + 1), running.clone()));

        let handle: JoinHandle<()> = tokio::spawn(async move {
            job.await;
            running_clone.store(false, Ordering::SeqCst);
        });

        wrapped_jobs.push(handle);
    }

    // Spawn the status printer
    let status_handle = tokio::spawn({
        let job_flags = job_flags.clone();
        async move {
            loop {
                sleep(Duration::from_secs(5)).await;

                let still_running: Vec<_> = job_flags
                    .iter()
                    .filter(|(_, flag)| flag.load(Ordering::SeqCst))
                    .collect();

                if still_running.is_empty() {
                    break;
                }

                println!("!! shutting tracker !!");
                for (name, _) in still_running {
                    println!("Waiting for {name} to finish ...");
                }
                println!();
            }
        }
    });

    // Wait for CTRL+C
    tokio::signal::ctrl_c().await.unwrap();
    tracing::info!("Torrust tracker shutting down ...");

    // Wait for all job handles
    for handle in wrapped_jobs {
        let _ = handle.await;
    }

    // Wait for the status task to exit
    let _ = status_handle.await;

    tracing::info!("Torrust tracker successfully shutdown.");
}

However, I would prefer to:

• Use a new JobManager type that holds:
  • The "raw" jobs
  • The job names
  • I could be consumed in the main function to return the job names and job handles.

The final goal is to know and show which jobs are preventing the tracker from shutting down.

3 Future Improvements

Tracker Shutdown Strategy: Centralized vs Decentralized

I see two approaches to handle graceful shutdown of background jobs (cleaners, event listeners, etc.) in the torrust-tracker using Tokio:

Option 1: Decentralized signal handling

Each job independently listens to CTRL+C (tokio::signal::ctrl_c())

✅ Pros:

• Simpler to implement in isolated jobs.
• Jobs are fully autonomous (can run standalone in other contexts).
• No need for extra shutdown coordination logic.

❌ Cons:

• Risk of inconsistent state: some tasks might exit, others might not.
• Hard to handle global failures or propagate stop signals from one job to another.
• Less control from main over the full shutdown process.
• Redundant listeners for the same signal (wasted resources, possible confusion).

Option 2: Centralized signal handling (recommended)

Only the main function listens to CTRL+C, then sends a stop signal to all tasks.

✅ Pros:

• Main acts as a clear supervisor (single source of truth).
• Easier to coordinate shutdown and error handling across jobs.
• More flexible: you can trigger shutdowns from other sources (not just CTRL+C).
• Safer and more robust for long-running services.
• Better if in the future we start new jobs dynamically. For example, run a new tracker instance from an admin panel.

❌ Cons:

• Requires explicit shutdown communication (e.g. broadcast, watch, or Notify).
• Slightly more boilerplate in each job to handle shutdown signals.

Recommendation

Use centralized signal handling: let main listen for shutdown, notify tasks via a shared channel, and wait for all jobs to finish gracefully.

For now, we can continue using option 1 even for the new event listener jobs. I will open a new issue when this is implemented.

cc @da2ce7