Add HTTPS Support with Caddy for All HTTP Services

[josecelano]

Parent Epic: #1 - Roadmap (Section 6: Add HTTPS support)
Related: #270 - Evaluate Caddy for HTTPS Termination (Research Complete)

Overview

Implement official HTTPS support for all HTTP services (Tracker API, HTTP Tracker, Grafana) using Caddy v2.10 as a TLS termination proxy. This integrates the proven Caddy configuration from production deployment into the deployer's Tera templates, enabling automated HTTPS setup for all new deployments.

Background: Issue #270 successfully evaluated Caddy v2.10, proving it works in production with automatic Let's Encrypt certificates, WebSocket support, HTTP/3 (QUIC), and simple configuration (21-line Caddyfile).

Goals

• Integrate Caddy into deployer Tera templates
• Support HTTPS for all HTTP services (Tracker API, HTTP Tracker, Grafana)
• Enable automatic Let's Encrypt certificate management with configurable staging environment
• Add HTTPS configuration to environment schema (service-based TLS)
• Implement security scanning for Caddy in CI/CD
• Document HTTPS setup in user guide
• Add E2E tests for HTTPS functionality

Architecture

Configuration Pattern: Service-based TLS where each service has optional tls field:

{
  "https": {
    "admin_email": "admin@example.com",
    "use_staging": false
  },
  "tracker": {
    "http_api": {
      "bind_address": "0.0.0.0:1212",
      "admin_token": "...",
      "tls": { "domain": "api.torrust-tracker.com" }
    },
    "http_trackers": [
      {
        "bind_address": "0.0.0.0:7070",
        "tls": { "domain": "http1.torrust-tracker.com" }
      }
    ]
  },
  "grafana": {
    "admin_user": "admin",
    "admin_password": "...",
    "tls": { "domain": "grafana.torrust-tracker.com" }
  }
}

Key Features:

• Optional HTTPS per service (selective HTTPS support)
• Ports extracted from existing bind_address fields (no duplication)
• Configurable Let's Encrypt staging environment (use_staging: true for testing)
• Validation: admin_email required if any service has TLS configured

Implementation Phases

1. Template Creation (3-4 hours): Create Caddyfile.tera, update docker-compose templates
2. Configuration DTOs (3-4 hours): Add HttpsConfig, TlsConfig, update service DTOs
3. Template Rendering (3-4 hours): Implement extract_port filter, handle conditional rendering
4. Security Workflow (1 hour): Add Caddy to security scans
5. Documentation (4-5 hours): Create HTTPS setup guide with configuration examples
6. E2E Testing (5-6 hours): Automated + manual tests
7. Schema Generation (30 minutes): Regenerate JSON schema
8. ADR (1 hour): Document decision rationale

Total Estimated Time: 18-21 hours

Detailed Specification

Complete implementation specification with examples, code samples, acceptance criteria, and architectural decisions: docs/issues/add-https-support-with-caddy.md

Reference

• Production deployment: /opt/torrust/ on Hetzner server (46.224.206.37)
• Evaluation: Evaluate Caddy as TLS Proxy for HTTPS Termination #270, docs/research/caddy-tls-proxy-evaluation/
• Security scan: docs/research/caddy-tls-proxy-evaluation/security-scan.md