diff --git a/docs/research/pingoo-tls-proxy-evaluation/README.md b/docs/research/pingoo-tls-proxy-evaluation/README.md new file mode 100644 index 00000000..1c7cadc3 --- /dev/null +++ b/docs/research/pingoo-tls-proxy-evaluation/README.md @@ -0,0 +1,128 @@ +# Pingoo TLS Proxy Evaluation + +**Issue**: [#234](https://github.com/torrust/torrust-tracker-deployer/issues/234) +**Specification**: [docs/issues/234-evaluate-pingoo-for-https-termination.md](../../issues/234-evaluate-pingoo-for-https-termination.md) +**Status**: ✅ CLOSED - Not Adopting +**Started**: 2026-01-12 +**Completed**: 2026-01-13 + +## Overview + +This research evaluated [Pingoo](https://pingoo.io/) as a potential replacement for nginx+certbot +for automatic HTTPS/TLS termination in Torrust Tracker deployments. + +**Outcome**: Pingoo works excellently for the Tracker (API + HTTP Tracker) but does not +support WebSocket connections required for Grafana Live. We have decided **not to adopt +Pingoo** at this time and will instead evaluate **Caddy** as a simpler alternative that +supports both HTTP proxying and WebSocket. + +## Test Environment + +- **Server**: Hetzner ccx23, Ubuntu 24.04, nbg1 location +- **IP**: 46.224.206.37 +- **Domain**: torrust-tracker.com (with subdomains) + +### Subdomains + +| Subdomain | Purpose | DNS Status | +| ----------------------------- | -------------------------- | ------------- | +| `test.torrust-tracker.com` | Experiment 1: Hello World | ✅ Propagated | +| `api.torrust-tracker.com` | Experiment 2: Tracker API | ✅ Propagated | +| `http1.torrust-tracker.com` | Experiment 3: HTTP Tracker | ✅ Propagated | +| `grafana.torrust-tracker.com` | Experiment 4: Grafana UI | ✅ Propagated | + +## Phases + +### Phase 1: Environment Preparation + +See [phase-1-environment-preparation.md](phase-1-environment-preparation.md) for: + +- DNS configuration and propagation verification +- Server accessibility checks +- Prerequisites for running experiments + +**Status**: ✅ Complete + +### Phase 2: Experiments + +| Experiment | Document | Status | +| ---------------------- | ------------------------------------------------------------ | -------------------- | +| 1. Minimal Hello World | [experiment-1-hello-world.md](experiment-1-hello-world.md) | ✅ Complete | +| 2. Tracker API HTTPS | [experiment-2-tracker-api.md](experiment-2-tracker-api.md) | ✅ Complete | +| 3. HTTP Tracker HTTPS | [experiment-3-http-tracker.md](experiment-3-http-tracker.md) | ✅ Complete | +| 4. Grafana WebSocket | [experiment-4-grafana.md](experiment-4-grafana.md) | ⚠️ Partial (WS fail) | + +## Key Questions to Answer + +1. Does Pingoo automatically generate Let's Encrypt certificates? **✅ YES** +2. Does certificate renewal work without manual intervention? **⏳ Cannot test (90-day validity)** +3. Does Pingoo support WebSocket connections (needed for Grafana Live)? **❌ NO** +4. How does configuration complexity compare to nginx+certbot? **Much simpler (~10 lines vs ~50+)** +5. Are there any issues with TLS 1.3-only support? **✅ No issues detected** + +## Findings Summary + +### ✅ Successful Tests + +- **Automatic certificate generation** - Pingoo obtained Let's Encrypt certs without manual steps +- **TLS 1.3 with post-quantum cryptography** - Uses X25519MLKEM768 key exchange +- **Minimal configuration** - Only ~10 lines of YAML needed +- **No email required** - Unlike certbot, no email setup needed +- **Tracker API proxying** - Health checks and API endpoints work perfectly +- **HTTP Tracker proxying** - BitTorrent announce/scrape work via HTTPS + +### ❌ Failed Test + +- **WebSocket support** - Pingoo strips the `Upgrade` header, breaking WebSocket connections + - Root cause: `Upgrade` header treated as hop-by-hop header in `http_proxy_service.rs` + - Impact: Grafana Live (real-time streaming) does not work + - Workaround: Use nginx for services requiring WebSocket + +### ⏳ Pending Verification + +- **Certificate renewal** - Cannot test yet (cert valid for 90 days) + +## Final Decision + +**Not adopting Pingoo** - We will evaluate Caddy instead. + +### Why Not Adopt Pingoo? + +While Pingoo works excellently for the Tracker, the WebSocket limitation means we would +need a **hybrid architecture** (Pingoo for Tracker + another proxy for Grafana). This +adds unnecessary complexity for our simple deployment setup: + +1. **Two proxies is overkill** - For a simple Tracker + Grafana setup, maintaining two + different TLS proxies adds operational complexity without significant benefit. + +2. **Caddy as alternative** - [Caddy](https://caddyserver.com/) offers similar benefits + to Pingoo (automatic HTTPS, simple config) while also supporting WebSocket natively. + It can handle both Tracker and Grafana with a single proxy. + +3. **Maturity considerations** - Caddy is more mature and battle-tested than Pingoo. + While Pingoo has attractive features (post-quantum crypto, Rust implementation), + Caddy's stability may be more valuable for production deployments. + +4. **Performance is not the bottleneck** - One concern about Caddy (written in Go) vs + Pingoo (written in Rust) is performance. However, based on running a tracker demo + for a couple of years, the proxy is unlikely to be the bottleneck. Most users prefer + the UDP tracker which doesn't use the proxy at all. + +### Open Issue on Pingoo + +We filed [pingooio/pingoo#23](https://github.com/pingooio/pingoo/issues/23) to confirm +the WebSocket limitation and discuss potential solutions. If Pingoo adds WebSocket +support in the future, it could be reconsidered. + +### Next Steps + +A new research issue will be opened to evaluate **Caddy** as an alternative to +nginx+certbot, following the same experimental approach used here. + +See [conclusion.md](conclusion.md) for full rationale. + +## Timeline + +- **2026-01-12**: Research started, Experiments 1-4 completed +- **2026-01-12**: WebSocket limitation discovered, issue filed on Pingoo repo +- **2026-01-13**: Decision made not to adopt Pingoo, research closed diff --git a/docs/research/pingoo-tls-proxy-evaluation/conclusion.md b/docs/research/pingoo-tls-proxy-evaluation/conclusion.md new file mode 100644 index 00000000..8a0fc597 --- /dev/null +++ b/docs/research/pingoo-tls-proxy-evaluation/conclusion.md @@ -0,0 +1,177 @@ +# Pingoo TLS Proxy Evaluation - Conclusion + +**Status**: ✅ EVALUATION COMPLETE - NOT ADOPTING +**Last Updated**: 2026-01-13 + +## Final Decision + +**Not adopting Pingoo.** We will evaluate Caddy as an alternative instead. + +Pingoo provides excellent TLS termination for HTTP-based services but **does not support +WebSocket connections**, which are required for Grafana Live. Rather than adopting a +hybrid architecture (Pingoo + another proxy), we have decided to evaluate Caddy, which +can handle both the Tracker and Grafana with a single proxy. + +## Why Not Adopt Pingoo? + +### The Hybrid Architecture Problem + +Initially, we considered using Pingoo for the Tracker and nginx/Caddy for Grafana: + +```text +┌─────────────────────────────────────────────────────────────────┐ +│ Public Internet │ +└─────────────────────────────────────────────────────────────────┘ + │ │ + ▼ ▼ + ┌──────────────────┐ ┌───────────────────┐ + │ Pingoo │ │ nginx or Caddy │ + │ (port 443) │ │ (port 3443) │ + └────────┬─────────┘ └────────┬──────────┘ + │ │ + ▼ ▼ + ┌──────────────────┐ ┌───────────────────┐ + │ Tracker API │ │ Grafana │ + │ HTTP Tracker │ │ (WebSocket) │ + └──────────────────┘ └───────────────────┘ +``` + +However, this approach has significant drawbacks for our use case: + +1. **Operational complexity** - Two different TLS proxies means two different + configurations, two sets of certificates to manage, and two potential points of + failure. + +2. **Overkill for simple setups** - For a typical Torrust deployment (Tracker + + Grafana), maintaining two proxies adds unnecessary complexity without clear benefits. + +3. **Better alternatives exist** - Caddy offers similar benefits to Pingoo (automatic + HTTPS, simple configuration) while also supporting WebSocket natively. + +### Caddy as Alternative + +[Caddy](https://caddyserver.com/) is a more mature alternative that can replace +nginx+certbot entirely: + +| Feature | Pingoo | Caddy | +| -------------------- | ----------------- | -------------- | +| Automatic HTTPS | ✅ Yes | ✅ Yes | +| Simple config | ✅ ~10 lines | ✅ ~5-10 lines | +| WebSocket support | ❌ No | ✅ Native | +| Post-quantum crypto | ✅ X25519MLKEM768 | ❌ No | +| TLS versions | 1.3 only | 1.2 and 1.3 | +| Language | Rust | Go | +| Maturity | Newer | Very mature | +| Single proxy for all | ❌ Needs hybrid | ✅ Yes | + +### Performance Considerations + +One concern about Caddy (Go) vs Pingoo (Rust) is performance. However: + +- **Proxy is not the bottleneck** - Based on running a Torrust tracker demo for a couple + of years, the TLS proxy is unlikely to be the performance bottleneck. + +- **UDP tracker dominates** - Most BitTorrent clients prefer the UDP tracker protocol, + which doesn't go through the HTTP/HTTPS proxy at all. + +- **Stability over raw speed** - For a production deployment, Caddy's maturity and + stability may be more valuable than Pingoo's potential performance advantages. + +### What We Lose by Not Using Pingoo + +1. **Post-quantum cryptography** - Pingoo's X25519MLKEM768 key exchange provides + protection against future quantum computers. Caddy doesn't have this yet. + +2. **TLS 1.3-only enforcement** - Pingoo only supports TLS 1.3, which simplifies + security configuration. Caddy supports both 1.2 and 1.3. + +3. **Rust implementation** - Pingoo's Rust codebase may offer better memory safety + and performance characteristics. + +These are trade-offs we accept in favor of a simpler, single-proxy architecture. + +## Open Issue on Pingoo Repository + +We filed [pingooio/pingoo#23](https://github.com/pingooio/pingoo/issues/23) to: + +1. Confirm that WebSocket proxying is not currently supported +2. Discuss potential solutions or workarounds +3. Request consideration for adding WebSocket support + +If Pingoo adds WebSocket support in the future, it could be reconsidered for Torrust +deployments. + +## WebSocket Limitation - Root Cause + +Pingoo's HTTP proxy explicitly removes the `Upgrade` header, which is required for +WebSocket protocol upgrades. From the source code: + +```rust +// https://github.com/pingooio/pingoo/blob/main/pingoo/services/http_proxy_service.rs +const HOP_HEADERS: &[&str] = &[ + "Connection", + // ... other headers ... + "Upgrade", // This breaks WebSocket! +]; +``` + +This means any service requiring WebSocket connections cannot use Pingoo's `http_proxy`. +This is a fundamental limitation, not a configuration issue. + +## Experiment Results Summary + +| Experiment | Status | Result | +| ---------------------- | ----------- | -------------------------------------------- | +| 1. Hello World | ✅ Complete | SUCCESS - Certificate auto-generated | +| 2. Tracker API | ✅ Complete | SUCCESS - API endpoints work via HTTPS | +| 3. HTTP Tracker | ✅ Complete | SUCCESS - BitTorrent announce/scrape working | +| 4. Grafana (WebSocket) | ⚠️ Partial | HTTP works, WebSocket FAILS | + +## Key Findings from Experiments + +### Experiment 1: Hello World + +- ✅ Automatic Let's Encrypt certificate generation works +- ✅ No email or manual steps required +- ✅ TLS 1.3 with post-quantum key exchange (X25519MLKEM768) +- ✅ ECDSA certificate from Let's Encrypt E8 intermediate +- ✅ Certificate stored with domain-named files for easy identification +- ✅ ACME account persisted for future renewals + +### Experiment 2: Tracker API + +- ✅ JSON API responses proxied correctly +- ✅ Health check endpoints work +- ✅ No issues with TLS 1.3 for API clients + +### Experiment 3: HTTP Tracker + +- ✅ BitTorrent `announce` endpoint works via HTTPS +- ✅ BitTorrent `scrape` endpoint works via HTTPS +- ✅ Binary bencoded responses handled correctly + +### Experiment 4: Grafana (WebSocket) + +- ✅ HTTP dashboard access works +- ✅ Login and navigation work +- ❌ **WebSocket fails** - `Upgrade` header stripped by Pingoo +- ❌ Grafana Live (real-time streaming) does not work + +## Next Steps + +1. ✅ ~~Complete Experiment 1 (Hello World)~~ - Certificate auto-generation verified +2. ✅ ~~Complete Experiment 2 (Tracker API)~~ - JSON API proxying verified +3. ✅ ~~Complete Experiment 3 (HTTP Tracker)~~ - BitTorrent protocol verified +4. ✅ ~~Complete Experiment 4 (Grafana)~~ - WebSocket limitation discovered +5. ✅ ~~File issue with Pingoo project~~ - [pingooio/pingoo#23](https://github.com/pingooio/pingoo/issues/23) +6. ✅ ~~Decision made~~ - Not adopting Pingoo, will evaluate Caddy instead +7. 🔲 Open new issue to evaluate Caddy as nginx+certbot replacement +8. 🔲 Run Caddy experiments following similar methodology + +## References + +- [Pingoo Documentation](https://pingoo.io/docs) +- [Pingoo WebSocket Issue](https://github.com/pingooio/pingoo/issues/23) - Our issue asking about WebSocket support +- [Issue #234 - Evaluate Pingoo](https://github.com/torrust/torrust-tracker-deployer/issues/234) +- [Issue Specification](../../issues/234-evaluate-pingoo-for-https-termination.md) +- [Caddy Server](https://caddyserver.com/) - Alternative to evaluate next diff --git a/docs/research/pingoo-tls-proxy-evaluation/experiment-1-hello-world.md b/docs/research/pingoo-tls-proxy-evaluation/experiment-1-hello-world.md new file mode 100644 index 00000000..6aaa0d7b --- /dev/null +++ b/docs/research/pingoo-tls-proxy-evaluation/experiment-1-hello-world.md @@ -0,0 +1,260 @@ +# Experiment 1: Minimal HTTPS Setup (Hello World) + +**Status**: ✅ Complete +**Started**: 2026-01-12 +**Completed**: 2026-01-12 +**Domain**: `test.torrust-tracker.com` + +## Objective + +Test Pingoo certificate generation in isolation with a minimal setup containing only: + +- Pingoo (TLS proxy) +- A simple nginx container serving static "Hello World" content + +This isolates the certificate generation testing from any tracker-specific complexity. + +## Pre-requisites + +- [x] Hetzner server running (IP: 46.224.206.37) +- [x] DNS propagated for `test.torrust-tracker.com` → 46.224.206.37 +- [x] Port 443 accessible on the server + +## Setup + +### Files Created + +```text +/root/experiments/experiment-1/ +├── docker-compose.yml +├── pingoo/ +│ └── pingoo.yml +└── www/ + └── index.html +``` + +### docker-compose.yml + +```yaml +services: + pingoo: + image: pingooio/pingoo:latest + ports: + - "443:443" + volumes: + - ./pingoo:/etc/pingoo + networks: + - test-network + depends_on: + - webserver + + webserver: + image: nginx:alpine + volumes: + - ./www:/usr/share/nginx/html:ro + networks: + - test-network + +networks: + test-network: + driver: bridge +``` + +### pingoo/pingoo.yml + +```yaml +listeners: + https: + address: https://0.0.0.0:443 + +tls: + acme: + domains: ["test.torrust-tracker.com"] + +services: + static: + http_proxy: ["http://webserver:80"] +``` + +### www/index.html + +```html + + +
+If you see this page via HTTPS, Pingoo certificate generation works!
+Certificate info: Check browser padlock for details.
+ + +``` + +## Deployment Steps + +1. SSH to the Hetzner server +2. Create the experiment directory structure +3. Copy the configuration files +4. Run `docker compose up -d` +5. Check Pingoo logs for certificate generation +6. Test HTTPS access + +## Results + +### DNS Check + +```text +$ dig +short test.torrust-tracker.com A @8.8.8.8 +46.224.206.37 +``` + +### Deployment Log + +```text +$ ssh -i ~/.ssh/torrust_tracker_rsa root@46.224.206.37 \ + "cd /root/experiments/experiment-1 && docker compose up -d" + + Network experiment-1_test-network Created + Container experiment-1-webserver-1 Created + Container experiment-1-pingoo-1 Created + Container experiment-1-webserver-1 Started + Container experiment-1-pingoo-1 Started +``` + +### Certificate Generation + +Pingoo automatically generated a Let's Encrypt certificate: + +```text +$ ls /root/experiments/experiment-1/pingoo/tls/ +acme.json +default.key +default.pem +test.torrust-tracker.com.key +test.torrust-tracker.com.pem +``` + +Certificate details: + +```text +$ openssl x509 -in test.torrust-tracker.com.pem -text -noout + +Certificate: + Data: + Version: 3 (0x2) + Serial Number: 06:a4:c1:28:dc:d8:6d:53:86:d0:e4:5d:cc:cb:db:72:68:a3 + Signature Algorithm: ecdsa-with-SHA384 + Issuer: C = US, O = Let's Encrypt, CN = E8 + Validity + Not Before: Jan 12 15:22:51 2026 GMT + Not After : Apr 12 15:22:50 2026 GMT + Subject: CN = test.torrust-tracker.com +``` + +### HTTPS Test + +```text +$ curl -v https://test.torrust-tracker.com 2>&1 | grep -E "(SSL|subject|issuer|expire)" + +* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519MLKEM768 / id-ecPublicKey +* subject: CN=test.torrust-tracker.com +* expire date: Apr 12 15:22:50 2026 GMT +* issuer: C=US; O=Let's Encrypt; CN=E8 +* SSL certificate verify ok. + +$ curl -s https://test.torrust-tracker.com + + + + +If you see this page via HTTPS, Pingoo certificate generation works!
+Certificate info: Check browser padlock for details.
+ + +``` + +## Success Criteria + +- [x] `https://test.torrust-tracker.com` shows the Hello World page +- [x] Browser shows valid Let's Encrypt certificate +- [x] No manual certificate generation required +- [x] Pingoo logs show successful ACME challenge completion + +## Issues Encountered + +### Initial ACME Error (False Alarm) + +When first checking Pingoo logs, an error appeared: + +```text +{"level":"ERROR","message":"TLS: error ordering TLS certificate: error loading config: +error getting ACME authorization for test.torrust-tracker.com: API error: No such +authorization (urn:ietf:params:acme:error:malformed)"} +``` + +However, this was a **false alarm** - the certificate had already been successfully +generated earlier. The error appears to be related to Pingoo retrying an already-completed +authorization. The certificate files were present and valid. + +## Observations + +1. **Automatic Certificate Generation**: Pingoo successfully obtained a Let's Encrypt + certificate without any manual intervention. The only configuration needed was + specifying the domain in `pingoo.yml`. + +2. **TLS 1.3 Only**: As documented, Pingoo only supports TLS 1.3. The connection used + `TLS_AES_256_GCM_SHA384` cipher. + +3. **Post-Quantum Cryptography**: Pingoo used `X25519MLKEM768` for key exchange, + which is a post-quantum hybrid key agreement algorithm. + +4. **ECDSA Certificate**: Let's Encrypt issued an ECDSA certificate (prime256v1) + signed by the E8 intermediate CA. + +5. **Certificate Storage**: Pingoo stores certificates in `/etc/pingoo/tls/` with + `.key` and `.pem` files named after the domain. + +6. **ACME State**: ACME account credentials are stored in `acme.json` for reuse + in future certificate renewals. + +7. **No Email Required**: Unlike certbot, Pingoo does **not** require an email + address for ACME account registration. With certbot, you typically must provide + an email (or explicitly opt out with `--register-unsafely-without-email`). + + **Comparison:** + + - **Certbot**: `certbot certonly --email admin@example.com -d example.com` + - **Pingoo**: Just specify the domain in YAML - no email configuration needed + + **Trade-off**: The email in certbot is used for: + + - Expiration warnings (before certificate expires) + - Security notices (if certificate is revoked) + - Account recovery + + With Pingoo, you won't receive these notifications, so you must rely on Pingoo's + automatic renewal working correctly, or implement your own monitoring. + +8. **Minimal Configuration**: The Pingoo configuration is significantly simpler than + nginx+certbot - just 10 lines of YAML vs. multiple nginx config files plus certbot + setup. + +## Conclusion + +**Experiment 1 is SUCCESSFUL.** Pingoo successfully: + +- Automatically generated a valid Let's Encrypt certificate +- Terminated TLS with modern TLS 1.3 and post-quantum key exchange +- Proxied requests to the backend nginx container +- Required minimal configuration (10 lines of YAML) + +This validates that Pingoo can handle automatic certificate generation, which is the +core requirement for simplifying our deployment infrastructure. + +**Next**: Proceed to Experiment 2 to test Pingoo with the actual Tracker API. diff --git a/docs/research/pingoo-tls-proxy-evaluation/experiment-2-tracker-api.md b/docs/research/pingoo-tls-proxy-evaluation/experiment-2-tracker-api.md new file mode 100644 index 00000000..5bae428b --- /dev/null +++ b/docs/research/pingoo-tls-proxy-evaluation/experiment-2-tracker-api.md @@ -0,0 +1,286 @@ +# Experiment 2: Tracker API with HTTPS + +**Status**: ✅ Complete +**Started**: 2026-01-12 +**Completed**: 2026-01-12 +**Domain**: `api.torrust-tracker.com` + +## Objective + +Test Pingoo with the actual Torrust Tracker API to verify: + +- HTTPS termination for JSON API endpoints +- Correct proxying of API requests to the tracker +- Certificate generation for `api.torrust-tracker.com` + +## Pre-requisites + +- [x] Experiment 1 completed successfully +- [x] DNS propagated for `api.torrust-tracker.com` → 46.224.206.37 +- [x] Port 443 available (Experiment 1 stopped) +- [x] Production stack stopped (`docker compose down` in `/opt/torrust`) + +## Setup + +The setup mirrors the production configuration from `build/docker-hetzner-test/docker-compose/` +to make it easier to add Pingoo to the real deployment later. + +### Files Created + +```text +/root/experiments/experiment-2/ +├── docker-compose.yml +├── pingoo/ +│ └── pingoo.yml +└── storage/ + └── tracker/ + ├── etc/ + │ └── tracker.toml + ├── lib/ + └── log/ +``` + +### docker-compose.yml + +```yaml +# Experiment 2: Tracker API with Pingoo TLS termination +# Mirrors production config from build/docker-hetzner-test/docker-compose/ + +services: + pingoo: + image: pingooio/pingoo:latest + container_name: pingoo + restart: unless-stopped + ports: + - "443:443" + volumes: + - ./pingoo:/etc/pingoo + networks: + - tracker-network + depends_on: + - tracker + + tracker: + image: torrust/tracker:develop + container_name: tracker + tty: true + restart: unless-stopped + environment: + - USER_ID=1000 + - TORRUST_TRACKER_CONFIG_TOML_PATH=/etc/torrust/tracker/tracker.toml + - TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER=sqlite3 + networks: + - tracker-network + # Ports NOT exposed externally - Pingoo handles external access + # ports: + # - 6969:6969/udp # UDP tracker + # - 7070:7070 # HTTP tracker + # - 1212:1212 # HTTP API + volumes: + - ./storage/tracker/lib:/var/lib/torrust/tracker:Z + - ./storage/tracker/log:/var/log/torrust/tracker:Z + - ./storage/tracker/etc:/etc/torrust/tracker:Z + logging: + options: + max-size: "10m" + max-file: "10" + +networks: + tracker-network: + driver: bridge +``` + +### pingoo/pingoo.yml + +```yaml +listeners: + https: + address: https://0.0.0.0:443 + +tls: + acme: + domains: ["api.torrust-tracker.com"] + +services: + tracker-api: + http_proxy: ["http://tracker:1212"] +``` + +### storage/tracker/etc/tracker.toml + +Production-like tracker configuration (mirrors `build/docker-hetzner-test/tracker/tracker.toml`): + +```toml +[metadata] +app = "torrust-tracker" +purpose = "configuration" +schema_version = "2.0.0" + +[logging] +threshold = "info" + +[core] +listed = false +private = false + +[core.tracker_policy] +persistent_torrent_completed_stat = true + +[core.announce_policy] +interval = 300 +interval_min = 300 + +[core.net] +on_reverse_proxy = true + +[core.database] +driver = "sqlite3" +path = "/var/lib/torrust/tracker/database/tracker.db" + +[[udp_trackers]] +bind_address = "0.0.0.0:6969" + +[[http_trackers]] +bind_address = "0.0.0.0:7070" + +[http_api] +bind_address = "0.0.0.0:1212" +``` + +## Deployment Steps + +1. Stop the existing production stack: `cd /opt/torrust && docker compose down` +2. SSH to the Hetzner server +3. Create the experiment directory structure +4. Copy the configuration files +5. Run `docker compose up -d` +6. Check Pingoo logs for certificate generation +7. Test API endpoints via HTTPS + +## Results + +### DNS Check + +```text +$ dig +short api.torrust-tracker.com A @8.8.8.8 +46.224.206.37 +``` + +### Deployment Log + +```text +$ ssh -i ~/.ssh/torrust_tracker_rsa root@46.224.206.37 \ + "cd /root/experiments/experiment-2 && docker compose up -d" + + Network experiment-2_tracker-network Creating + Network experiment-2_tracker-network Created + Container tracker Creating + Container tracker Created + Container pingoo Creating + Container pingoo Created + Container tracker Starting + Container tracker Started + Container pingoo Starting + Container pingoo Started +``` + +### Certificate Generation + +Pingoo automatically generated a Let's Encrypt certificate within seconds: + +```text +$ docker logs pingoo + +{"timestamp":"2026-01-12T16:55:32.144916Z","level":"INFO","message":"configuration successfully loaded from /etc/pingoo/pingoo.yml","services":1,"listeners":1} +{"timestamp":"2026-01-12T16:55:32.145792Z","level":"INFO","message":"docker socket (/var/run/docker.sock) not found. Docker service discovery disabled."} +{"timestamp":"2026-01-12T16:55:33.229813Z","level":"INFO","message":"Starting listener https on https://0.0.0.0:443","listener":"https"} +{"timestamp":"2026-01-12T16:55:39.825316Z","level":"INFO","message":"tls: ACME TLS certificate successfully saved","domain":"api.torrust-tracker.com"} +``` + +Certificate issued in ~7 seconds from container start. + +### TLS Details + +```text +$ curl -v https://api.torrust-tracker.com/api/health_check 2>&1 | grep -E "(SSL|subject|issuer|expire)" + +* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519MLKEM768 / id-ecPublicKey +* subject: CN=api.torrust-tracker.com +* expire date: Apr 12 15:57:07 2026 GMT +* subjectAltName: host "api.torrust-tracker.com" matched cert's "api.torrust-tracker.com" +* issuer: C=US; O=Let's Encrypt; CN=E7 +* SSL certificate verify ok. +``` + +### API Tests + +#### Health Check + +```text +$ curl -s https://api.torrust-tracker.com/api/health_check +{"status":"Ok"} +``` + +#### Stats Endpoint (no admin token configured) + +```text +$ curl -s https://api.torrust-tracker.com/api/v1/stats +Unhandled rejection: Err { reason: "unauthorized" } +``` + +The stats endpoint returns "unauthorized" because no admin token was configured in this +experiment (no `TORRUST_TRACKER_CONFIG_OVERRIDE_HTTP_API__ACCESS_TOKENS__ADMIN` env var). +This is expected - the health check endpoint is sufficient to verify Pingoo is correctly +proxying API requests. + +## Success Criteria + +- [x] `https://api.torrust-tracker.com/api/health_check` returns OK +- [x] Valid Let's Encrypt certificate for `api.torrust-tracker.com` +- [x] API responses are valid JSON +- [x] Tracker is functional via HTTPS + +## Issues Encountered + +### Container Name Conflict + +When first deploying, there was a conflict with the existing `tracker` container from the +production stack: + +```text +Error response from daemon: Conflict. The container name "/tracker" is already in use +``` + +**Resolution**: Stopped the production stack first with `docker compose down` in `/opt/torrust`. + +## Observations + +1. **Fast Certificate Generation**: Certificate was issued in ~7 seconds after container + start, similar to Experiment 1. + +2. **Same TLS Quality**: TLS 1.3 with `TLS_AES_256_GCM_SHA384` cipher and `X25519MLKEM768` + post-quantum key exchange, consistent with Experiment 1. + +3. **Different CA**: This certificate was issued by Let's Encrypt E7 (vs E8 in Experiment 1). + This is normal - Let's Encrypt rotates between intermediate CAs. + +4. **Transparent Proxying**: The tracker API works identically whether accessed directly + or through Pingoo. Headers, authentication, and JSON responses all work correctly. + +5. **Production-Ready Configuration**: Using the same tracker configuration as production + (`on_reverse_proxy = true`) validates this setup for real deployment. + +6. **Minimal Pingoo Config**: Only 10 lines of YAML to add HTTPS to the tracker API. + +## Conclusion + +**Experiment 2 is SUCCESSFUL.** Pingoo successfully: + +- Generated a valid Let's Encrypt certificate for `api.torrust-tracker.com` +- Proxied all API requests correctly to the tracker +- Handled JSON responses transparently +- Required minimal configuration (10 lines) + +This validates that Pingoo can serve as the TLS proxy for the Tracker API in production. + +**Next**: Proceed to Experiment 3 to test Pingoo with the HTTP Tracker (announce/scrape endpoints). diff --git a/docs/research/pingoo-tls-proxy-evaluation/experiment-3-http-tracker.md b/docs/research/pingoo-tls-proxy-evaluation/experiment-3-http-tracker.md new file mode 100644 index 00000000..fe4249a2 --- /dev/null +++ b/docs/research/pingoo-tls-proxy-evaluation/experiment-3-http-tracker.md @@ -0,0 +1,232 @@ +# Experiment 3: HTTP Tracker with HTTPS + +**Status**: ✅ Complete +**Started**: 2026-01-12 +**Completed**: 2026-01-12 +**Domain**: `http1.torrust-tracker.com` + +## Objective + +Test Pingoo with the Torrust HTTP Tracker to verify: + +- HTTPS termination for BitTorrent announce/scrape endpoints +- Correct proxying of tracker protocol requests +- Certificate generation for `http1.torrust-tracker.com` + +## Pre-requisites + +- [x] Experiment 2 completed successfully +- [x] DNS propagated for `http1.torrust-tracker.com` → 46.224.206.37 +- [x] Port 443 available (Experiment 2 stopped) + +## Setup + +### Files Created + +```text +/root/experiments/experiment-3/ +├── docker-compose.yml +├── pingoo/ +│ └── pingoo.yml +└── storage/ + └── tracker/ + ├── etc/ + │ └── tracker.toml + ├── lib/ + └── log/ +``` + +### docker-compose.yml + +```yaml +# Experiment 3: HTTP Tracker with Pingoo TLS termination +# Tests announce/scrape endpoints via HTTPS + +services: + pingoo: + image: pingooio/pingoo:latest + container_name: pingoo + restart: unless-stopped + ports: + - "443:443" + volumes: + - ./pingoo:/etc/pingoo + networks: + - tracker-network + depends_on: + - tracker + + tracker: + image: torrust/tracker:develop + container_name: tracker + tty: true + restart: unless-stopped + environment: + - USER_ID=1000 + - TORRUST_TRACKER_CONFIG_TOML_PATH=/etc/torrust/tracker/tracker.toml + - TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER=sqlite3 + networks: + - tracker-network + volumes: + - ./storage/tracker/lib:/var/lib/torrust/tracker:Z + - ./storage/tracker/log:/var/log/torrust/tracker:Z + - ./storage/tracker/etc:/etc/torrust/tracker:Z + logging: + options: + max-size: "10m" + max-file: "10" + +networks: + tracker-network: + driver: bridge +``` + +### pingoo/pingoo.yml + +```yaml +listeners: + https: + address: https://0.0.0.0:443 + +tls: + acme: + domains: ["http1.torrust-tracker.com"] + +services: + http-tracker: + http_proxy: ["http://tracker:7070"] +``` + +### storage/tracker/etc/tracker.toml + +Same as Experiment 2 - production-like tracker configuration. + +## Deployment Steps + +1. Stop Experiment 2: `cd /root/experiments/experiment-2 && docker compose down` +2. Create the experiment directory structure +3. Copy the configuration files +4. Run `docker compose up -d` +5. Check Pingoo logs for certificate generation +6. Test tracker endpoints via HTTPS + +## Results + +### Deployment Log + +```text +$ ssh -i ~/.ssh/torrust_tracker_rsa root@46.224.206.37 \ + "cd /root/experiments/experiment-3 && docker compose up -d" + + Network experiment-3_tracker-network Creating + Network experiment-3_tracker-network Created + Container tracker Creating + Container tracker Created + Container pingoo Creating + Container pingoo Created + Container tracker Starting + Container tracker Started + Container pingoo Starting + Container pingoo Started +``` + +### Certificate Generation + +```text +$ docker logs pingoo + +{"timestamp":"2026-01-12T17:19:49.432914Z","level":"INFO","message":"configuration successfully loaded from /etc/pingoo/pingoo.yml","services":1,"listeners":1} +{"timestamp":"2026-01-12T17:19:49.433578Z","level":"INFO","message":"docker socket (/var/run/docker.sock) not found. Docker service discovery disabled."} +{"timestamp":"2026-01-12T17:19:50.428199Z","level":"INFO","message":"Starting listener https on https://0.0.0.0:443","listener":"https"} +{"timestamp":"2026-01-12T17:19:57.329901Z","level":"INFO","message":"tls: ACME TLS certificate successfully saved","domain":"http1.torrust-tracker.com"} +``` + +Certificate issued in ~7 seconds. + +### TLS Details + +```text +$ curl -v https://http1.torrust-tracker.com/health_check 2>&1 | grep -E "(SSL|subject|issuer|expire)" + +* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519MLKEM768 / id-ecPublicKey +* subject: CN=http1.torrust-tracker.com +* expire date: Apr 12 16:21:24 2026 GMT +* subjectAltName: host "http1.torrust-tracker.com" matched cert's "http1.torrust-tracker.com" +* issuer: C=US; O=Let's Encrypt; CN=E8 +* SSL certificate verify ok. +``` + +### Tracker Protocol Tests + +#### Health Check + +```text +$ curl -s https://http1.torrust-tracker.com/health_check +{"status":"Ok"} +``` + +#### Announce Endpoint + +```text +$ curl -s 'https://http1.torrust-tracker.com/announce?info_hash=%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00&peer_id=-qB0000-000000000000&port=6881&uploaded=0&downloaded=0&left=0&compact=1' + +d8:completei1e10:incompletei0e8:intervali300e12:min intervali300e5:peers0:6:peers60:e +``` + +Valid bencoded response showing: + +- `complete`: 1 (seeder count) +- `incomplete`: 0 (leecher count) +- `interval`: 300 seconds +- `min interval`: 300 seconds +- `peers`: empty (no other peers for this test torrent) + +#### Scrape Endpoint + +```text +$ curl -s 'https://http1.torrust-tracker.com/scrape?info_hash=01234567890123456789' + +d5:filesd20:01234567890123456789d8:completei0e10:downloadedi0e10:incompletei0eeee +``` + +Valid bencoded scrape response showing torrent statistics. + +## Success Criteria + +- [x] `https://http1.torrust-tracker.com/health_check` returns OK +- [x] Valid Let's Encrypt certificate for `http1.torrust-tracker.com` +- [x] Announce endpoint returns valid bencoded response +- [x] Scrape endpoint returns valid bencoded response +- [x] BitTorrent protocol works correctly via HTTPS + +## Issues Encountered + +None. + +## Observations + +1. **BitTorrent Protocol Works**: Both announce and scrape endpoints work correctly + through Pingoo. The binary bencoded responses are proxied without corruption. + +2. **Same Performance**: Certificate generation (~7 seconds) and TLS configuration + (TLS 1.3, X25519MLKEM768) are consistent with previous experiments. + +3. **Back to E8 CA**: This certificate was issued by Let's Encrypt E8 (same as + Experiment 1), confirming the CA rotation is normal. + +4. **Transparent Binary Proxying**: Pingoo correctly handles the non-JSON tracker + responses (bencoded binary format) without any issues. + +## Conclusion + +**Experiment 3 is SUCCESSFUL.** Pingoo successfully: + +- Generated a valid Let's Encrypt certificate for `http1.torrust-tracker.com` +- Proxied BitTorrent tracker protocol requests correctly +- Handled binary bencoded responses without corruption +- Required minimal configuration (10 lines) + +This validates that Pingoo can serve as the TLS proxy for the HTTP Tracker in production. +The BitTorrent protocol works identically whether accessed directly or through Pingoo. + +**Next**: Proceed to Experiment 4 to test Pingoo with Grafana (WebSocket support - CRITICAL). diff --git a/docs/research/pingoo-tls-proxy-evaluation/experiment-4-grafana.md b/docs/research/pingoo-tls-proxy-evaluation/experiment-4-grafana.md new file mode 100644 index 00000000..f777c4ee --- /dev/null +++ b/docs/research/pingoo-tls-proxy-evaluation/experiment-4-grafana.md @@ -0,0 +1,215 @@ +# Experiment 4: Grafana Dashboard with WebSocket + +**Status**: ⚠️ PARTIAL SUCCESS (HTTP works, WebSocket FAILS) + +## Objective + +Test if Pingoo can serve a full Torrust monitoring stack including Grafana, which requires WebSocket connections for the Grafana Live feature (real-time updates). + +## Configuration + +**Domain**: `grafana.torrust-tracker.com` + +### pingoo.yml + +```yaml +listeners: + https: + address: https://0.0.0.0:443 + +tls: + acme: + domains: + - grafana.torrust-tracker.com + contact: admin@torrust.com + +services: + grafana: + http_proxy: ["http://grafana:3000"] +``` + +### docker-compose.yml + +```yaml +services: + pingoo: + image: pingooio/pingoo:latest + container_name: pingoo + network_mode: host + volumes: + - ./pingoo.yml:/etc/pingoo/pingoo.yml:ro + restart: unless-stopped + + tracker: + image: torrust/tracker:develop + container_name: tracker + ports: + - "1212:1212" # API (internal) + - "6969:6969/udp" # UDP tracker + - "7070:7070" # HTTP tracker + volumes: + - ./tracker.toml:/etc/torrust/tracker/tracker.toml:ro + restart: unless-stopped + + prometheus: + image: prom/prometheus:v3.5.0 + container_name: prometheus + ports: + - "9090:9090" + volumes: + - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro + restart: unless-stopped + + grafana: + image: grafana/grafana:12.3.1 + container_name: grafana + ports: + - "3000:3000" + environment: + - GF_SECURITY_ADMIN_PASSWORD=