diff --git a/README.md b/README.md index 500a8ef1..096bcc7b 100644 --- a/README.md +++ b/README.md @@ -2,20 +2,21 @@ # Torrust Tracker Deployer -> ⚠️ **DEVELOPMENT STATUS: Not Production-Ready** +> ⚠️ **DEVELOPMENT STATUS: Early Production Phase** > -> This project is currently in **active development** and is **not production-ready**. It currently only supports **local deployment** using LXD virtualization for development and testing purposes. +> This project is in **active development** with initial cloud provider support now available. > > **Current Scope:** > > - ✅ Local LXD virtual machine provisioning +> - ✅ **Hetzner Cloud support** for production deployments > - ✅ Development and testing workflows -> - ❌ **No real cloud provider support** (AWS, GCP, Azure, Hetzner, etc.) -> - ❌ **No production deployment capabilities** +> - ✅ Multi-provider architecture (provider selection via configuration) +> - ❌ Application deployment (Torrust Tracker stack) - coming soon > -> 📋 **MVP Goal:** After completing the [roadmap](docs/roadmap.md), we will have a Minimum Viable Product (MVP) that supports real cloud providers and production deployments. +> 📋 **MVP Goal:** After completing the [roadmap](docs/roadmap.md), we will have a fully automated deployment solution for Torrust Tracker with complete application stack management. -This Rust application provides automated deployment infrastructure for Torrust tracker projects. Currently focused on **local development and testing**, it manages VM provisioning and cloud-init execution using LXD virtual machines. The goal is to establish the foundational deployment patterns that will later support real cloud providers. +This Rust application provides automated deployment infrastructure for Torrust tracker projects. It supports **local development** with LXD and **production deployments** with Hetzner Cloud. The multi-provider architecture allows easy extension to additional cloud providers. ## 🎯 Project Goals @@ -26,11 +27,12 @@ This Rust application provides automated deployment infrastructure for Torrust t - ✅ **Support Docker Compose** inside VMs for application stacks - ✅ **Fast, easy to install and use** local development solution - ✅ **No nested virtualization dependency** (CI compatibility) +- ✅ **Multi-provider support** (LXD for local, Hetzner Cloud for production) **Future MVP Goals:** (See [roadmap](docs/roadmap.md)) -- 🔄 **Real cloud provider support** (Hetzner, AWS, GCP, Azure) -- 🔄 **Production deployment capabilities** +- 🔄 **Additional cloud providers** (AWS, GCP, Azure) +- 🔄 **Application stack deployment** (Torrust Tracker with Docker Compose) - 🔄 **Multi-environment management** ## 🔧 Local Development Approach @@ -166,11 +168,11 @@ cargo run --bin e2e-tests-full # Run individual E2E test suites cargo run --bin e2e-config-tests # Configuration generation and validation tests -cargo run --bin e2e-provision-tests # Infrastructure provisioning tests +cargo run --bin e2e-provision-and-destroy-tests # Infrastructure provisioning tests # Keep the test environment after completion for inspection cargo run --bin e2e-tests-full -- --keep -cargo run --bin e2e-provision-tests -- --keep +cargo run --bin e2e-provision-and-destroy-tests -- --keep # Use custom templates directory cargo run --bin e2e-tests-full -- --templates-dir ./custom/templates @@ -334,7 +336,7 @@ The repository includes comprehensive GitHub Actions workflows for CI testing: > **Note:** The full E2E tests (`e2e-tests-full`) can only be executed locally due to connectivity issues documented in [`docs/github-actions-issues/`](docs/github-actions-issues/). -## � Roadmap +## 🗺️ Roadmap This project follows a structured development roadmap to evolve from the current local development focus to a production-ready deployment solution. @@ -373,6 +375,9 @@ This project follows a structured development roadmap to evolve from the current │ │ ├── opentofu.md # OpenTofu installation and usage │ │ ├── ansible.md # Ansible installation and usage │ │ └── lxd.md # LXD virtual machines +│ ├── user-guide/ # User documentation +│ │ ├── commands/ # Command reference documentation +│ │ └── providers/ # Provider-specific guides (LXD, Hetzner) │ ├── decisions/ # Architecture Decision Records (ADRs) │ ├── contributing/ # Contributing guidelines and conventions │ │ ├── README.md # Main contributing guide @@ -404,6 +409,7 @@ This project follows a structured development roadmap to evolve from the current │ │ └── e2e*.rs # End-to-end testing binaries │ ├── application/ # Application layer (use cases, commands) │ ├── domain/ # Domain layer (business logic, entities) +│ │ └── provider/ # Provider types (LXD, Hetzner) │ ├── infrastructure/ # Infrastructure layer (external systems) │ ├── presentation/ # Presentation layer (CLI interface) │ ├── adapters/ # External tool adapters (OpenTofu, Ansible, SSH, LXD) @@ -414,7 +420,8 @@ This project follows a structured development roadmap to evolve from the current │ └── e2e/ # End-to-end testing infrastructure ├── templates/ # 📁 Template configurations (git-tracked) │ ├── tofu/ # 🏗️ OpenTofu/Terraform templates -│ │ └── lxd/ # LXD VM template configuration +│ │ ├── lxd/ # LXD VM template configuration +│ │ └── hetzner/ # Hetzner Cloud template configuration │ └── ansible/ # 🤖 Ansible playbook templates ├── tests/ # Integration and system tests ├── target/ # 🦀 Rust build artifacts (git-ignored) @@ -434,6 +441,7 @@ This project follows a structured development roadmap to evolve from the current ## 📚 Documentation - **[👤 User Guide](docs/user-guide/README.md)** - Getting started, command reference, and usage examples +- **[☁️ Provider Guides](docs/user-guide/providers/README.md)** - LXD and Hetzner Cloud provider configuration - **[🤝 Contributing Guide](docs/contributing/README.md)** - Git workflow, commit process, and linting conventions - **[🗺️ Roadmap](docs/roadmap.md)** - Development roadmap and MVP goals - **[📖 Documentation Organization Guide](docs/documentation.md)** - How documentation is organized and where to contribute @@ -443,14 +451,19 @@ This project follows a structured development roadmap to evolve from the current ## 🔮 Next Steps -This project is currently focused on local development. The path to production-ready deployment is outlined in our [📋 **Roadmap**](docs/roadmap.md). +This project now supports multiple infrastructure providers. The path to production-ready deployment is outlined in our [📋 **Roadmap**](docs/roadmap.md). + +**Recent achievements:** + +- ✅ **Multi-Provider Support**: LXD for local development, Hetzner Cloud for production deployments +- ✅ **Provider Selection**: Choose your provider via `provider_config` in environment configuration +- ✅ **Complete CLI Commands**: `create`, `provision`, `configure`, `test`, and `destroy` commands **Key upcoming milestones:** -- **Real Cloud Provider Support**: Starting with Hetzner, then expanding to AWS, GCP, and Azure -- **Production Commands**: `create`, `deploy`, `destroy`, and `status` commands for production environments - **Application Stack Management**: Complete Docker Compose stacks with Torrust Tracker, MySQL, Prometheus, and Grafana - **HTTPS Support**: SSL/TLS configuration for all services - **Backup & Recovery**: Database backups and disaster recovery procedures +- **Additional Cloud Providers**: AWS, GCP, and Azure support **[📖 See full roadmap →](docs/roadmap.md)** diff --git a/docs/user-guide/commands.md b/docs/user-guide/commands.md index 96d60954..89f7d306 100644 --- a/docs/user-guide/commands.md +++ b/docs/user-guide/commands.md @@ -4,35 +4,47 @@ This guide provides an overview of the available commands for the Torrust Tracke ## ⚠️ Implementation Status -**Current State**: The `create` and `destroy` commands are fully implemented with CLI interface. +**Current State**: Core deployment commands are fully implemented with CLI interface. -- ✅ **Implemented**: `create environment`, `create template`, `destroy` commands -- ❌ **CLI Interface**: Other commands not yet implemented +- ✅ **Implemented**: `create`, `provision`, `configure`, `test`, `destroy` commands +- ✅ **Multi-Provider Support**: LXD (local) and Hetzner Cloud (production) +- ❌ **CLI Interface**: `deploy`, `run`, `status` commands not yet implemented -The CLI commands documented here represent the planned MVP implementation. +## Provider Selection -## Planned Commands (MVP) +All commands work with both supported providers. The provider is specified in your environment configuration file: + +| Provider | Configuration | Use Case | +| ----------- | ----------------------- | ------------------------ | +| **LXD** | `"provider": "lxd"` | Local development, CI/CD | +| **Hetzner** | `"provider": "hetzner"` | Production deployments | + +📖 **See [Provider Guides](providers/README.md)** for detailed provider configuration. + +## Available Commands ```bash # Available commands -torrust-tracker-deployer create environment --env-file # Create new deployment environment -torrust-tracker-deployer create template [path] # Generate configuration template -torrust-tracker-deployer destroy # Destroy infrastructure and clean up resources +torrust-tracker-deployer create template --provider [path] # Generate configuration template +torrust-tracker-deployer create environment --env-file # Create new deployment environment +torrust-tracker-deployer provision # Provision VM infrastructure +torrust-tracker-deployer configure # Configure infrastructure (Docker, etc.) +torrust-tracker-deployer test # Verify deployment +torrust-tracker-deployer destroy # Destroy infrastructure and clean up # Future commands torrust-tracker-deployer deploy # Future - Full deployment (provision → configure → release) torrust-tracker-deployer run # Future - Start application services torrust-tracker-deployer status # Future - Check environment status -torrust-tracker-deployer test # Future - Run validation tests ``` **Note**: The `deploy` command will internally orchestrate the complete deployment workflow: provision → configure → release. Individual commands for these phases may be added later for advanced users. -## Available Commands +## Command Details ### Environment Management -#### [`create environment`](commands/create-environment.md) +#### [`create environment`](commands/create.md) Create a new deployment environment from a configuration file. diff --git a/docs/user-guide/commands/provision.md b/docs/user-guide/commands/provision.md index 3b55b070..b4593b15 100644 --- a/docs/user-guide/commands/provision.md +++ b/docs/user-guide/commands/provision.md @@ -4,7 +4,12 @@ Provision virtual machine infrastructure for a deployment environment. ## Purpose -Creates and configures VM infrastructure using OpenTofu (Terraform) and LXD. This command takes an environment from the "Created" state to the "Provisioned" state with running VM instances. +Creates and configures VM infrastructure using OpenTofu (Terraform). This command takes an environment from the "Created" state to the "Provisioned" state with running VM instances. + +The provision command works with all supported providers: + +- **LXD** - Creates local VMs for development and testing +- **Hetzner Cloud** - Creates cloud servers for production deployments ## Command Syntax @@ -19,10 +24,14 @@ torrust-tracker-deployer provision ## Prerequisites 1. **Environment created** - Must run `create environment` first -2. **LXD installed** - Local LXD installation configured +2. **Provider-specific requirements**: + - **LXD**: Local LXD installation configured + - **Hetzner**: Valid API token in environment configuration 3. **OpenTofu installed** - OpenTofu CLI available in PATH 4. **SSH keys** - SSH key pair referenced in environment configuration +📖 **See [Provider Guides](../providers/README.md)** for provider-specific setup. + ## State Transition ```text @@ -33,7 +42,7 @@ torrust-tracker-deployer provision When you provision an environment: -1. **Renders OpenTofu templates** - Generates infrastructure-as-code files +1. **Renders OpenTofu templates** - Generates provider-specific infrastructure-as-code files 2. **Initializes OpenTofu** - Sets up backend and providers (`tofu init`) 3. **Creates execution plan** - Validates configuration (`tofu plan`) 4. **Applies infrastructure** - Creates VM resources (`tofu apply`) @@ -73,18 +82,30 @@ torrust-tracker-deployer provision dev-local # Staging torrust-tracker-deployer provision staging -# Production +# Production (Hetzner) torrust-tracker-deployer provision production ``` ## Output -The provision command creates: +The provision command creates provider-specific resources: + +### LXD Provider -- **VM instance** - LXD virtual machine +- **VM instance** - LXD virtual machine (`torrust-tracker-vm-`) - **LXD profile** - Custom profile with cloud-init configuration - **Network configuration** - Bridged network with IP assignment - **OpenTofu state** - Infrastructure state in `build//tofu/lxd/` + +### Hetzner Provider + +- **Cloud server** - Hetzner Cloud server instance +- **Firewall** - Hetzner firewall with SSH access +- **SSH key** - Uploaded SSH public key +- **OpenTofu state** - Infrastructure state in `build//tofu/hetzner/` + +### Common Outputs (All Providers) + - **Ansible inventory** - Generated inventory in `build//ansible/` - **Environment state update** - State file updated to "Provisioned" @@ -116,7 +137,7 @@ ls -la data/my-environment/ torrust-tracker-deployer create environment -f config.json ``` -### LXD not initialized +### LXD not initialized (LXD provider only) **Problem**: LXD is not properly initialized diff --git a/docs/user-guide/quick-start.md b/docs/user-guide/quick-start.md index 963c6f6d..3273f221 100644 --- a/docs/user-guide/quick-start.md +++ b/docs/user-guide/quick-start.md @@ -4,13 +4,25 @@ Get up and running with Torrust Tracker Deployer in minutes. ## Prerequisites -- **LXD** - Local LXD installation configured +- **LXD** - Local LXD installation configured (for local development) +- **Hetzner Account** - Hetzner Cloud account with API token (for production) - **OpenTofu** - OpenTofu CLI installed - **Ansible** - Ansible for configuration management - **SSH keys** - SSH key pair for VM access > **Tip**: Run `cargo run --bin dependency-installer check` to verify all prerequisites are met. +## Choose Your Provider + +Before starting, decide which provider to use: + +| Provider | Best For | Requirements | +| ----------------- | --------------------------------- | --------------------------- | +| **LXD** | Local development, CI/CD, testing | Linux with LXD installed | +| **Hetzner Cloud** | Production deployments | Hetzner account + API token | + +📖 **See [Provider Guides](providers/README.md)** for detailed setup instructions. + ## Installation ```bash @@ -29,10 +41,18 @@ This example walks through the complete deployment lifecycle from template gener ### Step 1: Generate Environment Template -Create a template configuration file: +Create a template configuration file for your chosen provider: + +**For LXD (local development)**: + +```bash +torrust-tracker-deployer create template --provider lxd my-environment.json +``` + +**For Hetzner Cloud (production)**: ```bash -torrust-tracker-deployer create template my-environment.json +torrust-tracker-deployer create template --provider hetzner my-environment.json ``` **Output**: @@ -41,7 +61,7 @@ torrust-tracker-deployer create template my-environment.json ✓ Template generated: my-environment.json ``` -This creates a pre-filled template with default values that you can customize. +This creates a pre-filled template with provider-specific values that you can customize. ### Step 2: Customize Configuration @@ -51,7 +71,7 @@ Edit the generated template: nano my-environment.json ``` -**Example configuration**: +**Example LXD configuration**: ```json { @@ -63,11 +83,37 @@ nano my-environment.json "public_key_path": "fixtures/testing_rsa.pub", "username": "torrust", "port": 22 + }, + "provider": { + "provider": "lxd", + "profile_name": "torrust-profile-local" + } +} +``` + +**Example Hetzner configuration**: + +```json +{ + "environment": { + "name": "my-production-env" + }, + "ssh_credentials": { + "private_key_path": "~/.ssh/id_ed25519", + "public_key_path": "~/.ssh/id_ed25519.pub", + "username": "torrust", + "port": 22 + }, + "provider": { + "provider": "hetzner", + "api_token": "your-hetzner-api-token-here", + "server_type": "cx22", + "location": "nbg1" } } ``` -> **Note**: This example uses the test SSH keys from `fixtures/` directory. For production, use your own SSH keys (e.g., `~/.ssh/id_ed25519`). +> **Note**: For LXD testing, use the test SSH keys from `fixtures/` directory. For production, use your own SSH keys (e.g., `~/.ssh/id_ed25519`). **Key fields to customize**: @@ -76,6 +122,7 @@ nano my-environment.json - `ssh_credentials.public_key_path` - Path to your SSH public key file - `ssh_credentials.username` - SSH username for VM access (default: torrust) - `ssh_credentials.port` - SSH port (default: 22) +- `provider` - Provider-specific configuration (see [Provider Guides](providers/README.md)) ### Step 3: Create Environment