Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 28 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,21 @@

# Torrust Tracker Deployer

> ⚠️ **DEVELOPMENT STATUS: Not Production-Ready**
> ⚠️ **DEVELOPMENT STATUS: Early Production Phase**
>
> This project is currently in **active development** and is **not production-ready**. It currently only supports **local deployment** using LXD virtualization for development and testing purposes.
> This project is in **active development** with initial cloud provider support now available.
>
> **Current Scope:**
>
> - ✅ Local LXD virtual machine provisioning
> - ✅ **Hetzner Cloud support** for production deployments
> - ✅ Development and testing workflows
> - ❌ **No real cloud provider support** (AWS, GCP, Azure, Hetzner, etc.)
> - ❌ **No production deployment capabilities**
> - ✅ Multi-provider architecture (provider selection via configuration)
> - ❌ Application deployment (Torrust Tracker stack) - coming soon
>
> 📋 **MVP Goal:** After completing the [roadmap](docs/roadmap.md), we will have a Minimum Viable Product (MVP) that supports real cloud providers and production deployments.
> 📋 **MVP Goal:** After completing the [roadmap](docs/roadmap.md), we will have a fully automated deployment solution for Torrust Tracker with complete application stack management.

This Rust application provides automated deployment infrastructure for Torrust tracker projects. Currently focused on **local development and testing**, it manages VM provisioning and cloud-init execution using LXD virtual machines. The goal is to establish the foundational deployment patterns that will later support real cloud providers.
This Rust application provides automated deployment infrastructure for Torrust tracker projects. It supports **local development** with LXD and **production deployments** with Hetzner Cloud. The multi-provider architecture allows easy extension to additional cloud providers.

## 🎯 Project Goals

Expand All @@ -26,11 +27,12 @@ This Rust application provides automated deployment infrastructure for Torrust t
- ✅ **Support Docker Compose** inside VMs for application stacks
- ✅ **Fast, easy to install and use** local development solution
- ✅ **No nested virtualization dependency** (CI compatibility)
- ✅ **Multi-provider support** (LXD for local, Hetzner Cloud for production)

**Future MVP Goals:** (See [roadmap](docs/roadmap.md))

- 🔄 **Real cloud provider support** (Hetzner, AWS, GCP, Azure)
- 🔄 **Production deployment capabilities**
- 🔄 **Additional cloud providers** (AWS, GCP, Azure)
- 🔄 **Application stack deployment** (Torrust Tracker with Docker Compose)
- 🔄 **Multi-environment management**

## 🔧 Local Development Approach
Expand Down Expand Up @@ -166,11 +168,11 @@ cargo run --bin e2e-tests-full

# Run individual E2E test suites
cargo run --bin e2e-config-tests # Configuration generation and validation tests
cargo run --bin e2e-provision-tests # Infrastructure provisioning tests
cargo run --bin e2e-provision-and-destroy-tests # Infrastructure provisioning tests

# Keep the test environment after completion for inspection
cargo run --bin e2e-tests-full -- --keep
cargo run --bin e2e-provision-tests -- --keep
cargo run --bin e2e-provision-and-destroy-tests -- --keep

# Use custom templates directory
cargo run --bin e2e-tests-full -- --templates-dir ./custom/templates
Expand Down Expand Up @@ -334,7 +336,7 @@ The repository includes comprehensive GitHub Actions workflows for CI testing:

> **Note:** The full E2E tests (`e2e-tests-full`) can only be executed locally due to connectivity issues documented in [`docs/github-actions-issues/`](docs/github-actions-issues/).

## Roadmap
## 🗺️ Roadmap

This project follows a structured development roadmap to evolve from the current local development focus to a production-ready deployment solution.

Expand Down Expand Up @@ -373,6 +375,9 @@ This project follows a structured development roadmap to evolve from the current
│ │ ├── opentofu.md # OpenTofu installation and usage
│ │ ├── ansible.md # Ansible installation and usage
│ │ └── lxd.md # LXD virtual machines
│ ├── user-guide/ # User documentation
│ │ ├── commands/ # Command reference documentation
│ │ └── providers/ # Provider-specific guides (LXD, Hetzner)
│ ├── decisions/ # Architecture Decision Records (ADRs)
│ ├── contributing/ # Contributing guidelines and conventions
│ │ ├── README.md # Main contributing guide
Expand Down Expand Up @@ -404,6 +409,7 @@ This project follows a structured development roadmap to evolve from the current
│ │ └── e2e*.rs # End-to-end testing binaries
│ ├── application/ # Application layer (use cases, commands)
│ ├── domain/ # Domain layer (business logic, entities)
│ │ └── provider/ # Provider types (LXD, Hetzner)
│ ├── infrastructure/ # Infrastructure layer (external systems)
│ ├── presentation/ # Presentation layer (CLI interface)
│ ├── adapters/ # External tool adapters (OpenTofu, Ansible, SSH, LXD)
Expand All @@ -414,7 +420,8 @@ This project follows a structured development roadmap to evolve from the current
│ └── e2e/ # End-to-end testing infrastructure
├── templates/ # 📁 Template configurations (git-tracked)
│ ├── tofu/ # 🏗️ OpenTofu/Terraform templates
│ │ └── lxd/ # LXD VM template configuration
│ │ ├── lxd/ # LXD VM template configuration
│ │ └── hetzner/ # Hetzner Cloud template configuration
│ └── ansible/ # 🤖 Ansible playbook templates
├── tests/ # Integration and system tests
├── target/ # 🦀 Rust build artifacts (git-ignored)
Expand All @@ -434,6 +441,7 @@ This project follows a structured development roadmap to evolve from the current
## 📚 Documentation

- **[👤 User Guide](docs/user-guide/README.md)** - Getting started, command reference, and usage examples
- **[☁️ Provider Guides](docs/user-guide/providers/README.md)** - LXD and Hetzner Cloud provider configuration
- **[🤝 Contributing Guide](docs/contributing/README.md)** - Git workflow, commit process, and linting conventions
- **[🗺️ Roadmap](docs/roadmap.md)** - Development roadmap and MVP goals
- **[📖 Documentation Organization Guide](docs/documentation.md)** - How documentation is organized and where to contribute
Expand All @@ -443,14 +451,19 @@ This project follows a structured development roadmap to evolve from the current

## 🔮 Next Steps

This project is currently focused on local development. The path to production-ready deployment is outlined in our [📋 **Roadmap**](docs/roadmap.md).
This project now supports multiple infrastructure providers. The path to production-ready deployment is outlined in our [📋 **Roadmap**](docs/roadmap.md).

**Recent achievements:**

- ✅ **Multi-Provider Support**: LXD for local development, Hetzner Cloud for production deployments
- ✅ **Provider Selection**: Choose your provider via `provider_config` in environment configuration
- ✅ **Complete CLI Commands**: `create`, `provision`, `configure`, `test`, and `destroy` commands

**Key upcoming milestones:**

- **Real Cloud Provider Support**: Starting with Hetzner, then expanding to AWS, GCP, and Azure
- **Production Commands**: `create`, `deploy`, `destroy`, and `status` commands for production environments
- **Application Stack Management**: Complete Docker Compose stacks with Torrust Tracker, MySQL, Prometheus, and Grafana
- **HTTPS Support**: SSL/TLS configuration for all services
- **Backup & Recovery**: Database backups and disaster recovery procedures
- **Additional Cloud Providers**: AWS, GCP, and Azure support

**[📖 See full roadmap →](docs/roadmap.md)**
34 changes: 23 additions & 11 deletions docs/user-guide/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,47 @@ This guide provides an overview of the available commands for the Torrust Tracke

## ⚠️ Implementation Status

**Current State**: The `create` and `destroy` commands are fully implemented with CLI interface.
**Current State**: Core deployment commands are fully implemented with CLI interface.

- ✅ **Implemented**: `create environment`, `create template`, `destroy` commands
- ❌ **CLI Interface**: Other commands not yet implemented
- ✅ **Implemented**: `create`, `provision`, `configure`, `test`, `destroy` commands
- ✅ **Multi-Provider Support**: LXD (local) and Hetzner Cloud (production)
- ❌ **CLI Interface**: `deploy`, `run`, `status` commands not yet implemented

The CLI commands documented here represent the planned MVP implementation.
## Provider Selection

## Planned Commands (MVP)
All commands work with both supported providers. The provider is specified in your environment configuration file:

| Provider | Configuration | Use Case |
| ----------- | ----------------------- | ------------------------ |
| **LXD** | `"provider": "lxd"` | Local development, CI/CD |
| **Hetzner** | `"provider": "hetzner"` | Production deployments |

📖 **See [Provider Guides](providers/README.md)** for detailed provider configuration.

## Available Commands

```bash
# Available commands
torrust-tracker-deployer create environment --env-file <file> # Create new deployment environment
torrust-tracker-deployer create template [path] # Generate configuration template
torrust-tracker-deployer destroy <env> # Destroy infrastructure and clean up resources
torrust-tracker-deployer create template --provider <lxd|hetzner> [path] # Generate configuration template
torrust-tracker-deployer create environment --env-file <file> # Create new deployment environment
torrust-tracker-deployer provision <env> # Provision VM infrastructure
torrust-tracker-deployer configure <env> # Configure infrastructure (Docker, etc.)
torrust-tracker-deployer test <env> # Verify deployment
torrust-tracker-deployer destroy <env> # Destroy infrastructure and clean up

# Future commands
torrust-tracker-deployer deploy <env> # Future - Full deployment (provision → configure → release)
torrust-tracker-deployer run <env> # Future - Start application services
torrust-tracker-deployer status <env> # Future - Check environment status
torrust-tracker-deployer test <env> # Future - Run validation tests
```

**Note**: The `deploy` command will internally orchestrate the complete deployment workflow: provision → configure → release. Individual commands for these phases may be added later for advanced users.

## Available Commands
## Command Details

### Environment Management

#### [`create environment`](commands/create-environment.md)
#### [`create environment`](commands/create.md)

Create a new deployment environment from a configuration file.

Expand Down
35 changes: 28 additions & 7 deletions docs/user-guide/commands/provision.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,12 @@ Provision virtual machine infrastructure for a deployment environment.

## Purpose

Creates and configures VM infrastructure using OpenTofu (Terraform) and LXD. This command takes an environment from the "Created" state to the "Provisioned" state with running VM instances.
Creates and configures VM infrastructure using OpenTofu (Terraform). This command takes an environment from the "Created" state to the "Provisioned" state with running VM instances.

The provision command works with all supported providers:

- **LXD** - Creates local VMs for development and testing
- **Hetzner Cloud** - Creates cloud servers for production deployments

## Command Syntax

Expand All @@ -19,10 +24,14 @@ torrust-tracker-deployer provision <ENVIRONMENT>
## Prerequisites

1. **Environment created** - Must run `create environment` first
2. **LXD installed** - Local LXD installation configured
2. **Provider-specific requirements**:
- **LXD**: Local LXD installation configured
- **Hetzner**: Valid API token in environment configuration
3. **OpenTofu installed** - OpenTofu CLI available in PATH
4. **SSH keys** - SSH key pair referenced in environment configuration

📖 **See [Provider Guides](../providers/README.md)** for provider-specific setup.

## State Transition

```text
Expand All @@ -33,7 +42,7 @@ torrust-tracker-deployer provision <ENVIRONMENT>

When you provision an environment:

1. **Renders OpenTofu templates** - Generates infrastructure-as-code files
1. **Renders OpenTofu templates** - Generates provider-specific infrastructure-as-code files
2. **Initializes OpenTofu** - Sets up backend and providers (`tofu init`)
3. **Creates execution plan** - Validates configuration (`tofu plan`)
4. **Applies infrastructure** - Creates VM resources (`tofu apply`)
Expand Down Expand Up @@ -73,18 +82,30 @@ torrust-tracker-deployer provision dev-local
# Staging
torrust-tracker-deployer provision staging

# Production
# Production (Hetzner)
torrust-tracker-deployer provision production
```

## Output

The provision command creates:
The provision command creates provider-specific resources:

### LXD Provider

- **VM instance** - LXD virtual machine
- **VM instance** - LXD virtual machine (`torrust-tracker-vm-<env-name>`)
- **LXD profile** - Custom profile with cloud-init configuration
- **Network configuration** - Bridged network with IP assignment
- **OpenTofu state** - Infrastructure state in `build/<env>/tofu/lxd/`

### Hetzner Provider

- **Cloud server** - Hetzner Cloud server instance
- **Firewall** - Hetzner firewall with SSH access
- **SSH key** - Uploaded SSH public key
- **OpenTofu state** - Infrastructure state in `build/<env>/tofu/hetzner/`

### Common Outputs (All Providers)

- **Ansible inventory** - Generated inventory in `build/<env>/ansible/`
- **Environment state update** - State file updated to "Provisioned"

Expand Down Expand Up @@ -116,7 +137,7 @@ ls -la data/my-environment/
torrust-tracker-deployer create environment -f config.json
```

### LXD not initialized
### LXD not initialized (LXD provider only)

**Problem**: LXD is not properly initialized

Expand Down
59 changes: 53 additions & 6 deletions docs/user-guide/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,25 @@ Get up and running with Torrust Tracker Deployer in minutes.

## Prerequisites

- **LXD** - Local LXD installation configured
- **LXD** - Local LXD installation configured (for local development)
- **Hetzner Account** - Hetzner Cloud account with API token (for production)
- **OpenTofu** - OpenTofu CLI installed
- **Ansible** - Ansible for configuration management
- **SSH keys** - SSH key pair for VM access

> **Tip**: Run `cargo run --bin dependency-installer check` to verify all prerequisites are met.

## Choose Your Provider

Before starting, decide which provider to use:

| Provider | Best For | Requirements |
| ----------------- | --------------------------------- | --------------------------- |
| **LXD** | Local development, CI/CD, testing | Linux with LXD installed |
| **Hetzner Cloud** | Production deployments | Hetzner account + API token |

📖 **See [Provider Guides](providers/README.md)** for detailed setup instructions.

## Installation

```bash
Expand All @@ -29,10 +41,18 @@ This example walks through the complete deployment lifecycle from template gener

### Step 1: Generate Environment Template

Create a template configuration file:
Create a template configuration file for your chosen provider:

**For LXD (local development)**:

```bash
torrust-tracker-deployer create template --provider lxd my-environment.json
```

**For Hetzner Cloud (production)**:

```bash
torrust-tracker-deployer create template my-environment.json
torrust-tracker-deployer create template --provider hetzner my-environment.json
```

**Output**:
Expand All @@ -41,7 +61,7 @@ torrust-tracker-deployer create template my-environment.json
✓ Template generated: my-environment.json
```

This creates a pre-filled template with default values that you can customize.
This creates a pre-filled template with provider-specific values that you can customize.

### Step 2: Customize Configuration

Expand All @@ -51,7 +71,7 @@ Edit the generated template:
nano my-environment.json
```

**Example configuration**:
**Example LXD configuration**:

```json
{
Expand All @@ -63,11 +83,37 @@ nano my-environment.json
"public_key_path": "fixtures/testing_rsa.pub",
"username": "torrust",
"port": 22
},
"provider": {
"provider": "lxd",
"profile_name": "torrust-profile-local"
}
}
```

**Example Hetzner configuration**:

```json
{
"environment": {
"name": "my-production-env"
},
"ssh_credentials": {
"private_key_path": "~/.ssh/id_ed25519",
"public_key_path": "~/.ssh/id_ed25519.pub",
"username": "torrust",
"port": 22
},
"provider": {
"provider": "hetzner",
"api_token": "your-hetzner-api-token-here",
"server_type": "cx22",
"location": "nbg1"
}
}
```

> **Note**: This example uses the test SSH keys from `fixtures/` directory. For production, use your own SSH keys (e.g., `~/.ssh/id_ed25519`).
> **Note**: For LXD testing, use the test SSH keys from `fixtures/` directory. For production, use your own SSH keys (e.g., `~/.ssh/id_ed25519`).

**Key fields to customize**:

Expand All @@ -76,6 +122,7 @@ nano my-environment.json
- `ssh_credentials.public_key_path` - Path to your SSH public key file
- `ssh_credentials.username` - SSH username for VM access (default: torrust)
- `ssh_credentials.port` - SSH port (default: 22)
- `provider` - Provider-specific configuration (see [Provider Guides](providers/README.md))

### Step 3: Create Environment

Expand Down
Loading