Skip to content

vm-providers.md

Latest commit

 

History

History
315 lines (221 loc) · 9.8 KB

vm-providers.md

File metadata and controls

315 lines (221 loc) · 9.8 KB

VM Providers Comparison: LXD vs Multipass

This document provides a detailed comparison between LXD containers and Multipass VMs for the Torrust tracker deployment infrastructure, explaining why LXD was chosen as the official provider.

🔄 Overview

For the Torrust tracker deployment infrastructure, we evaluated two main approaches for creating virtual environments that support cloud-init:

  1. LXD Containers - System containers with cloud-init support
  2. Multipass VMs - Full virtual machines with nested virtualization

📊 Detailed Comparison

Feature LXD Containers Multipass VMs
Technology Type System containers (shared kernel) Full virtual machines (separate kernel)
GitHub Actions Support ✅ Official provider support ⚠️ Undocumented support
Nested Virtualization ❌ Not required ✅ Required
Resource Usage ✅ Lower (containers) ❌ Higher (full VMs)
Boot Time ✅ Faster (~5-10s) ❌ Slower (~30-60s)
Isolation Level 🔶 Process-level ✅ Complete hardware-level
Cloud-init Support ✅ Container initialization ✅ Full VM boot process
Docker Support ✅ Full support ✅ Full support
Setup Complexity 🔶 Requires LXD daemon setup ✅ Simple installation
Network Configuration ✅ Flexible bridge/overlay ✅ Standard VM networking
Storage Management ✅ Efficient with snapshots 🔶 Standard VM disk images
Multi-architecture ✅ ARM64, x86_64 support ✅ Limited by hypervisor

🎯 Why LXD is Our Official Choice

✅ Official GitHub Actions Support

LXD has official support from the GitHub Actions team:

  • Documented and guaranteed compatibility
  • Long-term stability assurance
  • Official troubleshooting support

Multipass support is undocumented:

  • Works currently but no official guarantees
  • Could break without notice in future GitHub runner updates
  • No official support channel for issues

⚡ Performance Benefits

LXD containers provide superior performance:

  • Boot time: 5-10 seconds vs 30-60 seconds for Multipass
  • Resource usage: Shared kernel means lower memory and CPU overhead
  • Storage efficiency: Container filesystems are more efficient than VM disk images

🔧 CI/CD Optimization

LXD is better suited for automated testing:

  • Faster spin-up times reduce CI job duration
  • Lower resource requirements allow more parallel jobs
  • Predictable performance characteristics

🏗️ Infrastructure Consistency

LXD provides better DevOps alignment:

  • Container-based approach aligns with modern cloud practices
  • Easier to version and distribute container images
  • Better integration with container orchestration if needed

🧪 When to Use Each Approach

Use LXD Containers When:

  • ✅ CI/CD environments (GitHub Actions, GitLab CI, etc.)
  • ✅ Development environments requiring fast iteration
  • ✅ Testing scenarios where performance matters
  • ✅ Container-based workflows
  • ✅ Resource-constrained environments

Use Multipass VMs When:

  • 🔬 Complete isolation is required
  • 🔬 Kernel-level testing (different kernels, kernel modules)
  • 🔬 Legacy application testing requiring specific kernel features
  • 🔬 Security testing requiring hardware-level isolation
  • 🔬 Experimental development where official support isn't critical

🚀 Getting Started with Each

LXD Quick Start

# Install LXD
sudo snap install lxd
sudo lxd init --auto
sudo usermod -a -G lxd $USER
newgrp lxd

# Deploy with OpenTofu
cd templates/tofu/lxd
tofu init && tofu apply

Multipass Quick Start (Deprecated)

Note: Multipass support has been removed from this project as of the LXD over Multipass decision. For Multipass-related documentation and examples, see: https://github.com/torrust/torrust-tracker-deployer

# Multipass is no longer supported in this repository
# See the link above for Multipass examples and documentation

🔍 Technical Deep Dive

LXD Architecture

System Containers:

  • Shared kernel with host system
  • Isolated userspace processes
  • Native performance for most operations
  • Container-specific init system (systemd)

Cloud-init Integration:

  • Cloud-init runs during container initialization
  • Supports full cloud-init functionality
  • Network configuration, user setup, package installation

Multipass Architecture

Full Virtualization:

  • Complete virtual machine with separate kernel
  • Hardware-level isolation
  • Full Ubuntu VM experience
  • Standard VM boot process

Cloud-init Integration:

  • Cloud-init runs during VM boot
  • Complete VM initialization process
  • Full hardware simulation

🛠️ Setup Requirements

LXD Requirements

Local Development:

# System requirements
- Linux host (any distribution)
- 1GB+ RAM available
- 10GB+ disk space
- No special kernel features required

# Installation
sudo snap install lxd
sudo lxd init --auto

GitHub Actions:

# In GitHub Actions workflow
- name: Setup LXD
  uses: canonical/setup-lxd@v0.1.1

Multipass Requirements

Local Development:

# System requirements
- Linux/macOS/Windows host
- 2GB+ RAM available
- 20GB+ disk space
- Nested virtualization support (KVM/HyperV)

# Installation
sudo snap install multipass

GitHub Actions:

# Nested virtualization required (may not be available)
- name: Enable nested virtualization
  run: |
    # Various approaches depending on runner type
    # Not officially documented

📈 Performance Benchmarks

Boot Time Comparison

Operation LXD Container Multipass VM
Cold start 5-8 seconds 35-45 seconds
Warm start 2-3 seconds 25-30 seconds
Cloud-init +10-15 seconds +20-30 seconds

Resource Usage

Metric LXD Container Multipass VM
Base RAM ~50MB ~512MB
Boot RAM spike ~200MB ~1GB
Disk overhead ~100MB ~1-2GB
CPU overhead Minimal Hypervisor layer

🔒 Security Considerations

LXD Security

Isolation Level:

  • Process-level isolation
  • Shared kernel space
  • Container escape possible but rare
  • Good for development/testing environments

Security Features:

  • AppArmor/SELinux profiles
  • User namespace isolation
  • Capability restrictions
  • Resource limits

Multipass Security

Isolation Level:

  • Hardware-level isolation
  • Separate kernel space
  • VM escape extremely difficult
  • Suitable for security-sensitive testing

Security Features:

  • Complete VM isolation
  • Hardware virtualization boundaries
  • Separate network stacks
  • Independent file systems

🚧 Limitations and Considerations

LXD Limitations

  • Kernel sharing: Cannot test different kernel versions
  • Hardware simulation: Limited hardware device simulation
  • Container-specific: Some VM-specific features unavailable
  • Setup complexity: Requires LXD daemon configuration

Multipass Limitations

  • Nested virtualization: Requires special hardware/software support
  • Resource overhead: Higher memory and CPU usage
  • Boot time: Slower startup affects CI performance
  • GitHub Actions: Unofficial support may break

🔮 Future Considerations

LXD Evolution

  • Enhanced cloud-init support
  • Better integration with container orchestration
  • Improved security isolation features
  • GPU and special device passthrough

Multipass Evolution

  • Potential official GitHub Actions support
  • Performance improvements
  • Better integration with cloud providers
  • Enhanced local development features

📋 Decision Matrix

Use this matrix to decide which approach fits your specific needs:

Requirement Weight LXD Score Multipass Score
CI/CD Support High 10 6
Performance High 9 6
Complete Isolation Medium 5 10
Setup Simplicity Medium 7 9
Resource Efficiency High 10 5
Official Support High 10 3
Total 51 39

Scoring: 1-10 scale (10 = excellent, 1 = poor)

📚 Additional Resources

LXD Resources

Multipass Resources

Virtualization Resources