EPIC: Refactor Ansible Templates to Variables Pattern

[josecelano]

EPIC: Refactor Ansible Templates to Variables Pattern

Type: EPIC
Parent Epic: #16 - Finish ConfigureCommand - System Security Configuration
Depends On: #18 - Configure UFW Firewall

Overview

This epic tracks the refactoring of Ansible templates to use a centralized variables pattern similar to OpenTofu's variables.tfvars.tera approach. This consolidates multiple Tera templates into a single variables file, reducing complexity and establishing a consistent pattern for future Ansible template additions.

After implementing the security updates and firewall configuration, we now have 2 Tera templates (inventory.yml.tera and configure-firewall.yml.tera). This refactoring consolidates them into a single variables-based approach that will simplify future service additions.

Sub-Tasks

This EPIC uses vertical slice approach - each sub-task is a complete, independently deployable increment:

• Create Variables Template Infrastructure #105 - Create Variables Template Infrastructure - Complete implementation with unit tests and documentation
• Convert Firewall Template to Variables Pattern (Complete Vertical Slice) #106 - Convert Firewall to Variables Pattern - Complete conversion including cleanup, documentation updates, and full validation

Goals

• Centralized Variables: Create single variables.yml.tera for all Ansible variables
• Template Consolidation: Reduce template complexity through variables pattern
• Consistent Pattern: Match OpenTofu's elegant variables approach
• Future-Proofing: Establish pattern for easy addition of new services
• Reduced Complexity: Minimize Rust template handling boilerplate (~500 lines removed)
• Maintain Functionality: Ensure all existing functionality continues to work

Implementation Approach

This epic uses vertical slice methodology - each sub-task is a complete, independently deployable increment that includes implementation, cleanup, documentation, and validation. This follows lean/agile principles rather than waterfall phases.

Vertical Slice 1: Create Variables Template Infrastructure (Estimated: 2.5 days)

Complete increment including:

• Create templates/ansible/variables.yml.tera template file
• Implement AnsibleVariablesContext with validation
• Implement AnsibleVariablesTemplate wrapper
• Implement VariablesTemplateRenderer orchestrator
• Integrate into AnsibleTemplateRenderer workflow
• Write unit tests for all components
• Add Rustdoc documentation
• Verify with linters and tests

Outcome: System works with new variables infrastructure in place, fully tested and documented.

See: Issue #105

Vertical Slice 2: Convert Firewall to Variables Pattern (Estimated: 4.5 days)

Complete increment including:

• Convert configure-firewall.yml.tera → configure-firewall.yml (static)
• Add vars_files: [variables.yml] to playbook
• Register as static template in copy_static_templates()
• Update AnsibleClient to accept extra arguments
• Update all call sites
• Remove firewall renderer from workflow
• Delete old firewall renderer/wrapper code (~500 lines)
• Update template system architecture documentation
• Update contributing templates guide
• Update templates README
• Run full test suite (unit, config, linters)
• Verify build directory structure
• Document E2E test plan for human reviewer

Outcome: System works with firewall using variables pattern, old code removed, documentation updated, all tests passing.

See: Issue #106

Total Estimated Time: 7 days (2.5 + 4.5)

Dependencies

• Convert Firewall Template to Variables Pattern (Complete Vertical Slice) #106 requires Create Variables Template Infrastructure #105 to be completed (variables template must exist)

High-Level Architecture Changes

Before Refactoring

Current Architecture (2 Tera templates + per-template infrastructure):

templates/ansible/
  ├── inventory.yml.tera              (Tera: connection details)
  └── configure-firewall.yml.tera     (Tera: with ssh_port variable)

src/infrastructure/.../ansible/template/
  ├── wrappers/
  │   ├── inventory/
  │   │   ├── mod.rs                  (InventoryTemplate wrapper)
  │   │   └── context.rs              (InventoryContext)
  │   └── firewall_playbook/          (~150 lines)
  │       ├── mod.rs                  (FirewallPlaybookTemplate wrapper)
  │       └── context.rs              (FirewallPlaybookContext)
  └── renderer/
      ├── inventory.rs                (InventoryTemplateRenderer)
      └── firewall_playbook.rs        (~350 lines - dedicated renderer)

After Refactoring

New Architecture (2 Tera templates + centralized variables):

templates/ansible/
  ├── inventory.yml.tera              (Tera: connection details - UNCHANGED)
  ├── variables.yml.tera              (Tera: NEW - centralized system variables)
  └── configure-firewall.yml          (Static: uses vars_files)

src/infrastructure/.../ansible/template/
  ├── wrappers/
  │   ├── inventory/
  │   │   ├── mod.rs                  (InventoryTemplate wrapper - UNCHANGED)
  │   │   └── context.rs              (InventoryContext - UNCHANGED)
  │   └── variables/                  (NEW ~150 lines)
  │       ├── mod.rs                  (AnsibleVariablesTemplate wrapper)
  │       └── context.rs              (AnsibleVariablesContext)
  └── renderer/
      ├── inventory.rs                (InventoryTemplateRenderer - UNCHANGED)
      └── variables.rs                (NEW ~200 lines - VariablesTemplateRenderer)

[DELETED: ~500 lines]
  - wrappers/firewall_playbook/ directory
  - renderer/firewall_playbook.rs file

Key Changes

• Added: variables.yml.tera template + supporting Rust infrastructure (~350 lines)
• Converted: configure-firewall.yml.tera → static configure-firewall.yml (no rendering needed)
• Deleted: Firewall playbook renderer and wrapper (~500 lines)
• Net Result: ~150 lines less code, simpler architecture

Acceptance Criteria

Architecture

• Variables Template Created: templates/ansible/variables.yml.tera exists with system configuration variables
• Firewall Template Converted: configure-firewall.yml is static (no .tera extension) and uses vars_files
• Inventory Template Unchanged: inventory.yml.tera remains as Tera template (inventory limitation)
• Old Components Removed: firewall_playbook renderer and wrapper deleted (~500 lines)
• AnsibleClient Enhanced: Accepts optional extra arguments for passing variables file

Functionality

• Functionality Preserved: All existing Ansible functionality continues to work
• Variables Loaded: Playbooks correctly load variables from variables.yml via -e @variables.yml
• Firewall Configuration: UFW firewall configures correctly using centralized variables
• SSH Access Maintained: SSH connectivity preserved throughout deployment
• Pattern Established: Clear pattern documented for adding future service variables

Code Quality

• Tests Pass: All unit tests pass (cargo test)
• Config Tests Pass: E2E configuration tests pass (cargo run --bin e2e-config-tests)
• Linters Pass: All linters pass (cargo run --bin linter all)
• No Compilation Errors: Project compiles without warnings or errors
• Test Coverage: Comprehensive tests for AnsibleVariablesContext, AnsibleVariablesTemplate, and VariablesTemplateRenderer

Documentation

• Architecture Doc Updated: docs/technical/template-system-architecture.md documents the variables pattern
• Contributing Guide Updated: docs/contributing/templates.md explains how to use centralized variables
• Templates README Updated: templates/ansible/README.md documents the variables pattern

Benefits

Architectural Consistency

• Matches OpenTofu's successful variables.tfvars.tera pattern
• Consistent approach to variable management across infrastructure tools
• Single source of truth for environment-specific values

Reduced Complexity

• Only 1 Tera template instead of multiple
• Less Rust boilerplate for template handling
• Simpler debugging and maintenance

Future-Proofing

• Easy pattern for adding new services (just add variables, write static playbook)
• Scalable approach for the full roadmap implementation
• Clear separation of concerns (variables vs. logic)

Developer Experience

• Easier to understand variable flow
• Centralized variable management
• Reduced cognitive overhead when adding new features

Related Documentation

• Full epic document: docs/issues/19-refactor-ansible-templates-variables-pattern.md
• Subissue Create Variables Template Infrastructure #105: docs/issues/105-create-variables-template.md
• Subissue Convert Firewall Template to Variables Pattern (Complete Vertical Slice) #106: docs/issues/106-convert-firewall-template-to-static.md
• Template System Architecture
• Parent Epic #16