Tool Dependencies Integration

Related PRDs: Toolchain Implementation | Lock File Support | Custom Hooks

Overview

Integrate tool dependencies into Atmos workflows, custom commands, and components, enabling automatic tool installation and version management based on declarative configuration.

Requirements & Implementation Status

Requirement	Scope	Status	Reference
Resolve `dependencies.tools` from declaration scope	All scopes	Implemented	`pkg/dependencies/` resolver
Auto-install missing tools before execution	Workflows	Implemented	`pkg/workflow/executor.go` calls `EnsureTools` before running each step
Auto-install missing tools before execution	Custom commands	Implemented	`cmd/cmd_utils.go` calls `EnsureTools` before invoking the user's command
Auto-install missing tools before execution	Ansible components	Implemented	`pkg/component/ansible/executor.go::ensureDependencies` resolves deps, calls `EnsureTools`, builds toolchain PATH
Auto-install missing tools before execution	Hooks	Implemented	`pkg/hooks/hooks.go::preflight` resolves `dependencies.tools` from the component section, installs, builds toolchain PATH for the subprocess
Auto-install missing tools before execution	Terraform components	Not implemented	`atmos terraform plan/apply` does not call `EnsureTools` for the terraform binary itself. Tools that the hook engine needs are installed by the hook pre-flight; tools the terraform invocation itself needs still rely on operator PATH.
Auto-install missing tools before execution	Helmfile components	Not implemented	Same gap as terraform components.
Auto-install missing tools before execution	Packer components	Not implemented	Same gap as terraform components.
Normalize `"latest"` so `BuildToolchainPATH` resolves to a real bin directory	All scopes	Not implemented	`isConstraint("latest") == false`, so `resolveConstraints` does not rewrite the version, and `BuildToolchainPATH` constructs a bin directory containing the literal string `"latest"`, which does not exist on disk. Users must pin a concrete version or use a SemVer constraint (e.g., `~> 0.10`). The fix is either to treat `"latest"` as a constraint resolving to the highest installed/available version, or to have `BuildToolchainPATH` look up the resolved version.
Validate SemVer constraints (`~>`, `^`, exact)	All scopes	Implemented	`pkg/dependencies/` resolver
Inherit `dependencies` through stack imports with deep merge	Stack scopes	Implemented	Standard Atmos import + merge pipeline
Surface a clear error when a child version does not satisfy a parent constraint	All scopes	Implemented	`pkg/dependencies/` resolver returns a structured validation error

Problem Statement

No Automatic Installation: Users must manually install tools before running commands.
No Dependency Declaration: Cannot declare tool requirements at component, stack, workflow, or command level.
No Version Enforcement: No way to ensure specific tool versions are used for specific contexts.
Manual PATH Management: Users must manage PATH environment variables themselves.

Goals

Declarative Dependencies: Allow tool dependencies to be declared at multiple levels (stack, component, workflow, command)
Automatic Installation: Auto-install missing tools before execution
Version Constraints: Support SemVer constraints with validation
Inheritance: Tool dependencies inherit through stack imports with deep merge
Seamless Integration: No changes to existing user workflows - just works

Non-Goals

Dependency conflict resolution (install all required versions)
Tool upgrade management (use declared versions)
Custom registries (Aqua registry only in v1)

Architecture

Configuration Levels

Tool dependencies can be declared at multiple scopes with proper inheritance and merging:

Stack Configuration Scopes

For components (terraform/helmfile/packer), dependencies are resolved from stack configuration files with 3 scopes (lowest to highest priority):

Global Scope - Top-level dependencies in stack files (applies to all components)
Component Type Scope - terraform.dependencies / helmfile.dependencies / packer.dependencies (applies to all components of that type)
Component Instance Scope - components.terraform.vpc.dependencies (applies to specific component)

Stack inheritance applies to all 3 scopes through Atmos's existing import mechanism.

Workflow and Command Scopes

Additionally, for workflows and custom commands:

Workflow Scope - workflows.<name>.dependencies in atmos.yaml
Custom Command Scope - commands[].dependencies in atmos.yaml

Schema Structure

// Dependencies declares required tools and component dependencies.
type Dependencies struct {
    // Tools maps tool names to version constraints (e.g., "terraform": "1.5.0" or "latest").
    Tools map[string]string `yaml:"tools,omitempty" json:"tools,omitempty" mapstructure:"tools"`
    // Components lists component dependencies that must be applied before this component.
    // This is the recommended location for component dependencies (replaces settings.depends_on).
    // Uses list format with append merge behavior (child lists extend parent lists).
    Components []ComponentDependency `yaml:"components,omitempty" json:"components,omitempty" mapstructure:"components"`
}

Component Dependencies

In addition to tool dependencies, the dependencies section also supports component dependencies (previously settings.depends_on):

dependencies:
  tools:
    terraform: "1.9.8"
  components:
    - component: vpc
    - component: rds
      stack: tenant1-ue1-prod

Component dependencies define execution order and are used by:

atmos describe dependents - Find components that depend on a given component
atmos describe affected - Find components affected by changes
CI/CD integrations (Spacelift, Atlantis, etc.)

Key differences from settings.depends_on:

Uses list format (not map with numeric keys)
Uses append merge behavior (child lists extend parent lists)
Located under dependencies alongside tool dependencies

See the Component Dependencies documentation (website/docs/stacks/dependencies/components.mdx) for detailed usage.

Usage in Stack Configuration:

# Scope 1: Global dependencies (applies to all components)
dependencies:
  tools:
    aws-cli: "^2.0.0"       # All components get aws-cli
    jq: "latest"            # All components get jq

# Scope 2: Component type dependencies (applies to all terraform components)
terraform:
  dependencies:
    tools:
      terraform: "~> 1.10.0"  # All terraform components
      tflint: "^0.54.0"       # All terraform components

# Scope 3: Component instance dependencies (specific component)
components:
  terraform:
    vpc:
      dependencies:
        tools:
          terraform: "1.10.3"  # Must satisfy terraform: "~> 1.10.0" from scope 2
          checkov: "latest"     # Component-specific tool

Inheritance Example:

# stacks/catalog/terraform.yaml (imported by other stacks)
terraform:
  dependencies:
    tools:
      terraform: "~> 1.10.0"
      tflint: "latest"

# stacks/prod/us-east-1.yaml
import:
  - catalog/terraform

# Inherits terraform.dependencies from catalog
# Can override at component level:
components:
  terraform:
    vpc:
      dependencies:
        tools:
          terraform: "1.10.3"  # Overrides but must satisfy ~> 1.10.0

Resolution Algorithm

For component execution (terraform/helmfile/packer):
  1. Load stack configuration with imports (Atmos handles this)
  2. Collect dependencies from stack config in order:
     a. Extract global dependencies (top-level dependencies.tools)
     b. Extract component-type dependencies (terraform.dependencies.tools)
     c. Extract component instance dependencies (components.terraform.vpc.dependencies.tools)
  3. Deep merge with override (higher priority overrides lower)
  4. Validate constraints (child version must satisfy parent constraint)
  5. Return merged dependency map

For workflow execution:
  1. Load workflow definition from atmos.yaml
  2. Extract workflow.dependencies.tools
  3. Return workflow dependency map

For custom command execution:
  1. Load command definition from atmos.yaml
  2. Extract command.dependencies.tools
  3. Return command dependency map

Auto-Install Hook

Before executing any command, check tool dependencies and auto-install if missing:

Before execution:
  1. Resolve dependencies for current context
  2. For each tool in dependencies:
     a. Parse tool@version specifier
     b. Check if version installed (.tools/bin/owner/repo/version/)
     c. If missing: toolchain.InstallExec(tool@version)
  3. Update PATH to include .tools/bin with correct versions
  4. Execute command

Implementation Plan

Each phase below carries a Status: line describing where it stands today. The detailed code sketches that follow each phase document the intended shape of the work; consult the actual source for the canonical implementation.

Phase 1: Schema Updates

Status: Implemented.

1.1 Add Dependencies Struct

File: pkg/schema/dependencies.go (new)

package schema

// Dependencies declares required tools and component dependencies.
type Dependencies struct {
	Tools      map[string]string     `yaml:"tools,omitempty" json:"tools,omitempty" mapstructure:"tools"`
	Components []ComponentDependency `yaml:"components,omitempty" json:"components,omitempty" mapstructure:"components"`
}

1.2 Update Existing Schemas

File: pkg/schema/workflow.go

type WorkflowDefinition struct {
	Description  string         `yaml:"description,omitempty" json:"description,omitempty" mapstructure:"description"`
	Dependencies *Dependencies  `yaml:"dependencies,omitempty" json:"dependencies,omitempty" mapstructure:"dependencies"`
	Steps        []WorkflowStep `yaml:"steps" json:"steps" mapstructure:"steps"`
	Stack        string         `yaml:"stack,omitempty" json:"stack,omitempty" mapstructure:"stack"`
}

File: pkg/schema/command.go

type Command struct {
	Name            string                 `yaml:"name" json:"name" mapstructure:"name"`
	Description     string                 `yaml:"description" json:"description" mapstructure:"description"`
	Dependencies    *Dependencies          `yaml:"dependencies,omitempty" json:"dependencies,omitempty" mapstructure:"dependencies"`
	Env             []CommandEnv           `yaml:"env" json:"env" mapstructure:"env"`
	// ... existing fields
}

Stack YAML (no schema change needed - uses map[string]any):

Top-level dependencies.tools
Component-level components.terraform.vpc.dependencies.tools

Phase 2: Dependency Resolution

Status: Implemented.

2.1 Dependency Resolver Package

File: pkg/dependencies/resolver.go (new)

package dependencies

import (
	"fmt"
	"github.com/cloudposse/atmos/pkg/schema"
)

// Resolver resolves tool dependencies with inheritance and validation.
type Resolver struct {
	atmosConfig *schema.AtmosConfiguration
}

// NewResolver creates a new dependency resolver.
func NewResolver(atmosConfig *schema.AtmosConfiguration) *Resolver

// ResolveComponentDependencies resolves tool dependencies for a component.
// Merges: stack catalog → stack instance → component catalog → component instance
func (r *Resolver) ResolveComponentDependencies(
	componentType string,
	component string,
	stack string,
) (map[string]string, error)

// ResolveWorkflowDependencies resolves tool dependencies for a workflow.
func (r *Resolver) ResolveWorkflowDependencies(
	workflow string,
) (map[string]string, error)

// ResolveCommandDependencies resolves tool dependencies for a custom command.
func (r *Resolver) ResolveCommandDependencies(
	command string,
) (map[string]string, error)

// mergeDependencies merges child dependencies into parent with validation.
func mergeDependencies(
	parent map[string]string,
	child map[string]string,
) (map[string]string, error)

// validateConstraint validates that specific version satisfies constraint.
func validateConstraint(version string, constraint string) error

2.2 SemVer Constraint Validation

Use existing github.com/Masterminds/semver/v3 dependency:

import "github.com/Masterminds/semver/v3"

func validateConstraint(version string, constraint string) error {
	// "latest" always satisfies
	if constraint == "latest" || version == "latest" {
		return nil
	}

	// Parse constraint (e.g., "~> 1.10.0", "^0.54.0")
	c, err := semver.NewConstraint(constraint)
	if err != nil {
		return fmt.Errorf("invalid constraint %q: %w", constraint, err)
	}

	// Parse version
	v, err := semver.NewVersion(version)
	if err != nil {
		return fmt.Errorf("invalid version %q: %w", version, err)
	}

	// Validate
	if !c.Check(v) {
		return fmt.Errorf("version %q does not satisfy constraint %q", version, constraint)
	}

	return nil
}

Phase 3: Auto-Install Integration

Status: Partial. Workflows, custom commands, ansible components, and hooks are wired up. Terraform, helmfile, and packer components do not yet auto-install their declared tools — the component executors do not call EnsureTools.

3.1 Tool Installer Package

File: pkg/dependencies/installer.go (new)

package dependencies

import (
	"github.com/cloudposse/atmos/pkg/toolchain"
)

// Installer handles automatic tool installation.
type Installer struct{}

// NewInstaller creates a new tool installer.
func NewInstaller() *Installer

// EnsureTools ensures all required tools are installed.
// Installs missing tools automatically.
func (i *Installer) EnsureTools(dependencies map[string]string) error {
	for tool, version := range dependencies {
		if err := i.ensureTool(tool, version); err != nil {
			return err
		}
	}
	return nil
}

// ensureTool ensures a specific tool version is installed.
func (i *Installer) ensureTool(tool string, version string) error {
	// Check if already installed
	binaryPath, err := toolchain.FindToolBinary(tool, version)
	if err == nil && binaryPath != "" {
		return nil // Already installed
	}

	// Install missing tool
	toolSpec := fmt.Sprintf("%s@%s", tool, version)
	return toolchain.InstallExec(toolSpec)
}

3.2 Component Execution Hook

File: internal/exec/terraform_component_executor.go (modify existing)

func ExecuteTerraformComponent(
	atmosConfig *schema.AtmosConfiguration,
	component string,
	stack string,
	// ... other params
) error {
	defer perf.Track(atmosConfig, "exec.ExecuteTerraformComponent")()

	// Resolve tool dependencies
	resolver := dependencies.NewResolver(atmosConfig)
	deps, err := resolver.ResolveComponentDependencies("terraform", component, stack)
	if err != nil {
		return fmt.Errorf("failed to resolve dependencies: %w", err)
	}

	// Auto-install missing tools
	installer := dependencies.NewInstaller()
	if err := installer.EnsureTools(deps); err != nil {
		return fmt.Errorf("failed to ensure tools: %w", err)
	}

	// Update PATH to include installed tools
	if err := updatePathForTools(deps); err != nil {
		return fmt.Errorf("failed to update PATH: %w", err)
	}

	// Continue with existing execution logic
	// ...
}

3.3 Workflow Execution Hook

File: internal/exec/workflow.go (modify existing)

func ExecuteWorkflow(
	atmosConfig *schema.AtmosConfiguration,
	workflow string,
	// ... other params
) error {
	defer perf.Track(atmosConfig, "exec.ExecuteWorkflow")()

	// Resolve workflow dependencies
	resolver := dependencies.NewResolver(atmosConfig)
	deps, err := resolver.ResolveWorkflowDependencies(workflow)
	if err != nil {
		return fmt.Errorf("failed to resolve workflow dependencies: %w", err)
	}

	// Auto-install missing tools
	installer := dependencies.NewInstaller()
	if err := installer.EnsureTools(deps); err != nil {
		return fmt.Errorf("failed to ensure tools: %w", err)
	}

	// Continue with existing workflow execution
	// ...
}

3.4 Custom Command Hook

File: internal/exec/custom_command.go (modify existing)

Similar pattern to workflow execution.

Phase 4: PATH Management

Status: Implemented, with one open edge case. BuildToolchainPATH builds the subprocess PATH so installed pinned versions win over any inherited PATH. "latest" is not normalized to a concrete version before the bin-directory path is constructed, so users currently must pin a concrete version or a SemVer constraint — see the requirements table above.

File: pkg/dependencies/path.go (new)

package dependencies

import (
	"fmt"
	"os"
	"path/filepath"
	"strings"
)

// updatePathForTools updates PATH to include tool binaries.
func updatePathForTools(dependencies map[string]string) error {
	toolsDir := os.Getenv("ATMOS_TOOLCHAIN_TOOLS_DIR")
	if toolsDir == "" {
		toolsDir = ".tools"
	}

	var paths []string
	for tool, version := range dependencies {
		// Resolve tool to owner/repo
		owner, repo, err := resolveToolPath(tool)
		if err != nil {
			return err
		}

		// Add versioned bin directory to PATH
		binPath := filepath.Join(toolsDir, "bin", owner, repo, version)
		paths = append(paths, binPath)
	}

	// Prepend to existing PATH
	currentPath := os.Getenv("PATH")
	newPath := strings.Join(append(paths, currentPath), string(os.PathListSeparator))
	os.Setenv("PATH", newPath)

	return nil
}

Testing Strategy

Unit Tests

Dependency Resolution
- Test stack-level inheritance
- Test component-level inheritance
- Test deep merge behavior
- Test constraint validation
SemVer Constraint Validation
- Test tilde constraints (~> 1.10.0)
- Test caret constraints (^0.54.0)
- Test exact versions
- Test "latest" handling
Auto-Install Logic
- Test tool already installed (skip)
- Test tool missing (install)
- Test installation failure handling

Integration Tests

Component Execution
- Component with dependencies → auto-install → execute
- Component with invalid constraint → error
Workflow Execution
- Workflow with dependencies → auto-install → execute
Custom Command Execution
- Command with dependencies → auto-install → execute

Migration Path

Backward Compatibility

No breaking changes: Tool dependencies are optional
Existing workflows continue working: No dependencies = no auto-install
Opt-in adoption: Users add dependencies when ready

Migration Guide

Before (manual installation):

atmos toolchain install terraform@1.10.3
atmos terraform plan vpc -s prod

After (automatic installation):

# stacks/prod.yaml
dependencies:
  tools:
    terraform: "1.10.3"

components:
  terraform:
    vpc:
      # ...

atmos terraform plan vpc -s prod  # Auto-installs terraform@1.10.3

Configuration Examples

Stack-Level Dependencies

# stacks/catalog/base.yaml
dependencies:
  tools:
    terraform: "~> 1.10.0"
    tflint: "^0.54.0"
    trivy: "~> 0.70.0"

components:
  terraform:
    vpc:
      vars:
        name: vpc

Component-Level Dependencies

# stacks/catalog/terraform/database.yaml
components:
  terraform:
    rds:
      dependencies:
        tools:
          terraform: "~> 1.9.0"  # Older version for legacy DB
          checkov: "^3.0.0"
      vars:
        engine: postgres

Workflow Dependencies

# stacks/workflows/deploy.yaml
workflows:
  deploy-infra:
    description: Deploy infrastructure
    dependencies:
      tools:
        terraform: "~> 1.10.0"
        aws-cli: "^2.0.0"
        jq: "latest"
    steps:
      - name: plan
        command: terraform plan vpc -s prod
      - name: apply
        command: terraform apply vpc -s prod

Custom Command Dependencies

# atmos.yaml
commands:
  - name: deploy
    description: Deploy with required tools
    dependencies:
      tools:
        terraform: "~> 1.10.0"
        kubectl: "^1.32.0"
    steps:
      - atmos terraform plan vpc -s {{ .stack }}
      - atmos terraform apply vpc -s {{ .stack }}

Error Handling

Constraint Validation Errors

Error: Tool dependency constraint validation failed
  Component: vpc
  Stack: prod
  Tool: terraform
  Parent constraint: ~> 1.10.0
  Child version: 1.9.8

  The version "1.9.8" does not satisfy parent constraint "~> 1.10.0"

Installation Errors

Error: Failed to install required tool
  Tool: terraform@1.10.3
  Reason: Failed to download from GitHub: rate limit exceeded

  Suggestion: Set ATMOS_GITHUB_TOKEN environment variable

Performance Considerations

Tool Check Caching: Cache "already installed" checks per execution
Parallel Installation: Install multiple tools concurrently
Registry Caching: Reuse existing toolchain registry cache

Security Considerations

Version Pinning: Encourage specific versions over "latest"
Constraint Validation: Prevent downgrade attacks via constraints
GitHub Token: Support authenticated downloads for rate limits

Metrics and Observability

Track via performance monitoring:

dependencies.resolve.component - Dependency resolution time
dependencies.resolve.workflow - Workflow dependency resolution
dependencies.install - Tool installation time
dependencies.validate - Constraint validation time

Future Enhancements

Lock Files: Generate .tool-versions.lock for reproducible builds
Custom Registries: Support private/custom tool registries
Dependency Caching: Shared cache across projects
Conflict Resolution: Smart handling of conflicting constraints
Tool Updates: atmos toolchain upgrade command

References

Toolchain PRD: docs/prd/toolchain-implementation.md
SemVer Spec: https://semver.org/
Aqua Registry: https://github.com/aquaproj/aqua-registry

Open Questions

Should we support multiple versions of the same tool in one execution context? (No - use constraints)
How to handle PATH priority when multiple components need different versions? (Per-component PATH setup)
Should we validate constraints at config load time or execution time? (Execution time - lazy validation)

Success Criteria

Success criteria describe what "done" looks like for the feature as a whole. See the Requirements & Implementation Status table for what currently meets each criterion and what does not.

Users can declare tool dependencies at any level (stack, component, workflow, custom command).
Tools are automatically installed before execution at every level where dependencies can be declared.
SemVer constraints are validated and enforced; child versions that do not satisfy a parent constraint produce a clear error.
Dependencies inherit through stack imports with deep merge.
Zero breaking changes to existing workflows — dependencies are opt-in.
Test coverage ≥ 80% for new code.

Uh oh!