Skip to content

fix(workflows): resolve step vars in inline shell/atmos/exec steps - #2711

Merged
Erik Osterman (Cloud Posse) (osterman) merged 3 commits into
cloudposse:mainfrom
sgtoj:fix/workflow-step-variable-templating
Jul 9, 2026
Merged

fix(workflows): resolve step vars in inline shell/atmos/exec steps#2711
Erik Osterman (Cloud Posse) (osterman) merged 3 commits into
cloudposse:mainfrom
sgtoj:fix/workflow-step-variable-templating

Conversation

@sgtoj

@sgtoj Brian Ojeda (sgtoj) commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What

Workflow step types handled inline (shell, atmos, exec) now resolve step-variable templates — {{ .steps.*.value }}, {{ .env.* }}, {{ .flags.* }}, plus Sprig/Gomplate functions — in both the step command and the step env: values, matching how custom command steps already behave.

workflows:
  deploy:
    steps:
      - name: component
        type: format
        content: "vpc"
      - type: atmos
        command: terraform apply {{ .steps.component.value }} -auto-approve
        env:
          COMPONENT: "{{ .steps.component.value }}"

Before this change, terraform apply {{ .steps.component.value }} -auto-approve was passed through verbatim (literal {{ .steps.component.value }}); now it resolves to terraform apply vpc -auto-approve.

Why

This is a bug, not a new feature. The documented contract (PRD goal “capture step outputs for use in subsequent steps via Go templates”, and its own examples using type: atmos / type: shell command steps) is that any step can consume prior steps' outputs. Handler-routed step types (toast, markdown, container, the interactive prompts, …) and custom command steps already resolved these templates; workflow shell/atmos/exec steps were the outlier — they executed the raw command and merged env: values without running either through the step-variable engine, so the templates were emitted literally. It went unnoticed because the shipped examples only surface step values through handler-routed display steps, never a raw shell/atmos command.

How

  • ExecuteWorkflow configures the workflow step executor with the same template engine as the custom command executor (cmd/cmd_utils.go): the full Atmos renderer (Sprig/Gomplate), multi-pass rendering, and flag protection — so templating behaves identically in workflows and custom commands.
  • Inline step commands and env: values are resolved through that engine before execution.
  • New Variables.ResolveWith applies a per-call environment overlay without mutating the shared executor env, so a step's environment is visible as {{ .env.* }} without leaking across steps.
  • prepareStepEnvironment stays a pure merger (its existing tests are unchanged); env values are resolved ahead of it.

Commands and env: values without template markers are returned unchanged.

Testing

  • Unit tests: Variables.ResolveWith (overlay + non-mutation), command resolution (.steps + env overlay, non-leak, Sprig parity, invalid-template error), and env: resolution.
  • End-to-end (built binary): a workflow feeding a captured value into a shell step's command, env:, and a Sprig function all resolve correctly.
  • go build ./..., pkg/runner/step + internal/exec suites, and atmos lint changed are clean.

References

  • docs/fixes/2026-07-07-workflow-step-variable-templating.md
  • Reference implementation mirrored: cmd/cmd_utils.go (custom command step executor)

@sgtoj
Brian Ojeda (sgtoj) requested a review from a team as a code owner July 9, 2026 17:33
@atmos-pro

atmos-pro Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@github-actions github-actions Bot added the size/m Medium size PR label Jul 9, 2026
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: eace3e2d-9176-433e-9e52-9986fe7b48ba

📥 Commits

Reviewing files that changed from the base of the PR and between 04eb9ff and 526a376.

📒 Files selected for processing (7)
  • docs/fixes/2026-07-07-workflow-step-variable-templating.md
  • internal/exec/workflow_command_templating.go
  • internal/exec/workflow_command_templating_test.go
  • internal/exec/workflow_utils.go
  • pkg/runner/step/variables.go
  • pkg/runner/step/variables_test.go
  • website/docs/workflows/workflows/workflow/steps/index.mdx
✅ Files skipped from review due to trivial changes (1)
  • website/docs/workflows/workflows/workflow/steps/index.mdx
🚧 Files skipped from review as they are similar to previous changes (5)
  • internal/exec/workflow_command_templating.go
  • pkg/runner/step/variables.go
  • internal/exec/workflow_command_templating_test.go
  • internal/exec/workflow_utils.go
  • pkg/runner/step/variables_test.go

📝 Walkthrough

Walkthrough

Adds workflow step templating for inline shell, atmos, and exec commands and env: values, routes them through the shared template engine, and wires the resolution into workflow execution. Adds Variables.ResolveWith, tests, and documentation updates.

Changes

Workflow step templating

Layer / File(s) Summary
Per-call env overlay on Variables
pkg/runner/step/variables.go, pkg/runner/step/variables_test.go
Adds Variables.ResolveWith to resolve templates against a per-call env overlay merged with persisted env, without mutating persisted state; tests confirm no leakage and overlay precedence.
Workflow command/env templating helpers
internal/exec/workflow_command_templating.go, internal/exec/workflow_command_templating_test.go
Adds workflowCommandSupportsTemplating, resolveWorkflowStepCommand, resolveWorkflowStepEnvs, and resolveWorkflowStepEnv to resolve inline commands and env maps via shared step executor variables, with tests for supported types, rendering, no-op cases, and error handling.
ExecuteWorkflow integration
internal/exec/workflow_utils.go
Adds workflowTemplatePasses, configures the step executor's ProcessTmpl renderer with flag protection, resolves workflow/step env before prepareStepEnvironment, and resolves inline commands before execution, marking steps as failed on resolution errors.
Documentation of the fix and templating usage
docs/fixes/2026-07-07-workflow-step-variable-templating.md, website/docs/workflows/workflows/workflow/steps/index.mdx
Documents the bug, root cause, and fix; adds a "Referencing values from other steps" section with a YAML example for commands and env.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • cloudposse/atmos#2050: Extends the workflow env-map merging path that this PR templates before prepareStepEnvironment.

Suggested labels: patch

Suggested reviewers: aknysh, osterman

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: resolving workflow step variables in inline shell, atmos, and exec steps.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…/exec commands and env

Workflow step types handled inline (shell, atmos, exec) ran their raw
command and merged their env: values without passing either through the
step-variable template engine, so {{ .steps.*.value }} / {{ .env.* }} /
{{ .flags.* }} were emitted literally. Handler-routed step types and custom
command steps already resolve these; workflows were the outlier, breaking the
documented 'use step outputs in subsequent steps' contract.

ExecuteWorkflow now resolves inline step commands and env: values through the
same template engine custom command steps use (full Atmos renderer with
Sprig/Gomplate + multi-pass), so behavior is identical in both. A new
Variables.ResolveWith applies a per-call env overlay without mutating the
shared executor env, avoiding cross-step env leakage.
@sgtoj
Brian Ojeda (sgtoj) force-pushed the fix/workflow-step-variable-templating branch from 04eb9ff to 526a376 Compare July 9, 2026 18:08
@osterman

Copy link
Copy Markdown
Member

CodeRabbit (@coderabbitai) is .steps.<name>.values the right convention or .steps.<name>.outputs. I thought we were using the latter.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n '\.steps\.' --type=go -C2 | head -100
echo "---outputs---"
rg -n 'steps\[.*\]\.Outputs|StepResult|\.values\b|\.outputs\b' --type=go -C2 pkg/runner/step | head -100
echo "---mdx---"
rg -n 'steps\.' website/docs/workflows -C2
echo "---variables.go outline---"
fd variables.go pkg/runner/step
ast-grep outline pkg/runner/step/variables.go --items all
</br>
</br>
</br>
</br>
</br>

Length of output: 534


🏁 Script executed:

#!/bin/bash
fd variables.go pkg/runner/step
echo "---"
cat pkg/runner/step/variables.go
echo "---struct with steps---"
rg -n '"steps"|steps\[' --type=go -C3 pkg/runner
echo "---mdx file---"
fd index.mdx website/docs/workflows/workflows/workflow/steps
cat website/docs/workflows/workflows/workflow/steps/index.mdx | sed -n '1,200p'
</br>

Length of output: 25122


@osterman Both exist, but they mean different things — not a naming inconsistency.

  • .steps.<name>.value / .steps.<name>.values — the step's own built-in result(s). value is singular (e.g., a choose step's picked option), values is the plural form for multi-select/multi-value steps. No declaration needed; it's always available.
  • .steps.<name>.outputs — user-declared named outputs, populated only when the step defines an outputs: map. Each entry there is itself a template expression evaluated against that step's result (see Variables.SetWithOutputs / ResolveOutputs in pkg/runner/step/variables.go).

So for the new doc section ("Referencing values from other steps"), .values is correct since it's documenting the step's raw multi-value result, not the declared-outputs mechanism. If you want, the doc could add a line clarifying the distinction and cross-link to the outputs field docs to avoid this exact confusion for readers.

@osterman Erik Osterman (Cloud Posse) (osterman) added the patch A minor, backward compatible change label Jul 9, 2026
@osterman
Erik Osterman (Cloud Posse) (osterman) merged commit 33d1a0d into cloudposse:main Jul 9, 2026
78 checks passed
@atmos-pro

atmos-pro Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.46377% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.20%. Comparing base (c839ac3) to head (3a1d11b).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
internal/exec/workflow_utils.go 50.00% 9 Missing and 2 partials ⚠️
internal/exec/workflow_command_templating.go 81.81% 3 Missing and 3 partials ⚠️
pkg/runner/step/variables.go 85.71% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2711   +/-   ##
=======================================
  Coverage   81.19%   81.20%           
=======================================
  Files        1632     1633    +1     
  Lines      154292   154360   +68     
=======================================
+ Hits       125279   125343   +64     
- Misses      22091    22094    +3     
- Partials     6922     6923    +1     
Flag Coverage Δ
unittests 81.20% <72.46%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pkg/runner/step/variables.go 93.36% <85.71%> (-0.48%) ⬇️
internal/exec/workflow_command_templating.go 81.81% <81.81%> (ø)
internal/exec/workflow_utils.go 73.77% <50.00%> (-0.90%) ⬇️

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

These changes were released in v1.223.0-rc.7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch A minor, backward compatible change size/m Medium size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants