--config with a conflicting array value in a second file makes stacks.included_paths unusable for stack discovery, while config get still reports the unchanged original value
Describe the Bug
Loading CLI config via --config main.yaml,fragment.yaml, where fragment.yaml sets a
different value for an array-typed key that main.yaml also sets (e.g.
stacks.included_paths), causes stack discovery (atmos list stacks,
atmos terraform test, etc.) to fail with No stacks found — even for manifests covered
by the original, unmodified value that both files agree should still apply.
Isolated the exact trigger with three otherwise-identical scenarios (same main.yaml, same
stack manifest at stacks/deploy/dev.yaml, covered by included_paths: [deploy/**/*]),
varying only the second --config file's value for that same key:
Second file sets stacks.included_paths to... |
list stacks |
[deploy/**/*] — identical to main.yaml's value |
✅ finds the stack |
[deploy/**/*, other/**/*] — a pure superset, still containing the original value |
❌ No stacks found |
[other/**/*] — a completely different value |
❌ No stacks found |
The middle case is the important one: the second file's value still contains
deploy/**/*, yet the stack manifest that path alone should cover is no longer found.
It gets more misleading: for the failing superset case, atmos config get stacks.included_paths reports - deploy/**/* — the original, single-element,
unchanged value — as if the merge attempt for this key had no effect at all. So the
config-inspection command says "everything is exactly as it was, nothing changed here,"
while the actual stack-discovery pipeline behaves as if the key were unusable, even for the
part of the value config get says is still there.
Root cause, traced to source, not just symptoms: atmos's own mergeConfigFile() (in
pkg/config/load.go) documents this general class of problem in its own comment, right
where it works around it for one specific key:
// Extract commands separately because viper.MergeConfig doesn't overwrite
// arrays. Commands are decoded through the YAML-function path so nested
// !include/!exec values are resolved before the command tree is merged.
newCommands, err := extractCommandsWithYamlFunctionsForFile(content, path)
...
// Now handle command merging manually.
// Merge commands: when duplicates exist, the file being processed (new) takes precedence.
if existingCommands != nil || newCommands != nil {
merged := mergeCommandArrays(existingCommands, newCommands)
v.Set(commandsKey, merged)
}
Atmos is aware that plain viper.MergeConfig() does not correctly merge array-typed keys
across multiple sequential merge calls, and has built a manual, custom merge path
(existingCommands/newCommands/mergeCommandArrays()) specifically for the top-level
commands array. No equivalent workaround exists for any other array-typed config key —
including stacks.included_paths, stacks.excluded_paths, and presumably others. For those
keys, a second --config file's conflicting value hits the same underlying viper limitation,
but without a custom merge path to correct it, and — worse than a plain no-op — the actual
value consumed by stack-processing ends up unusable, distinct from whatever config get
happens to still report from before the failed merge.
Expected Behavior
Either: (a) stacks.included_paths/excluded_paths (and other array-typed keys) get the
same manual merge treatment commands already receives, so multiple --config files
combine their array values correctly; or, at minimum, (b) atmos config get should never
report a value that diverges from what the rest of atmos actually uses for that same loaded
configuration — a stale "looks unchanged" report while processing silently breaks is worse
than a clear error.
Steps to Reproduce
Minimal, self-contained repro. Creates a throwaway project in a temp dir, then compares three
--config scenarios that differ only in the second file's included_paths value.
Copy-paste the whole block into a terminal:
REPRO_DIR="$(mktemp -d)"
echo "Reproducing in: $REPRO_DIR"
# Install atmos if not already on PATH (kept outside REPRO_DIR so atmos's own
# config-file discovery doesn't try to parse the binary itself as YAML)
if ! command -v atmos >/dev/null 2>&1; then
ATMOS_VERSION=1.225.0-rc.7
ATMOS_BIN_DIR="$(mktemp -d)"
curl -fsSL "https://github.com/cloudposse/atmos/releases/download/v${ATMOS_VERSION}/atmos_${ATMOS_VERSION}_linux_amd64" -o "$ATMOS_BIN_DIR/atmos"
chmod +x "$ATMOS_BIN_DIR/atmos"
ATMOS="$ATMOS_BIN_DIR/atmos"
else
ATMOS=atmos
fi
$ATMOS version
cd "$REPRO_DIR"
git init -q
mkdir -p stacks/deploy components/terraform/my-component
cat > atmos.yaml <<'EOF'
base_path: ''
components:
terraform:
base_path: components/terraform
stacks:
base_path: stacks
included_paths:
- deploy/**/*
name_pattern: '{stage}'
EOF
cat > stacks/deploy/dev.yaml <<'EOF'
vars:
stage: dev
components:
terraform:
my-component:
vars: {}
EOF
cat > components/terraform/my-component/main.tf <<'EOF'
variable "stage" {
type = string
}
output "stage" {
value = var.stage
}
EOF
echo "CASE 1: single file, plain auto-discovery (baseline)"
$ATMOS list stacks
echo "CASE 2: --config with a second file setting the SAME included_paths value"
cat > fragment_same.yaml <<'EOF'
stacks:
included_paths:
- deploy/**/*
EOF
$ATMOS --config atmos.yaml,fragment_same.yaml list stacks
echo "CASE 3: --config with a second file adding an ADDITIONAL path - a superset,"
echo " still containing the original value"
cat > fragment_superset.yaml <<'EOF'
stacks:
included_paths:
- deploy/**/*
- other/**/*
EOF
$ATMOS --config atmos.yaml,fragment_superset.yaml list stacks || true
echo "config get for Case 3 - reports the stale, unchanged single-element value:"
$ATMOS --config atmos.yaml,fragment_superset.yaml config get stacks.included_paths
Cases 1 and 2 print dev. Case 3 prints ▶ No stacks found, followed by config get
reporting - deploy/**/* for the exact same loaded configuration.
Screenshots
No screenshots — CLI output only, included in Steps to Reproduce above. The exact failure
for Case 3:
immediately followed by, for the identical --config invocation:
Environment
Additional Context
Suggested fix direction: generalize the existing commands-array merge workaround in
mergeConfigFile() to cover stacks.included_paths, stacks.excluded_paths, and any other
array-typed AtmosConfiguration field that a project might reasonably split across multiple
--config files — or, short of a full fix, make config get surface a warning (or the
merge itself fail loudly) when a later --config file's value for a key can't be merged,
instead of silently keeping the stale value while downstream processing behaves as if the
key were something else entirely.
Relationship to existing issues: distinct from #2863 (fixed by #2864). That issue was
about the top-level base_path scalar never resolving to git root when loaded via
--config. This issue reproduces independently of base_path/git-root entirely (the
reproduction above never leaves the directory containing atmos.yaml) and is specific to how
array-typed nested keys merge — or fail to — across multiple --config files.
--config with a conflicting array value in a second file makes
stacks.included_pathsunusable for stack discovery, whileconfig getstill reports the unchanged original valueDescribe the Bug
Loading CLI config via
--config main.yaml,fragment.yaml, wherefragment.yamlsets adifferent value for an array-typed key that
main.yamlalso sets (e.g.stacks.included_paths), causes stack discovery (atmos list stacks,atmos terraform test, etc.) to fail withNo stacks found— even for manifests coveredby the original, unmodified value that both files agree should still apply.
Isolated the exact trigger with three otherwise-identical scenarios (same
main.yaml, samestack manifest at
stacks/deploy/dev.yaml, covered byincluded_paths: [deploy/**/*]),varying only the second
--configfile's value for that same key:stacks.included_pathsto...list stacks[deploy/**/*]— identical tomain.yaml's value[deploy/**/*, other/**/*]— a pure superset, still containing the original valueNo stacks found[other/**/*]— a completely different valueNo stacks foundThe middle case is the important one: the second file's value still contains
deploy/**/*, yet the stack manifest that path alone should cover is no longer found.It gets more misleading: for the failing superset case,
atmos config get stacks.included_pathsreports- deploy/**/*— the original, single-element,unchanged value — as if the merge attempt for this key had no effect at all. So the
config-inspection command says "everything is exactly as it was, nothing changed here,"
while the actual stack-discovery pipeline behaves as if the key were unusable, even for the
part of the value
config getsays is still there.Root cause, traced to source, not just symptoms: atmos's own
mergeConfigFile()(inpkg/config/load.go) documents this general class of problem in its own comment, rightwhere it works around it for one specific key:
Atmos is aware that plain
viper.MergeConfig()does not correctly merge array-typed keysacross multiple sequential merge calls, and has built a manual, custom merge path
(
existingCommands/newCommands/mergeCommandArrays()) specifically for the top-levelcommandsarray. No equivalent workaround exists for any other array-typed config key —including
stacks.included_paths,stacks.excluded_paths, and presumably others. For thosekeys, a second
--configfile's conflicting value hits the same underlying viper limitation,but without a custom merge path to correct it, and — worse than a plain no-op — the actual
value consumed by stack-processing ends up unusable, distinct from whatever
config gethappens to still report from before the failed merge.
Expected Behavior
Either: (a)
stacks.included_paths/excluded_paths(and other array-typed keys) get thesame manual merge treatment
commandsalready receives, so multiple--configfilescombine their array values correctly; or, at minimum, (b)
atmos config getshould neverreport a value that diverges from what the rest of atmos actually uses for that same loaded
configuration — a stale "looks unchanged" report while processing silently breaks is worse
than a clear error.
Steps to Reproduce
Minimal, self-contained repro. Creates a throwaway project in a temp dir, then compares three
--configscenarios that differ only in the second file'sincluded_pathsvalue.Copy-paste the whole block into a terminal:
Cases 1 and 2 print
dev. Case 3 prints▶ No stacks found, followed byconfig getreporting
- deploy/**/*for the exact same loaded configuration.Screenshots
No screenshots — CLI output only, included in Steps to Reproduce above. The exact failure
for Case 3:
immediately followed by, for the identical
--configinvocation:Environment
atmos terraform test#2863/fix(config): resolve git-root base_path for --config/--config-path #2864 fix,which only addressed top-level
base_pathresolution)--configsplit — no custom commands, no unusualsettings, no
base_pathgymnastics required to trigger thisAdditional Context
Suggested fix direction: generalize the existing
commands-array merge workaround inmergeConfigFile()to coverstacks.included_paths,stacks.excluded_paths, and any otherarray-typed
AtmosConfigurationfield that a project might reasonably split across multiple--configfiles — or, short of a full fix, makeconfig getsurface a warning (or themerge itself fail loudly) when a later
--configfile's value for a key can't be merged,instead of silently keeping the stale value while downstream processing behaves as if the
key were something else entirely.
Relationship to existing issues: distinct from #2863 (fixed by #2864). That issue was
about the top-level
base_pathscalar never resolving to git root when loaded via--config. This issue reproduces independently ofbase_path/git-root entirely (thereproduction above never leaves the directory containing
atmos.yaml) and is specific to howarray-typed nested keys merge — or fail to — across multiple
--configfiles.