Skip to content

Commit 6577f9e

Browse files
ostermanclaude
andcommitted
test(registry): widen timing margin in provider-mirror concurrency test
Fixes a flaky Windows Acceptance Tests failure: resolving 10 platforms took 784ms against a 750ms threshold, even though that's nowhere near the 1.5s serial floor the test guards against. Raises the bound to 4/5 of the serial floor (1200ms) for headroom against normal CI timing variance, Windows runners especially. No production code changed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent c8e5fed commit 6577f9e

2 files changed

Lines changed: 64 additions & 1 deletion

File tree

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Fix: widen the timing margin in the provider-mirror concurrency regression test
2+
3+
**Date:** 2026-08-06
4+
5+
## Summary
6+
7+
`TestProviderMirror_VersionResolvesPlatformsConcurrently`
8+
(`pkg/terraform/registry/provider_mirror_test.go`) failed on the Windows
9+
Acceptance Tests CI job: resolving 10 platforms took 784ms against a 750ms
10+
threshold. The test's own concurrency assertion was actually satisfied —
11+
784ms is nowhere near the 1.5s serial floor the test exists to catch — it
12+
just tripped an overly tight pass/fail bound under normal CI timing
13+
variance. No production code changed; this is a flaky-test fix.
14+
15+
## Context
16+
17+
The test spins up a local HTTP mirror server where each of 10 platform
18+
downloads sleeps a fixed 150ms (`delay`), then asserts that resolving the
19+
version endpoint (which fetches all 10 platform archives) completes in
20+
under half of the theoretical serial floor (`platformCount * delay / 2` =
21+
750ms) — proving the platforms are fetched concurrently (~1 delay) rather
22+
than serially (~10 delays = 1.5s). A concurrent run's actual wall time is
23+
one delay plus scheduling/HTTP/goroutine overhead; on a loaded or
24+
virtualized CI runner — Windows acceptance-test runners in particular are
25+
slower and more variable than local dev machines — that overhead alone can
26+
approach several hundred milliseconds, which the 750ms bound had no real
27+
margin for. This is not related to any other change on this branch;
28+
`pkg/terraform/registry/provider_mirror_test.go` was last touched by
29+
unrelated PRs on `main` (#2534, #2582).
30+
31+
## Changes
32+
33+
- `pkg/terraform/registry/provider_mirror_test.go`: raised the pass
34+
threshold from `serialFloor/2` (750ms) to `serialFloor*4/5` (1200ms).
35+
This keeps a wide, unambiguous gap from the 1.5s serial floor a true
36+
regression to serial resolution would hit, while tolerating realistic CI
37+
timing variance that a tighter bound doesn't survive.
38+
39+
## Validation
40+
41+
- `go build ./...` — clean.
42+
- `go test ./pkg/terraform/registry/... -run TestProviderMirror_VersionResolvesPlatformsConcurrently -v -count=5`
43+
— 5/5 pass, each completing in ~310ms (comfortably under the new 1200ms
44+
bound, comfortably above what a serial regression would need to trip the
45+
assertion).
46+
- `go test ./pkg/terraform/registry/...` (full package) — pass.
47+
- `atmos lint --changed` — 0 issues.
48+
- Did not reproduce the exact 784ms Windows CI timing locally (this
49+
machine isn't the flaky runner); the fix targets the assertion's margin,
50+
not the code path's actual performance, which the test itself already
51+
confirmed was correct (concurrent, well under the serial floor) even in
52+
the failing run.
53+
54+
## Follow-ups
55+
56+
None.

pkg/terraform/registry/provider_mirror_test.go

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,8 +190,15 @@ func TestProviderMirror_VersionResolvesPlatformsConcurrently(t *testing.T) {
190190
require.NoError(t, json.Unmarshal(body, &ver))
191191
assert.Len(t, ver.Archives, platformCount, "all platforms should resolve")
192192

193+
// Bounded at 4/5 of the serial floor (not 1/2): a concurrent run should
194+
// complete in roughly one delay plus scheduling/HTTP overhead, but CI
195+
// runners -- Windows in particular -- occasionally add several hundred ms
196+
// of that overhead under load. A tighter bound flaked at 784ms against a
197+
// 750ms threshold despite being nowhere near the 1.5s serial floor it
198+
// exists to catch; this keeps a wide, unambiguous gap from a true serial
199+
// regression while tolerating realistic CI variance.
193200
serialFloor := time.Duration(platformCount) * delay
194-
assert.Less(t, elapsed, serialFloor/2,
201+
assert.Less(t, elapsed, serialFloor*4/5,
195202
"resolving %d platforms took %s - expected well under the %s serial floor, indicating platforms are fetched concurrently, not one at a time",
196203
platformCount, elapsed, serialFloor)
197204
}

0 commit comments

Comments
 (0)