Skip to content

Commit ba69e2d

Browse files
sdsrssclaude
andcommitted
release: v0.2.5 — audit batch 6 part 2 (driver concurrency lock, skill script-location note)
The two remaining second-audit findings. Full suite ALL GREEN (MoA 30, stop-gate 26, driver-limits 21, codex-limits 22). Version bumped in all three manifest spots. - fix(drivers): concurrency guard — a portable mkdir-based atomic lock at docs/looptesting/.driver.lock (no flock; absent on macOS) with holder PID-liveness. A live holder is refused (exit 2); a crashed driver's stale lock is stolen. Identical across both drivers (loop traps release_lock; codex folds it into its skill-dir restore cleanup). DR-4. (+4 driver tests.) - docs(skill): SKILL.md states the bundled scripts/templates live in the skill's own install dir, not the target cwd; gives the resolution (${CLAUDE_PLUGIN_ROOT} / codex skills path) and an inline fallback (scripts are optional conveniences). PL-1 / PL-4. Behavior note: a second concurrent driver on the same project is now refused (exit 2) instead of racing; a stale lock from a SIGKILL'd run is auto-stolen. Verification caveat: the SKILL.md change is LLM-visible metadata — it passes `claude plugin validate` and does not regress the suite, but its behavioral effect in a live loop is NOT yet verified by a real-loop run (outstanding). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 9ce4fca commit ba69e2d

8 files changed

Lines changed: 131 additions & 10 deletions

File tree

.claude-plugin/marketplace.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,14 @@
55
},
66
"metadata": {
77
"description": "loop-testing plugin marketplace",
8-
"version": "0.2.4"
8+
"version": "0.2.5"
99
},
1010
"plugins": [
1111
{
1212
"name": "loop-testing",
1313
"source": "./",
1414
"description": "Autonomous self-test / self-fix / self-iterate QA loop that drives a finished project to convergence as a simulated real user in a sandbox (自测·验收·QA 循环).",
15-
"version": "0.2.4",
15+
"version": "0.2.5",
1616
"category": "development"
1717
}
1818
]

.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "loop-testing",
3-
"version": "0.2.4",
3+
"version": "0.2.5",
44
"description": "Autonomous QA self-test loop for a finished project: Claude acts as a real user in a sandbox, exercising every feature to surface bugs, logic/flow/UX/security/hidden issues — fixing the safe low-risk ones with regression tests and atomic commits, and recording the rest with MoA multi-model decision advice — looping self-test → self-fix → self-iterate until two consecutive low-risk rounds converge. Use when the user wants hands-off acceptance testing / 自测 / 自我测试 / 验收 / QA 循环 / self-test loop / 自动测试并修复 of their app (CLI, API, Web, or library). Runs fully autonomous, pausing only for keys/payment/network permission, a suspected security vuln to report, or a total blocker.",
55
"author": {
66
"name": "sds"

CHANGELOG.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,34 @@
11
# Changelog
22

3+
## 0.2.5 — 2026-07-12
4+
5+
Audit batch 6 (part 2): the two remaining second-audit findings. Full suite
6+
`ALL GREEN` (MoA 30, stop-gate 26, driver-limits 21, codex-limits 22).
7+
8+
**Behavior change to note:** running a second unattended driver on a project while
9+
one is already running is now refused (exit 2) instead of racing STATE.md / the
10+
ledger / the worktree. A crashed driver's leftover lock (holder PID no longer alive)
11+
is auto-stolen, so a normal relaunch after a crash is unaffected. Remove
12+
`docs/looptesting/.driver.lock` by hand only if a run was SIGKILL'd and you are sure
13+
no driver is live.
14+
15+
- **fix(drivers)**: added a concurrency guard — a portable `mkdir`-based atomic lock
16+
at `docs/looptesting/.driver.lock` (no `flock`; it is absent on macOS) with holder
17+
PID-liveness: a live holder is refused (exit 2), a stale lock from a crashed driver
18+
is stolen. Kept identical across both drivers (loop driver traps `release_lock`;
19+
the codex driver folds it into its existing skill-dir-restore cleanup trap). DR-4.
20+
(+4 driver tests: live-holder refusal and stale-lock steal, per driver.)
21+
- **docs(skill)**: SKILL.md now states that the bundled scripts and templates live
22+
in the skill's own install dir (not the target project's cwd), so the
23+
`skills/loop-testing/…` paths in the references are relative-to-skill hints, not
24+
commands to copy verbatim under the target cwd. It gives the resolution (Claude:
25+
`${CLAUDE_PLUGIN_ROOT}/skills/loop-testing/`; Codex: `~/.codex/skills/loop-testing/`)
26+
and an inline-fallback: the scripts are optional conveniences — do the equivalent
27+
setup/clean inline if they can't be located. PL-1 / PL-4. NOTE: this is
28+
LLM-visible metadata; it passes `claude plugin validate` and does not regress the
29+
suite, but its behavioral effect (does the agent resolve/fall back correctly in a
30+
live loop) is NOT yet verified by a real-loop run — verification is outstanding.
31+
332
## 0.2.4 — 2026-07-12
433

534
Audit batch 6 (second-audit follow-up, functional/contract hardening). Lands the

skills/loop-testing/SKILL.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ description: Autonomous QA self-test / self-fix / self-iterate loop. Use after a
2525

2626
## 工作产物:docs/looptesting/
2727

28-
启动即建(或复用 `scripts/sandbox-setup.sh`)。八个固定产物:
28+
启动即建(或复用本技能的 `sandbox-setup.sh`,定位见下方「脚本与模板定位」)。八个固定产物:
2929

3030
| 文件 | 作用 |
3131
|------|------|
@@ -40,6 +40,8 @@ description: Autonomous QA self-test / self-fix / self-iterate loop. Use after a
4040

4141
**续跑协议**:启动时若 `STATE.md` 已存在 → 通读全部状态文件,从「下一动作」继续,**禁止重置轮数或清空总账**。模板在本技能 `templates/`
4242

43+
> **脚本与模板定位(重要)**:`sandbox-setup.sh` / `sandbox-clean.sh` / `moa.mjs` 及 `templates/` 位于**本技能自身的安装目录**,**不在目标项目里**。运行时 cwd 是目标项目,因此 references 里写成 `skills/loop-testing/scripts/…` 的路径**只是相对本技能目录的示意,不能在目标项目 cwd 下照抄执行**。按安装位置用绝对路径调用:Claude Code 插件的根一般经 `${CLAUDE_PLUGIN_ROOT}` 暴露(本插件的 hooks 即用此变量),脚本在 `${CLAUDE_PLUGIN_ROOT}/skills/loop-testing/`;Codex 在 `~/.codex/skills/loop-testing/`。若该变量在当前会话不可用或不确定,走下方兜底。**若一时无法定位这些脚本**:它们都是**可选便利工具**——直接内联完成等价动作即可,不阻塞循环:建 `docs/looptesting/` 目录与八个产物(模板照 `templates/` 结构手写)、用 `git worktree` 或 `qa/loop-testing` 分支隔离、清理时停掉自己启动的进程并删自建 worktree。MoA 不可用时按 `references/moa-decision.md` 降级单模型。
44+
4345
## 循环骨架(按需加载 references/,渐进披露以省上下文)
4446

4547
1. **第 0 轮**(只做一次):续跑检测 → 产品形态与入口发现 → 全功能交叉盘点 → 场景设计 → 基线检查 → 建沙箱 → 产出 `PLAN.md` + `FEATURE_MATRIX.md`。细则读 `references/round-0.md`

skills/loop-testing/scripts/unattended-codex.sh

Lines changed: 31 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -71,13 +71,38 @@ DLOG="$LT/driver.log"
7171
mkdir -p "$LT"
7272
: >> "$DLOG" || die "cannot write driver.log at $DLOG"
7373

74-
# Protect the installed skill from the full-access session; always restore.
75-
# Set the restore trap BEFORE the chmod so a signal in between can't leave the
76-
# dir read-only. Trap EXIT + INT + TERM explicitly (don't rely on bash's implicit
77-
# EXIT-on-signal, which is version/platform-dependent); only SIGKILL, which skips
78-
# traps, can leave it read-only — unavoidable.
74+
# Concurrency guard: refuse to run a second driver on the same target — two drivers
75+
# would race STATE.md / driver.log / ISSUES.md / the worktree and corrupt the
76+
# progress fingerprint and ledger. Portable atomic lock via mkdir (no flock — it is
77+
# absent on macOS). A crashed driver's lock (holder PID no longer alive) is stolen; a
78+
# live holder is refused (audit DR-4). Kept identical to unattended-loop.sh.
79+
LOCK_DIR="$LT/.driver.lock"
80+
LOCK_OWNED=0
81+
release_lock() { [ "$LOCK_OWNED" = 1 ] && rm -rf "$LOCK_DIR" 2>/dev/null; LOCK_OWNED=0; }
82+
acquire_lock() {
83+
if mkdir "$LOCK_DIR" 2>/dev/null; then echo "$$" > "$LOCK_DIR/pid"; LOCK_OWNED=1; return 0; fi
84+
local holder=""; [ -f "$LOCK_DIR/pid" ] && read -r holder < "$LOCK_DIR/pid" 2>/dev/null
85+
case "$holder" in ''|*[!0-9]*) holder="" ;; esac
86+
if [ -n "$holder" ] && kill -0 "$holder" 2>/dev/null; then
87+
die "another loop-testing driver (pid $holder) is running on this project; refusing to run concurrently (remove $LOCK_DIR if it is stale)"
88+
fi
89+
rm -rf "$LOCK_DIR" 2>/dev/null # stale lock from a crashed driver — steal it
90+
if mkdir "$LOCK_DIR" 2>/dev/null; then echo "$$" > "$LOCK_DIR/pid"; LOCK_OWNED=1; return 0; fi
91+
die "could not acquire driver lock at $LOCK_DIR"
92+
}
93+
94+
# Protect the installed skill from the full-access session; always restore. Set the
95+
# combined cleanup trap (skill-dir restore + lock release) BEFORE the chmod so a
96+
# signal in between can't leave the dir read-only. Trap EXIT + INT + TERM explicitly
97+
# (don't rely on bash's implicit EXIT-on-signal, which is version/platform-dependent);
98+
# only SIGKILL, which skips traps, can leave it read-only — unavoidable.
99+
cleanup() {
100+
[ "$PROTECT" = "1" ] && [ -d "$SKILL_DIR" ] && chmod -R u+w "$SKILL_DIR" 2>/dev/null
101+
release_lock
102+
}
103+
trap cleanup EXIT INT TERM
104+
acquire_lock
79105
if [ "$PROTECT" = "1" ] && [ -d "$SKILL_DIR" ]; then
80-
trap 'chmod -R u+w "$SKILL_DIR" 2>/dev/null || true' EXIT INT TERM
81106
chmod -R a-w "$SKILL_DIR" 2>/dev/null || true
82107
fi
83108

skills/loop-testing/scripts/unattended-loop.sh

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,28 @@ progress_sig() { # composite fingerprint: round|issues|streak|runsN:runsB|bootst
106106
mkdir -p "$LT"
107107
: >> "$DRIVER_LOG" || die "cannot write driver.log at $DRIVER_LOG"
108108

109+
# Concurrency guard: refuse to run a second driver on the same target — two drivers
110+
# would race STATE.md / driver.log / ISSUES.md / the worktree and corrupt the
111+
# progress fingerprint and ledger. Portable atomic lock via mkdir (no flock — it is
112+
# absent on macOS). A crashed driver's lock (holder PID no longer alive) is stolen; a
113+
# live holder is refused (audit DR-4). Kept identical to unattended-codex.sh.
114+
LOCK_DIR="$LT/.driver.lock"
115+
LOCK_OWNED=0
116+
release_lock() { [ "$LOCK_OWNED" = 1 ] && rm -rf "$LOCK_DIR" 2>/dev/null; LOCK_OWNED=0; }
117+
acquire_lock() {
118+
if mkdir "$LOCK_DIR" 2>/dev/null; then echo "$$" > "$LOCK_DIR/pid"; LOCK_OWNED=1; return 0; fi
119+
local holder=""; [ -f "$LOCK_DIR/pid" ] && read -r holder < "$LOCK_DIR/pid" 2>/dev/null
120+
case "$holder" in ''|*[!0-9]*) holder="" ;; esac
121+
if [ -n "$holder" ] && kill -0 "$holder" 2>/dev/null; then
122+
die "another loop-testing driver (pid $holder) is running on this project; refusing to run concurrently (remove $LOCK_DIR if it is stale)"
123+
fi
124+
rm -rf "$LOCK_DIR" 2>/dev/null # stale lock from a crashed driver — steal it
125+
if mkdir "$LOCK_DIR" 2>/dev/null; then echo "$$" > "$LOCK_DIR/pid"; LOCK_OWNED=1; return 0; fi
126+
die "could not acquire driver lock at $LOCK_DIR"
127+
}
128+
trap release_lock EXIT INT TERM
129+
acquire_lock
130+
109131
START_EPOCH=$(date +%s)
110132
DEADLINE=$(( START_EPOCH + MAX_MINUTES * 60 ))
111133

tests/driver/codex-limits.test.sh

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,4 +96,24 @@ bash "$CODEX_DRIVER" --project "$WS9" --codex-bin "$stub" --no-protect --max-ses
9696
assert_rc $? 2 "unwritable driver.log -> die exit 2"
9797
assert_eq "0" "$(sessions_in_log "$WS9")" "no session launched when driver.log is unwritable"
9898

99+
# L. Concurrency guard: a second codex driver is refused while a LIVE holder holds
100+
# the lock -> exit 2 (audit DR-4; mirrors loop driver J). Live holder = $$.
101+
WS10=$(mk_proj); trap 'rm -rf "$WS" "$WS2" "$WS3" "$WS4" "$WS5" "$WS6" "$WS7" "$WS8" "$WS9" "$WS10"' EXIT
102+
mkdir -p "$WS10/docs/looptesting/.driver.lock"
103+
echo "$$" > "$WS10/docs/looptesting/.driver.lock/pid"
104+
stub=$(write_stub "$WS10"); write_state "$WS10" RUNNING 0
105+
bash "$CODEX_DRIVER" --project "$WS10" --codex-bin "$stub" --no-protect --max-sessions 1 >/dev/null 2>&1
106+
assert_rc $? 2 "live driver holds the lock -> concurrent codex run refused (exit 2)"
107+
assert_eq "0" "$(sessions_in_log "$WS10")" "no session launched while the lock is held"
108+
109+
# M. Stale lock (dead holder) stolen; run proceeds and releases the lock on exit
110+
# (audit DR-4; mirrors loop driver K). Dead holder = a just-exited child PID.
111+
WS11=$(mk_proj); trap 'rm -rf "$WS" "$WS2" "$WS3" "$WS4" "$WS5" "$WS6" "$WS7" "$WS8" "$WS9" "$WS10" "$WS11"' EXIT
112+
mkdir -p "$WS11/docs/looptesting/.driver.lock"
113+
echo "$(bash -c 'echo $$')" > "$WS11/docs/looptesting/.driver.lock/pid"
114+
stub=$(write_stub "$WS11"); write_state "$WS11" RUNNING 0
115+
STUB_CONVERGE_AT=1 bash "$CODEX_DRIVER" --project "$WS11" --codex-bin "$stub" --no-protect --max-sessions 3 >/dev/null 2>&1
116+
assert_rc $? 0 "stale lock stolen -> run proceeds to convergence (exit 0)"
117+
if [ -e "$WS11/docs/looptesting/.driver.lock" ]; then FAIL=$((FAIL+1)); echo " FAIL: lock not released on normal exit" >&2; else PASS=$((PASS+1)); fi
118+
99119
report "codex-limits.test.sh"

tests/driver/driver-limits.test.sh

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,4 +93,27 @@ CLAUDE_CODE_CHILD_SESSION=1 CLAUDE_CODE_SESSION_ID=sess-abc \
9393
assert_rc $? 0 "converged after a sanitized session -> exit 0"
9494
assert_eq "" "$(cat "$WS7/docs/looptesting/child-env.txt" 2>/dev/null)" "coordinator-mode env vars unset for the child (F6)"
9595

96+
# J. Concurrency guard: a second driver is refused while a LIVE holder holds the lock
97+
# (two drivers would race STATE/ledger/worktree) -> exit 2 (audit DR-4). The live
98+
# holder is THIS test process ($$), so no background fixture is needed.
99+
WS8=$(mk_proj); trap 'rm -rf "$WS" "$WS2" "$WS3" "$WS4" "$WS5" "$WS6" "$WS7" "$WS8"' EXIT
100+
mkdir -p "$WS8/docs/looptesting/.driver.lock"
101+
echo "$$" > "$WS8/docs/looptesting/.driver.lock/pid"
102+
stub=$(write_stub "$WS8"); write_state "$WS8" RUNNING 0
103+
bash "$DRIVER" --project "$WS8" --claude-bin "$stub" --max-sessions 1 >/dev/null 2>&1
104+
assert_rc $? 2 "live driver holds the lock -> concurrent run refused (exit 2)"
105+
assert_eq "0" "$(sessions_in_log "$WS8")" "no session launched while the lock is held"
106+
107+
# K. A stale lock (holder PID no longer alive) is stolen; the run proceeds and the
108+
# lock is released on normal exit (audit DR-4). The dead PID is a just-exited
109+
# child ($(bash -c 'echo $$')) — no background job (which would inherit the EXIT
110+
# trap and delete the workspace mid-test).
111+
WS9=$(mk_proj); trap 'rm -rf "$WS" "$WS2" "$WS3" "$WS4" "$WS5" "$WS6" "$WS7" "$WS8" "$WS9"' EXIT
112+
mkdir -p "$WS9/docs/looptesting/.driver.lock"
113+
echo "$(bash -c 'echo $$')" > "$WS9/docs/looptesting/.driver.lock/pid"
114+
stub=$(write_stub "$WS9"); write_state "$WS9" RUNNING 0
115+
STUB_CONVERGE_AT=1 bash "$DRIVER" --project "$WS9" --claude-bin "$stub" --max-sessions 3 >/dev/null 2>&1
116+
assert_rc $? 0 "stale lock (dead holder) stolen -> run proceeds to convergence (exit 0)"
117+
if [ -e "$WS9/docs/looptesting/.driver.lock" ]; then FAIL=$((FAIL+1)); echo " FAIL: lock not released on normal exit" >&2; else PASS=$((PASS+1)); fi
118+
96119
report "driver-limits.test.sh"

0 commit comments

Comments
 (0)