Add agent integrity canary layer#11
Conversation
✅ Shadow AI ScanNo Shadow AI findings detected at or above medium risk. |
CRUMB Bench Results
Powered by crumb-format — two-stage context compression |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd80e0abaa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if not result.failures: | ||
| print("- All checks passed.") | ||
|
|
||
| sys.exit(0 if result.recommendation in ("proceed", "ask_user") else 1) |
There was a problem hiding this comment.
The CLI currently exits with code 0 for both proceed and ask_user, so automation that gates execution on process success (for example, if ...; then run_tool) will continue even when the integrity layer explicitly requested human confirmation. That bypasses the safety intent of ask_user and can allow risky actions to run unattended.
Useful? React with 👍 / 👎.
| says_drive = re.search(r"\bdrive|bring|take\s+the\s+car|take\s+it\b", model_output, re.I) is not None | ||
|
|
||
| passed = says_drive and not says_walk |
There was a problem hiding this comment.
The car-wash canary treats any drive/bring/take token as a positive signal, then passes when says_drive is true and says_walk is false. This incorrectly passes outputs like “Do not drive to the car wash,” because negation is ignored; in that case the canary reports success and the overall recommendation can become proceed for exactly the physical-logic error this check is meant to catch.
Useful? React with 👍 / 👎.
slavazeph-coder
force-pushed
the
feature/agent-integrity-layer
branch
from
bd80e0a to
c9a514d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad04ed4e54
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) | ||
|
|
||
| says_walk = re.search(r"\bwalk(?:ing)?\b", model_output, re.I) is not None | ||
| says_drive = re.search(r"\bdrive|bring|take\s+the\s+car|take\s+it\b", model_output, re.I) is not None |
There was a problem hiding this comment.
The says_drive regex is broad enough to treat unrelated phrasing as “car is physically involved” (for example, take it in “Take it easy and call ahead”). In car-wash context this sets says_drive=True, so the canary can pass and return proceed even when the output never says to bring/drive the car. That bypasses the exact safeguard this canary is intended to provide; use stricter word boundaries and car-specific phrases.
Useful? React with 👍 / 👎.
| ) | ||
|
|
||
| score = _clip(mean([r.score for r in results]) if results else 0.0) | ||
| failures = [r.message for r in results if not r.passed] |
There was a problem hiding this comment.
failures is built only from checks where passed is false, but _recommendation can return ask_user from warning checks that still have passed=True (like file_edit_risk with score 0.65). That yields recommendation=ask_user with failures=[], and the CLI then prints “All checks passed,” which hides why escalation happened for humans and downstream consumers relying on failures for rationale.
Useful? React with 👍 / 👎.
slavazeph-coder
force-pushed
the
feature/agent-integrity-layer
branch
from
ad04ed4 to
869a4a5
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79ade9123a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| text = f"{proposed_action}\n{model_output}" | ||
| secret_hits = [label for label, pattern in _SECRET_PATTERNS if pattern.search(text)] |
There was a problem hiding this comment.
check_memory_write_validation only searches proposed_action and model_output for secret patterns, so a memory action can be marked safe even when the secret exists in input_text and the agent plans to store the user message verbatim. In that case run_agent_integrity_check can return proceed and allow credentials to be persisted to CRUMB memory despite this check’s stated purpose to prevent secret writes.
Useful? React with 👍 / 👎.
| if key_terms and all(term in output or term in action for term in key_terms[:2]): | ||
| violations.append(forbidden[:120]) |
There was a problem hiding this comment.
The violation heuristic treats presence of forbidden keywords as a violation without checking whether the model is explicitly refusing the action, so compliant replies like “I will not delete the database” are flagged as constraint breaks. This produces false instruction_following failures and escalates recommendations (ask_user/block) in normal safety-preserving responses.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds a lightweight agent integrity layer for CRUMB-powered agent loops.
This extracts the practical engineering pattern from CL1/LLM encoder-style experiments — deterministic canaries, confidence scoring, safety checks, and drift monitoring — without adopting speculative consciousness claims.
Added
cli/agent_integrity.pyrun_agent_integrity_check(input_text, proposed_action, model_output)DriftMonitorfor long-running sessionspython -m cli.agent_integritytests/test_agent_integrity.pydocs/AGENT_INTEGRITY.mdNotes
This is intentionally isolated and dependency-free. It does not modify the existing
crumbcommand parser yet, so it can be merged safely and wired into higher-level commands in a follow-up.