fix(scripts): improve git-ai-commit diff handling

docs/dev/git-ai-commit-context/git-ai-commit-context-optimization.json

@@ -0,0 +1,149 @@
+{
+  "goal": "Implement applicable recommendations from docs/dev/git_ai_commit_context_guidelines.md to maximize commit-message quality while minimizing prompt/context bloat in scripts/git-ai-commit, per the resolved decisions in .kilo/plans/1782915491660-git-ai-commit-context-optimization.md (hunk-aware truncation, two-tier caps, raw-length escalation, binary/generated skipping, reduced history depth, prompt trims, tests, and docs).",
+  "idempotent": false,
+  "steps": [
+    {
+      "id": "step-1",
+      "description": "[coder-qwen | default free/open-weight model] Read scripts/git-ai-commit in full to capture exact current structure. Report the precise line ranges and current code for: module constants block (_KILO_MODEL_PINNED_BY_ENV ~130, _MAX_DIFF_CHARS ~134, _ESCALATION_THRESHOLD_CHARS ~141, _ESCALATION_MODEL_KILO ~142), the _SYSTEM prompt (~144), _staged_diff (~212), _recent_log (~228), _build_messages signature and body (~656-704 including the inline truncation at 671-672), and the main() escalation block (~882-925 including the sum(len(...)) computation at ~919). Do NOT edit anything in this step; produce a faithful map of anchors to be used by subsequent edit steps.",
+      "model": "coder-qwen",
+      "input": null,
+      "output": "Verified anchor map: exact current text/line ranges of constants, _SYSTEM, _staged_diff, _recent_log, _build_messages, and main() escalation logic.",
+      "gate": true,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 2,
+      "on_error": "abort",
+      "self_check": "Confirm all named anchors (_MAX_DIFF_CHARS at ~134, _build_messages at ~656, main escalation sum at ~919) are located and quoted; abort if any is missing since later edits depend on them."
+    },
+    {
+      "id": "step-2",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 1: add a hunk-aware diff-truncation helper `_truncate_diff(diff: str, max_chars: int) -> str` to scripts/git-ai-commit near _build_messages. Behavior: (a) return diff unchanged when len(diff) <= max_chars; (b) preserve the leading `git diff --cached --stat -p` diffstat block, then include whole `diff --git` file sections until the char budget is reached; (c) drop remaining file sections and append a clear marker naming counts, e.g. `\\n\\n[diff truncated: N of M files omitted to fit context budget]`; (d) if a single file section alone exceeds the budget, fall back to a line-boundary (never mid-line) cut within that section plus the marker. Split file sections on the `diff --git ` boundary. Add explanatory comments. Follow file-formatting rules: exactly one trailing newline, no trailing whitespace, ruff 99-char limit / double quotes. Do not yet wire it into _build_messages.",
+      "model": "coder-qwen",
+      "input": "step-1 anchor map",
+      "output": "New `_truncate_diff` helper added with hunk/file-boundary logic, stat-block preservation, single-oversized-file line-boundary fallback, and omitted-count marker.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "abort",
+      "self_check": "python3 -c \"from importlib.machinery import SourceFileLoader; m=SourceFileLoader('g','scripts/git-ai-commit').load_module(); assert callable(m._truncate_diff)\" — module imports and helper is callable."
+    },
+    {
+      "id": "step-3",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 2: replace the single `_MAX_DIFF_CHARS = 60_000` constant with two module constants: `_MAX_DIFF_CHARS_DEFAULT = 60_000` (Haiku/non-escalated path) and `_MAX_DIFF_CHARS_ESCALATED` set to a large ceiling matched to Qwen's practical context (effectively a no-truncation safety net; choose a value like 400_000 and justify in a comment). Add explanatory comments describing the two-tier intent. Update any internal reference to the old name. Keep ruff/formatting rules.",
+      "model": "coder-qwen",
+      "input": "step-1 anchor map",
+      "output": "Two-tier cap constants (_MAX_DIFF_CHARS_DEFAULT, _MAX_DIFF_CHARS_ESCALATED) defined with comments; old single constant removed.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "abort",
+      "self_check": "grep confirms both new constants exist and the bare `_MAX_DIFF_CHARS =` (without _DEFAULT/_ESCALATED suffix) no longer appears; module still imports."
+    },
+    {
+      "id": "step-4",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 4: add a `max_diff_chars: int = _MAX_DIFF_CHARS_DEFAULT` parameter to `_build_messages` (default preserves current behavior for direct/test callers). Replace the inline slice+marker at ~671-672 with a call to `_truncate_diff(diff, max_diff_chars)`. Ensure the rest of _build_messages (system/user message assembly at ~704) is otherwise unchanged. Keep formatting rules.",
+      "model": "coder-qwen",
+      "input": "step-2 helper; step-3 constants; step-1 anchor map",
+      "output": "_build_messages accepts max_diff_chars (defaulting to _MAX_DIFF_CHARS_DEFAULT) and delegates truncation to _truncate_diff; inline 60k slice removed.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "abort",
+      "self_check": "Inspect signature: `_build_messages` has max_diff_chars param; grep shows no remaining `diff[:_MAX_DIFF_CHARS]` inline slice; module imports."
+    },
+    {
+      "id": "step-5",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 3 + Task 4 wiring in main() (~882-925). (a) Compute the escalation decision from the RAW pre-truncation length: build the sum of untruncated diff + status + log + context + instructions lengths BEFORE _build_messages truncates, instead of the current post-truncation `sum(len(m['content']) for m in msgs)`. Preserve existing guards: backend == 'kilo', not model_explicit, not _KILO_MODEL_PINNED_BY_ENV, threshold _ESCALATION_THRESHOLD_CHARS, and keep the existing stderr escalation notice. (b) After resolving the model, select `max_diff_chars = _MAX_DIFF_CHARS_ESCALATED` when the resolved model is the escalation model (_ESCALATION_MODEL_KILO) or a user-pinned/explicit high-context model; otherwise `_MAX_DIFF_CHARS_DEFAULT`. (c) Pass that value into the `_build_messages(...)` call at ~909, reordering so message construction happens after the model/cap is resolved. Do not double-count or regress the env-pin / explicit-model suppression logic. Keep formatting rules.",
+      "model": "coder-qwen",
+      "input": "step-4 _build_messages signature; step-3 constants; step-1 anchor map",
+      "output": "main() decides escalation from raw pre-truncation length, resolves model, then passes the model-appropriate max_diff_chars into _build_messages; guards and stderr notice preserved.",
+      "gate": false,
+      "parallel": false,
+      "retry": 1,
+      "max_retries": 3,
+      "on_error": "replan",
+      "self_check": "grep confirms the escalation condition no longer relies on `sum(len(m['content']) for m in msgs)` for the decision; module imports; a manual reasoning check that guards (kilo/model_explicit/env-pin/threshold) remain intact."
+    },
+    {
+      "id": "step-6",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 5: skip binary/generated files in the diff. Add a small commented constant listing generated-file glob patterns (e.g. *.pb.cc, *.pb.h, *.gen.cpp and similar). Before prompting, filter `diff --git` sections that are either git binary sections (contain `Binary files ... differ`) or match a generated pattern, replacing each removed section with a one-line note so the model knows a file changed (e.g. `[skipped binary/generated file: <path>]`). Confirm no conflict with the `--stat` block (the stat line for skipped files may remain — that is acceptable). Integrate this filtering so it runs before/within truncation (apply filtering to the raw diff prior to _truncate_diff). Keep formatting rules.",
+      "model": "coder-qwen",
+      "input": "step-2 helper; step-4/step-5 wiring; step-1 anchor map",
+      "output": "Binary and generated-pattern diff sections are replaced with a one-line note; generated-pattern constant added; stat block left intact.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "continue",
+      "self_check": "Unit-level reasoning: a synthetic diff containing a `Binary files a/x b/x differ` section and a `*.pb.cc` section yields notes, not full hunks; module imports."
+    },
+    {
+      "id": "step-7",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 6: reduce default commit-history depth in `_recent_log` (~228) from `-10` to `-5`. Update the in-code docstring reference to `git log --oneline -10` accordingly. OPTIONAL (only if low-risk and cleanly threadable): add a `--history-depth N` argparse flag threaded into `_recent_log(depth)`, defaulting to 5, and update argparse help text and the module docstring. If the flag is omitted, add a brief comment noting it as a deliberate simplification. Do not break existing callers. Keep formatting rules.",
+      "model": "coder-qwen",
+      "input": "step-1 anchor map",
+      "output": "_recent_log default depth reduced to 5 (with docstring updated); optional --history-depth flag added only if clean, otherwise documented as deliberately omitted.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "continue",
+      "self_check": "grep confirms `_recent_log` uses -5 (or a depth variable defaulting to 5); if flag added, argparse help and docstring updated; module imports."
+    },
+    {
+      "id": "step-8",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 7: tighten the `_SYSTEM` prompt wording (~144) by removing only redundant/filler phrasing. MUST NOT weaken: Conventional Commits v1.0.0 rules, the 'respond with only the message' constraint, or subject/body/footer formatting requirements. Keep the change minimal (the prompt is already close to minimal-bloat form). Preserve textwrap.dedent structure and formatting rules.",
+      "model": "coder-qwen",
+      "input": "step-1 anchor map (_SYSTEM text)",
+      "output": "_SYSTEM prompt trimmed of filler while fully preserving Conventional Commits conformance, the message-only constraint, and formatting requirements.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "continue",
+      "self_check": "Diff review confirms only filler removed; the phrases enforcing Conventional Commits, 'only the message', and subject/body/footer rules remain present; module imports."
+    },
+    {
+      "id": "step-9",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 8: update scripts/test/test_git_ai_commit.py (loaded via SourceFileLoader). Add tests: (a) _truncate_diff under budget returns unchanged; (b) over budget cuts on file/hunk boundary and keeps the --stat header; (c) a single oversized file falls back to a line-boundary cut; (d) the omitted-count marker text is present; (e) escalation ordering: a large RAW diff triggers escalation using pre-truncation length, and the escalated path receives the larger cap (assert via _build_messages output length / absence of truncation marker when max_diff_chars is escalated); (f) binary/generated-file skipping produces the one-line note. Update any existing tests that assumed the old single 60k cap or the old inline marker string `[diff truncated — too large to show in full]`. Preserve _build_messages default-arg behavior so existing signature-based tests still pass where feasible. Keep ruff/formatting rules.",
+      "model": "coder-qwen",
+      "input": "steps 2-8 implemented source",
+      "output": "Test suite extended with truncation, escalation-ordering, and binary/generated-skip tests; obsolete-cap/marker assertions updated; default-arg compatibility preserved.",
+      "gate": false,
+      "parallel": false,
+      "retry": 1,
+      "max_retries": 3,
+      "on_error": "replan",
+      "self_check": "pytest scripts/test/ -v — all tests (existing + new) pass; no reference to the removed inline marker string remains except in intentionally updated assertions."
+    },
+    {
+      "id": "step-10",
+      "description": "[coder-qwen | default free/open-weight model] Implement Task 9: update documentation. (a) scripts/man/man1/git-ai-commit.1 — reflect new hunk-aware truncation behavior, reduced history depth (10->5), any new --history-depth flag, and binary/generated-file skipping. (b) The module docstring at the top of scripts/git-ai-commit — sync any option/default changes. (c) docs/dev/git_ai_commit_context_guidelines.md — append a short 'Implementation status' section recording the disposition table from the plan (item 1 Implement-adapted, item 2 Partial, item 3 Reject, item 4 Implement, item 5 Reject, item 6 Partial, item 7 Reject-table/tune-threshold) with done/partial/rejected + rationale. Follow Markdown lint rules (MD012/MD022/MD031/MD032/MD034/MD040) and file-formatting rules (one trailing newline, no trailing whitespace).",
+      "model": "coder-qwen",
+      "input": "steps 2-9 implemented behavior; plan disposition table",
+      "output": "Man page, module docstring, and guidelines doc updated to match new behavior; disposition/Implementation-status section appended.",
+      "gate": false,
+      "parallel": false,
+      "retry": 0,
+      "max_retries": 3,
+      "on_error": "continue",
+      "self_check": "Man page mentions truncation/history-depth/binary skip; guidelines doc has an 'Implementation status' section covering all 7 report items; markdownlint-style spot check passes."
+    },
+    {
+      "id": "step-11",
+      "description": "[coder-qwen | default free/open-weight model] Validation. Run: (1) `pytest scripts/test/ -v` — all tests must pass. (2) `ruff check scripts/git-ai-commit scripts/test/test_git_ai_commit.py` per pyproject.toml (99-char, double quotes) — fix any findings. (3) Manual dry-runs: `git ai-commit --dry-run` on a small diff (no truncation, default model, no escalation); a synthetic large diff over the escalation threshold (confirm escalation fires AND no truncation marker on the escalated path); and a diff containing a binary + a generated file (confirm both are skipped with a note). (4) `PREKCOMMAND=$(command -v prek || command -v pre-commit); $PREKCOMMAND run --all-files` and address hook findings. Report pass/fail per check with the exact commands and key output.",
+      "model": "coder-qwen",
+      "input": "all prior steps",
+      "output": "Validation report: pytest, ruff, three dry-run scenarios, and pre-commit hook results, with fixes applied for any failures.",
+      "gate": true,
+      "parallel": false,
+      "retry": 1,
+      "max_retries": 3,
+      "on_error": "replan",
+      "self_check": "All four checks pass; escalated-path dry-run shows no truncation marker; binary/generated dry-run shows skip notes; gate blocks completion until green."
+    }
+  ]
+}