Skip to content

fix: sanitize JSON responses in automatic chat renaming#1751

Open
gdeyoung wants to merge 1 commit into
agent0ai:mainfrom
gdeyoung:fix/rename-json-sanitization
Open

fix: sanitize JSON responses in automatic chat renaming#1751
gdeyoung wants to merge 1 commit into
agent0ai:mainfrom
gdeyoung:fix/rename-json-sanitization

Conversation

@gdeyoung

@gdeyoung gdeyoung commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Fix: Sanitize JSON responses in automatic chat renaming

Problem

After the v2.2 refactor, automatic chat renaming breaks when using a small utility model (e.g., Gemma 4B, Qwen 8B). The utility model sometimes echoes a structured JSON response instead of a plain-text chat name, and that raw JSON gets stored directly as context.name.

Example broken chat names

{"thoughts": ["The user is complaining about broken chat names"], "headline": "..."}
{"thought": "The user is initiating a detailed investigation into..."}

Root cause

The v2.2 refactor changed history.output_text() to serialize AI messages as raw JSON (e.g., {"thoughts": [...], "headline": "...", "tool_name": "..."}). When this JSON-rich history is fed to a small utility model for renaming, the model pattern-matches the JSON format in its context window instead of following the system prompt instruction to "respond with only the chat name."

The existing fallback in commit 0b38258d tried to extract the first user message, but this is unreliable because history.output() structure varies and the fallback silently fails.

Solution

Add a 3-tier sanitization strategy that activates when the utility model response starts with { or [:

  1. Parse JSON and extract headline — gives the highest-quality name since the model already generated a summary headline
  2. Fall back to first user message — extract topic from the first non-AI message in history
  3. Last resort — default to "Conversation"

This is strictly better than the existing approach because:

  • JSON headline extraction produces clean, descriptive names (e.g., "Checking configuration" instead of the raw JSON)
  • The fallback chain is more robust than the single-strategy approach
  • Zero behavioral change when the model returns correct plain text

Testing

  1. Set utility model to a small model (Gemma 4B via Ollama)
  2. Start a new chat and send several messages
  3. Verify the auto-generated name is clean text, not JSON
  4. Verify manually-named chats are not overwritten (manual lock still works)
  5. Test with a large utility model (should behave identically — JSON detection never triggers)

Changes

File: extensions/python/monologue_start/_60_rename_chat.py

Before (vanilla v2.2):

if new_name:
    new_name = " ".join(str(new_name).split())
    if len(new_name) > 40:
        new_name = new_name[:40] + "..."
    if not new_name:
        return
    self.agent.context.name = new_name

After:

if new_name:
    new_name = " ".join(str(new_name).split())
    # Guard against JSON-looking names (utility model echoes)
    stripped = new_name.lstrip()
    if stripped and stripped[0] in "{[":
        import json as _json
        extracted = None
        # Step 1: Try to parse JSON and extract headline (best quality)
        try:
            parsed = _json.loads(stripped)
            headline = parsed.get("headline")
            if isinstance(headline, str) and headline.strip():
                extracted = headline.strip()
            else:
                thoughts = parsed.get("thoughts")
                if isinstance(thoughts, list) and thoughts:
                    first = thoughts[0]
                    if isinstance(first, str) and first.strip():
                        extracted = first.strip()
        except (_json.JSONDecodeError, ValueError, AttributeError):
            pass
        # Step 2: Fall back to first user message topic
        if not extracted:
            try:
                msgs = self.agent.history.output()
                for msg in msgs:
                    if not msg.ai and msg.content:
                        content = str(msg.content)
                        first_line = content.split(chr(10))[0].split(".")[0].strip()
                        if first_line:
                            extracted = first_line[:40]
                            if len(first_line) > 40:
                                extracted += "..."
                            break
            except Exception:
                pass
        # Step 3: Last resort
        new_name = extracted or "Conversation"
    if len(new_name) > 40:
        new_name = new_name[:40] + "..."
    if not new_name:
        return
    self.agent.context.name = new_name

When utility model returns JSON instead of plain text (common with small
models like Gemma 4B), extract headline field, fall back to first user
message, then default to Conversation.

Addresses symptom of agent0ai#1750
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants