Skip to content

feat(openrouter): update model YAMLs [bot]#1688

Merged
architkumar-truefoundry merged 1 commit into
mainfrom
bot/update-openrouter-20260703-081717
Jul 7, 2026
Merged

feat(openrouter): update model YAMLs [bot]#1688
architkumar-truefoundry merged 1 commit into
mainfrom
bot/update-openrouter-20260703-081717

Conversation

@models-bot

@models-bot models-bot Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Auto-generated by poc-agent for provider openrouter.


Note

Low Risk
Metadata-only YAML updates with no runtime or pricing logic changes.

Overview
Sets status: active on the OpenRouter model definitions for qwen/qwen3.7-max and qwen/qwen3.7-plus, aligning them with other provider YAMLs that expose availability via the status field (e.g. active vs deprecated).

Reviewed by Cursor Bugbot for commit 660345e. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

/test-models

@harshiv-26

Copy link
Copy Markdown
Collaborator

Gateway test results

  • Total: 20
  • Passed: 16
  • Failed: 0
  • Validation failed: 4
  • Errored: 0
  • Skipped: 0
  • Success rate: 80.0%
Provider Model Scenarios
openrouter qwen/qwen3.7-max success: tool-call:text-text:stream, params:text-text:stream, tool-call:text-text, params:text-text, json-output:text-text, json-output:text-text:stream, reasoning:text-text:stream, reasoning:text-text

validation_failure: structured-output:text-text:stream, structured-output:text-text
openrouter qwen/qwen3.7-plus success: tool-call:text-text:stream, tool-call:text-text, params:text-text:stream, params:text-text, json-output:text-text, json-output:text-text:stream, reasoning:text-text, reasoning:text-text:stream

validation_failure: structured-output:text-text, structured-output:text-text:stream
Failures (4)

openrouter/qwen/qwen3.7-max — structured-output:text-text:stream (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmp3ak7isln/snippet.py", line 45, in <module>
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-openrouter/qwen-qwen3.7-max",
    messages=[
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=True,
)
import json as _json

_accumulated = ""
for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta
        if delta.content is not None:
            _accumulated += delta.content
            print(delta.content, end="", flush=True)

if not _accumulated:
    raise Exception("VALIDATION FAILED: structured-output stream - no content received")

_parsed = _json.loads(_accumulated)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output stream - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output stream - unexpected keys present: {set(_parsed.keys())}"
    )

print("\nVALIDATION: structured-output stream SUCCESS")
Output
{"event": "Science Fair", "day": "Friday", "participants": ["Alice", "Bob"]}

openrouter/qwen/qwen3.7-max — structured-output:text-text (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmpgjxxo4jy/snippet.py", line 40, in <module>
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-openrouter/qwen-qwen3.7-max",
    messages=[
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=False,
)
import json as _json

_content = response.choices[0].message.content
print(_content)

if not _content:
    raise Exception("VALIDATION FAILED: structured-output - response content is empty")

_parsed = _json.loads(_content)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output - unexpected keys present: {set(_parsed.keys())}"
    )

print("VALIDATION: structured-output SUCCESS")
Output
{"think": "The user wants me to extract event details from the given sentence and present them in JSON format. Let me identify the key details:\n\n1. Attendees: Alice and Bob\n2. Event: Science fair\n3. Date: Friday\n\nI'll structure this as a clean JSON object with relevant keys."}

openrouter/qwen/qwen3.7-plus — structured-output:text-text (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmp0mhhvxkm/snippet.py", line 40, in <module>
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-openrouter/qwen-qwen3.7-plus",
    messages=[
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=False,
)
import json as _json

_content = response.choices[0].message.content
print(_content)

if not _content:
    raise Exception("VALIDATION FAILED: structured-output - response content is empty")

_parsed = _json.loads(_content)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output - unexpected keys present: {set(_parsed.keys())}"
    )

print("VALIDATION: structured-output SUCCESS")
Output
{
  "people": [
    "Alice",
    "Bob"
  ],
  "event": "science fair",
  "day": "Friday"
}

openrouter/qwen/qwen3.7-plus — structured-output:text-text:stream (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmp9_l057hy/snippet.py", line 45, in <module>
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-openrouter/qwen-qwen3.7-plus",
    messages=[
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=True,
)
import json as _json

_accumulated = ""
for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta
        if delta.content is not None:
            _accumulated += delta.content
            print(delta.content, end="", flush=True)

if not _accumulated:
    raise Exception("VALIDATION FAILED: structured-output stream - no content received")

_parsed = _json.loads(_accumulated)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output stream - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output stream - unexpected keys present: {set(_parsed.keys())}"
    )

print("\nVALIDATION: structured-output stream SUCCESS")
Output
{"query": "Alice and Bob are going to a science fair on Friday."}
Successes (16)

openrouter/qwen/qwen3.7-max — tool-call:text-text:stream (success)

Output
{"location": "London"}
</think>

{"location": "London"}
VALIDATION: tool-call stream SUCCESS

openrouter/qwen/qwen3.7-max — params:text-text:stream (success)

Output
The capital of France is Paris.

openrouter/qwen/qwen3.7-max — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location": "London"}
VALIDATION: tool-call SUCCESS

openrouter/qwen/qwen3.7-max — params:text-text (success)

Output
The capital of France is Paris.

openrouter/qwen/qwen3.7-max — json-output:text-text (success)

Output
{
  "colors": [
    {"name": "Red", "hex": "#FF0000"},
    {"name": "Green", "hex": "#00FF00"},
    {"name": "Blue", "hex": "#0000FF"}
  ]
}
VALIDATIO
... (truncated, 23 chars omitted)

openrouter/qwen/qwen3.7-max — json-output:text-text:stream (success)

Output
{
  "colors": [
    {
      "name": "Cerulean Blue",
      "hex": "#007BA7"
    },
    {
      "name": "Emerald Green",
      "hex": "#50C878"
    },

... (truncated, 109 chars omitted)

openrouter/qwen/qwen3.7-max — reasoning:text-text:stream (success)

Output
To calculate the expression **$3^{3^{3^3}}$**, we must first understand the fundamental rule of mathematical notation regarding "power towers" (stacke
... (truncated, 2576 chars omitted)

openrouter/qwen/qwen3.7-max — reasoning:text-text (success)

Output
# How to Calculate $3^{3^{3^3}}$

## Step 1: Understand the Order of Operations for Exponent Towers

The most critical rule for stacked exponents is t
... (truncated, 1774 chars omitted)

openrouter/qwen/qwen3.7-plus — tool-call:text-text:stream (success)

Output
{"location": "London"}
VALIDATION: tool-call stream SUCCESS

openrouter/qwen/qwen3.7-plus — tool-call:text-text (success)

Output
Function: get_weather
Arguments: {"location": "London"}
VALIDATION: tool-call SUCCESS

openrouter/qwen/qwen3.7-plus — params:text-text:stream (success)

Output
The capital of France is Paris.

openrouter/qwen/qwen3.7-plus — params:text-text (success)

Output
The capital of France is Paris.

openrouter/qwen/qwen3.7-plus — json-output:text-text (success)

Output
{
  "colors": [
    {
      "name": "Crimson",
      "hex": "#DC143C"
    },
    {
      "name": "Dodger Blue",
      "hex": "#1E90FF"
    },
    {
  
... (truncated, 95 chars omitted)

openrouter/qwen/qwen3.7-plus — json-output:text-text:stream (success)

Output
[
  {
    "name": "Red",
    "hex": "#FF0000"
  },
  {
    "name": "Green",
    "hex": "#00FF00"
  },
  {
    "name": "Blue",
    "hex": "#0000FF"
  }
... (truncated, 42 chars omitted)

openrouter/qwen/qwen3.7-plus — reasoning:text-text (success)

Output
# Calculating $3^{3^{3^3}}$

## Step 1: Understand the Order of Operations

Exponent towers are evaluated **right-to-left** (top-down). This is called
... (truncated, 1777 chars omitted)

openrouter/qwen/qwen3.7-plus — reasoning:text-text:stream (success)

Output
# Calculating $3^{3^{3^3}}$

## Step 1: Understand the Order of Operations

Exponentiation towers are evaluated **top-down** (right-associative). So:

... (truncated, 1824 chars omitted)

@architkumar-truefoundry architkumar-truefoundry merged commit 7a7ede6 into main Jul 7, 2026
6 checks passed
@architkumar-truefoundry architkumar-truefoundry deleted the bot/update-openrouter-20260703-081717 branch July 7, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants