Skip to content

feat(mistral-ai): update model YAMLs [bot]#1677

Open
models-bot[bot] wants to merge 1 commit into
mainfrom
bot/update-mistral-ai-20260702-111706
Open

feat(mistral-ai): update model YAMLs [bot]#1677
models-bot[bot] wants to merge 1 commit into
mainfrom
bot/update-mistral-ai-20260702-111706

Conversation

@models-bot

@models-bot models-bot Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Auto-generated by poc-agent for provider mistral-ai.


Note

Medium Risk
OCR cost increases affect billing estimates; deprecation flags change how moderation and mistral-small are surfaced to users.

Overview
Updates five mistral-ai provider model YAMLs to match current docs and pricing.

ministral-14b-2512 gains assistant_prefill and doc as an input modality. mistral-vibe-cli-with-tools adds json_output, sets thinking: true, and links an extra CLI docs source.

mistral-moderation-latest is marked deprecated via isDeprecated: true and status: deprecated (deprecation date unchanged). mistral-small only adds isDeprecated: true in this diff.

mistral-ocr-latest per-page costs rise from $0.002 / $0.003 to $0.004 / $0.005 (page / annotated page), with mistral.ai/pricing/api added as a source.

Reviewed by Cursor Bugbot for commit 92f57e3. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

/test-models

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 92f57e3. Configure here.

- assistant_prefill
- json_output
- system_messages
isDeprecated: true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conflicting deprecation lifecycle fields

Medium Severity

This change sets isDeprecated: true on mistral-small while status remains active. The same PR updates mistral-moderation-latest with both isDeprecated: true and status: deprecated, and other deprecated Mistral entries follow that pairing. Downstream consumers that read only one field can misclassify the model’s lifecycle.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 92f57e3. Configure here.

@harshiv-26

Copy link
Copy Markdown
Collaborator

Gateway test results

  • Total: 21
  • Passed: 16
  • Failed: 0
  • Validation failed: 2
  • Errored: 0
  • Skipped: 3
  • Success rate: 88.89%
Provider Model Scenarios
mistral-ai ministral-14b-2512 success: tool-call, tool-call:stream, params, json-output, params:stream, json-output:stream

validation_failure: structured-output:stream, structured-output
mistral-ai mistral-moderation-latest skipped: skip-check
mistral-ai mistral-ocr-latest skipped: skip-check
mistral-ai mistral-small skipped: skip-check
mistral-ai mistral-vibe-cli-with-tools success: params:stream, tool-call, params, structured-output, tool-call:stream, structured-output:stream, json-output:stream, json-output, reasoning, reasoning:stream
Failures (2)

mistral-ai/ministral-14b-2512 — structured-output:stream (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmp1vgl70sg/snippet.py", line 48, in <module>
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-mistral-ai/ministral-14b-2512",
    messages=[
        {"role": "system", "content": "Extract the event information as JSON."},
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "Hi, how can I help you"},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=True,
)
import json as _json

_accumulated = ""
for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta
        if delta.content is not None:
            _accumulated += delta.content
            print(delta.content, end="", flush=True)

if not _accumulated:
    raise Exception("VALIDATION FAILED: structured-output stream - no content received")

_parsed = _json.loads(_accumulated)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output stream - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output stream - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output stream - unexpected keys present: {set(_parsed.keys())}"
    )

print("\nVALIDATION: structured-output stream SUCCESS")
Output
{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "date": {
      "type": "string"
    },
    "participants": {
      "type": "array",
      "items": {
        "type": "string"
      }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}

mistral-ai/ministral-14b-2512 — structured-output (validation_failure)

Error
Traceback (most recent call last):
  File "/tmp/tmp1j_v9_nb/snippet.py", line 43, in <module>
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")
Exception: VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)
Code snippet
from openai import OpenAI
import json

client = OpenAI(api_key="***", base_url="https://internal.devtest.truefoundry.tech/api/llm")

response_schema = json.loads('''{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "date": { "type": "string" },
    "participants": {
      "type": "array",
      "items": { "type": "string" }
    }
  },
  "required": ["name", "date", "participants"],
  "additionalProperties": false
}''')

response = client.chat.completions.create(
    model="test-v2-mistral-ai/ministral-14b-2512",
    messages=[
        {"role": "system", "content": "Extract the event information as JSON."},
        {"role": "user", "content": "Hi"},
        {"role": "assistant", "content": "Hi, how can I help you"},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday. Extract the event details as JSON."},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "CalendarEvent", "schema": response_schema}},
    stream=False,
)
import json as _json

_content = response.choices[0].message.content
print(_content)

if not _content:
    raise Exception("VALIDATION FAILED: structured-output - response content is empty")

_parsed = _json.loads(_content)

if "name" not in _parsed or "date" not in _parsed or "participants" not in _parsed:
    raise Exception("VALIDATION FAILED: structured-output - missing expected fields (name, date, participants)")

if not isinstance(_parsed.get("participants"), list):
    raise Exception("VALIDATION FAILED: structured-output - 'participants' is not a list, schema not enforced")

if set(_parsed.keys()) != {"name", "date", "participants"}:
    raise Exception(
        f"VALIDATION FAILED: structured-output - unexpected keys present: {set(_parsed.keys())}"
    )

print("VALIDATION: structured-output SUCCESS")
Output
{
  "title": "CalendarEvent",
  "type": "object",
  "properties": {},
  "required": [],
  "additionalProperties": false
}
Successes (16)

mistral-ai/ministral-14b-2512 — tool-call (success)

Output
Function: get_weather
Arguments: {"location": "London"}
VALIDATION: tool-call SUCCESS

mistral-ai/ministral-14b-2512 — tool-call:stream (success)

Output
{"location": "London"}
VALIDATION: tool-call stream SUCCESS

mistral-ai/ministral-14b-2512 — params (success)

Output
The capital of France is **Paris**! 🇫🇷

Would you like to know some fun facts about it? 😊

mistral-ai/ministral-14b-2512 — json-output (success)

Output
{
  "colors": [
    {
      "name": "Deep Sky Blue",
      "hex_code": "#00BFFF"
    },
    {
      "name": "Forest Green",
      "hex_code": "#228B22
... (truncated, 113 chars omitted)

mistral-ai/ministral-14b-2512 — params:stream (success)

Output
The capital of France is **Paris**! 🇫🇷

Would you like to know more about it? 😊

mistral-ai/ministral-14b-2512 — json-output:stream (success)

Output
{
  "colors": [
    {
      "name": "Red",
      "hex_code": "#FF0000"
    },
    {
      "name": "Blue",
      "hex_code": "#0000FF"
    },
    {
   
... (truncated, 99 chars omitted)

mistral-ai/mistral-vibe-cli-with-tools — params:stream (success)

Output
The capital of France is **Paris**.

mistral-ai/mistral-vibe-cli-with-tools — tool-call (success)

Output
Function: get_weather
Arguments: {"location": "London"}
VALIDATION: tool-call SUCCESS

mistral-ai/mistral-vibe-cli-with-tools — params (success)

Output
The capital of France is **Paris**.

mistral-ai/mistral-vibe-cli-with-tools — structured-output (success)

Output
{"name": "Science Fair", "date": "Friday", "participants": ["Alice", "Bob"]}
VALIDATION: structured-output SUCCESS

mistral-ai/mistral-vibe-cli-with-tools — tool-call:stream (success)

Output
{"location": "London"}
VALIDATION: tool-call stream SUCCESS

mistral-ai/mistral-vibe-cli-with-tools — structured-output:stream (success)

Output
{"name": "Science Fair", "date": "Friday", "participants": ["Alice", "Bob"]}
VALIDATION: structured-output stream SUCCESS

mistral-ai/mistral-vibe-cli-with-tools — json-output:stream (success)

Output
{"colors": [{"name": "Red", "hex": "#FF0000"}, {"name": "Green", "hex": "#00FF00"}, {"name": "Blue", "hex": "#0000FF"}]}
VALIDATION: json-output strea
... (truncated, 10 chars omitted)

mistral-ai/mistral-vibe-cli-with-tools — json-output (success)

Output
{"colors": [{"name": "Red", "hex": "#FF0000"}, {"name": "Green", "hex": "#00FF00"}, {"name": "Blue", "hex": "#0000FF"}]}
VALIDATION: json-output SUCCE
... (truncated, 3 chars omitted)

mistral-ai/mistral-vibe-cli-with-tools — reasoning (success)

Output
To calculate \( 3^{3^{3^3}} \), we must follow the **right-associative** rule of exponentiation (i.e., evaluate exponents from **top to bottom** or **
... (truncated, 1655 chars omitted)

mistral-ai/mistral-vibe-cli-with-tools — reasoning:stream (success)

Output
To calculate \( 3^{3^{3^3}} \), we must follow the **right-associative** rule of exponentiation, meaning we evaluate from the **top down** (or right t
... (truncated, 2574 chars omitted)
Skipped (3)

mistral-ai/mistral-moderation-latest — skip-check (skipped)

Skip reason
unsupported mode 'moderation'

mistral-ai/mistral-ocr-latest — skip-check (skipped)

Skip reason
unsupported mode 'unknown'

mistral-ai/mistral-small — skip-check (skipped)

Skip reason
deprecated or retired model

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant