nexus-mock-provider

A faithful, high-performance mock of the three mainstream LLM API specs — OpenAI, Gemini, and Anthropic — for load- and performance-testing gateways, proxies, and clients. It speaks each provider's real wire format (streaming SSE and non-streaming), echoes your request back, and reports plausible token usage — so traffic exercises your whole pipeline (routing, auth, accounting, streaming) without a real, paid, rate-limited provider behind it.

Built to sustain tens of thousands of concurrent connections: no per-request logging, a fast JSON path (go_json), zero-goroutine precomputed streaming, container-aware GOMAXPROCS, and a tuned HTTP server.

load generator ─▶ your gateway/proxy ─▶ nexus-mock-provider ─▶ echoed response
                  (the thing you're      (this project: free,
                   actually measuring)    instant, deterministic)

It's a test fixture, not a secure service. It does not validate API keys (any credential is accepted), echoes request content, and enables permissive CORS. Run it on loopback or a trusted network. See SECURITY.md.

---

Supported specs & endpoints

Select which to serve with --spec (default all mounts every spec; their path namespaces don't collide except GET /v1/models, which is dispatched by the anthropic-version header).

Spec	Chat	Embeddings	Models	Streaming
OpenAI	POST /v1/chat/completions · POST /v1/responses (+ GET/DELETE /v1/responses/{id})	POST /v1/embeddings	GET /v1/models, /v1/models/{id}	chat: data:{chunk}…[DONE]; responses: response.created…response.completed (no [DONE])
Gemini	POST /v1beta/models/{m}:generateContent · :streamGenerateContent (?alt=sse or JSON-array) · :countTokens	:embedContent · :batchEmbedContents	GET /v1beta/models, /v1beta/models/{m}	data: {GenerateContentResponse} (no [DONE])
Anthropic	POST /v1/messages · POST /v1/messages/count_tokens	— (no native embeddings API)	GET /v1/models, /v1/models/{id}	event: <name> + data: {…} (message_start … message_stop, no [DONE])
Gemini OpenAI-compat	POST /v1beta/openai/chat/completions	POST /v1beta/openai/embeddings	GET /v1beta/openai/models	OpenAI framing

Plus operational endpoints: GET /healthz (liveness), GET /version (build metadata).

Each spec's GET /v1/models-style endpoint returns that provider's real, current model ids in its native envelope (OpenAI gpt-5.5/gpt-5.4-mini/ text-embedding-3-*; Gemini gemini-3.5-flash/gemini-2.5-pro/gemini-embedding-2; Anthropic claude-fable-5/claude-opus-4-8/claude-sonnet-4-6). Add a prefix with --model-prefix (e.g. mock → mock-gpt-5.5), or replace the whole list with --models.

Quick start

make run                 # builds with go_json, serves :3062, --spec all
# or directly:
cd src && go run -tags go_json ./cmd --server-port 3062 --spec all

Build / Docker:

make build               # -> bin/nexus-mock-provider (host platform, version-stamped)
make build-linux         # -> static linux/amd64
docker build -t nexus-mock-provider src && \
  docker run --rm -p 3062:3062 nexus-mock-provider --server-port 3062 --spec all

Smoke test (all three specs)

# OpenAI
curl -s localhost:3062/v1/chat/completions -H 'content-type: application/json' \
  -d '{"model":"gpt-5.5","messages":[{"role":"user","content":"hi"}]}'
curl -s localhost:3062/v1/models | head -c 120

# Gemini
curl -s 'localhost:3062/v1beta/models/gemini-3.5-flash:generateContent' \
  -H 'content-type: application/json' \
  -d '{"contents":[{"parts":[{"text":"hi"}]}]}'

# Anthropic
curl -s localhost:3062/v1/messages \
  -H 'content-type: application/json' -H 'anthropic-version: 2023-06-01' \
  -d '{"model":"claude-opus-4-8","max_tokens":100,"messages":[{"role":"user","content":"hi"}]}'

The mock echoes your last user message back as the completion.

Configuration

Every setting is available as a flag or an environment variable (the env value becomes the flag's default; an explicit flag wins). This keeps it ergonomic under both systemd (env) and the command line (flags).

Flag	Env var	Default	Description
--server-port	MOCK_SERVER_PORT	3000	TCP port to bind (all interfaces).
--spec	MOCK_SPEC	all	Spec(s) to mount: all or comma list of openai,gemini,anthropic.
--stream-delay-ms	MOCK_STREAM_DELAY_MS	0	Per-frame streaming delay (ms); 0 = instant. Simulates a slow provider.
--pprof-addr	MOCK_PPROF_ADDR	(off)	Start net/http/pprof on this address (e.g. 127.0.0.1:6060).
--models	MOCK_MODELS	(per-spec defaults)	Comma-separated override for every spec's model catalog.
--model-prefix	MOCK_MODEL_PREFIX	(none)	Prefix added to advertised model ids, e.g. mock → mock-gpt-5.5.
(n/a)	GIN_MODE	release	Set to debug for gin debug mode.

Examples:

# Only the OpenAI surface, on :8080, advertising mock-prefixed ids
nexus-mock-provider --spec openai --server-port 8080 --model-prefix mock

# Anthropic + Gemini, custom catalog, 20ms/token streaming, pprof on
MOCK_SPEC=anthropic,gemini MOCK_MODELS=foo,bar MOCK_STREAM_DELAY_MS=20 \
  MOCK_PPROF_ADDR=127.0.0.1:6060 nexus-mock-provider

Performance & high concurrency

Designed to disappear into the background of a load test even at very high RPS:

• No per-request logging — access logging is off; only panics are logged.
• Fast JSON — gin's bind/render and the codecs' stream-frame marshaling both use goccy/go-json (-tags go_json, wired into Makefile/Docker/CI/release).
• Zero-goroutine streaming — SSE frames are precomputed and written through a single shared writer; no goroutine-per-stream. (Anthropic's named-event frames use typed structs, not maps, to stay allocation-light.)
• Container-aware — go.uber.org/automaxprocs sets GOMAXPROCS from the cgroup CPU quota, avoiding scheduler thrashing under CPU limits.
• Tuned http.Server — ReadHeaderTimeout (slowloris guard) + IdleTimeout (reclaim idle keep-alives); no WriteTimeout so long SSE streams aren't cut.
• Graceful shutdown — SIGINT/SIGTERM drains in-flight requests.

Tips: raise LimitNOFILE (the systemd unit sets 1048576); for max-throughput / gateway-overhead numbers prefer non-streaming requests (the mock returns instantly, so you measure your gateway, not it); profile under load with --pprof-addr and go tool pprof.

Behaviour & fidelity notes

• Echo semantics: the last user message is echoed back, capped to the request's max-output-tokens (default 256); empty input → "ok". Usage scales with input size.
• Deterministic: same input → same output (including embedding vectors, which are L2-normalized so ‖v‖≈1). Response ids are fixed (chatcmpl-mock, msg_mock) by design — handy for assertions, but a gateway that dedupes on response id will collapse them.
• No auth enforcement: any/no credential is accepted (it's a test fixture).
• Routing is by endpoint, not by model name. The response format is determined by the URL you call, not the model field — which is only echoed back. So a cross-spec model on the wrong endpoint (e.g. claude-… to /v1/chat/completions, or gpt-… to /v1/messages), an unknown model, or a typo all return 200 in that endpoint's format with the model echoed — never a model_not_found/404 (the catalog is configurable, and the mock deliberately doesn't get in the way).
• Tools/function-calling, images, and thinking blocks are accepted and ignored; the mock echoes text. Anthropic has no embeddings endpoint (faithful to the real API, which recommends Voyage AI).

Deployment

• Systemd + nginx: DEPLOY.md, deploy/.
• Behind a gateway as a provider: CONFIGURE-NEXUS.md.
• Behind LiteLLM / Bifrost: CONFIGURE-LITELLM-BIFROST.md.

Development

make check      # gofmt + go vet + go test
make cover      # tests + aggregate coverage (~97%)
make bench      # benchmarks
make lint       # golangci-lint

Architecture: a spec-agnostic core (pkg/core: echo + token + vector engine) and one SSE writer (pkg/sse) under three thin per-spec codecs (pkg/spec/{openai,gemini,anthropic}) that translate to/from each wire format. Adding a provider = adding one codec. The Go module lives under src/. See CONTRIBUTING.md.

License

Released under the Apache-2.0 License.