spike: codegen experiments (SDK-1107)

[Tr00d]

Status: 🚧 DRAFT — spike, not for merge

This is an evaluation spike. All generated code lives in isolated, standalone projects that are in the solution for visibility but are not referenced by Supabase.csproj (analyzers disabled, separate assemblies) — nothing here affects the product build, analyzers, or quality metrics.

Summary

Evaluates whether the HTTP layer of the C# SDK (Storage / Functions / PostgREST; Auth is blocked by model coverage) can be generated from the shared Smithy models in supabase/sdk#51, producing idiomatic C# that a maintainer would own long-term. Input is the committed Smithy→OpenAPI artifacts; Smithy is never run in the C# toolchain. All evaluated generators are deterministic, template-based tools — output is a function of the input document and configuration only, with no generative-AI component.

Two toolchains were run end-to-end (NSwag and Kiota), the rest of the brief's list assessed with rationale, and both generated clients were exercised against a live local Supabase platform. The model gaps found by the live runs were fixed upstream in supabase/sdk#55 and verified by rerunning the spike against the corrected artifacts.

What's included

Project (in Supabase.sln)	What
nswag-spike/ (spike-nswag)	NSwag-generated Storage/Functions/Database clients + models; evaluation + findings
kiota-spike/ (spike-kiota)	Kiota-generated clients + models; evaluation
nswag-testrun/ (spike-nswag.TestRun)	Console harness driving the NSwag client against a local platform
kiota-testrun/ (spike-kiota.TestRun)	Same, for Kiota
codegen-comparison.md (root)	Cross-toolchain comparison, cost analysis, and recommendation — start here

Key findings

• Judged against the current SDK (owned working transport, published public model types), neither generated client improves on the shipped code: NSwag's client re-implements the existing MakeRequest transport; Kiota's replaces it with 4,338 generated LOC plus an 8-package runtime, without the current ergonomics (progress reporting, caching, FailureHint mapping). Generated models do add value — as drift protection against the shared contract and as the type source for new endpoints.
• Separability decides the tool. NSwag's models are plain POCOs (System.Text.Json only) usable without its client; Kiota's models implement IParsable, require its runtime, and place Microsoft.Kiota.Abstractions on the SDK's public API — there is no models-only adoption path for Kiota.
• The live runs surfaced two defects, both located in the shared model (a work in progress), not in the generators: the list-response envelope (model wrapped arrays the API returns bare — Kiota returned empty results silently, NSwag threw) and missing nullable on optional fields (NSwag serialized defaults the server rejects; Kiota omits unset fields). Both generators translated their input faithfully and deterministically — which means the fixes, made once in the model, propagated to both toolchains on regeneration. Validation must target the model; the test-run harnesses are the mechanism.
• Fixed upstream and verified (feat(smithy): provides fixes based on C# spike supabase/sdk#55): list envelopes, optional-scalar nullability, streaming byte→binary, the {wildcardPath+} path label, and build reproducibility. Rerun results: both clients compile with 0 errors from the committed artifacts verbatim (previously 24 compile errors without local patching); Kiota live harness 4/4 — ListBuckets returns real counts (the false-zero failure is gone); NSwag deserializes lists and creates buckets without file_size_limit (no more 413-trap). NSwag's one remaining live failure (ListObjects 400) is generator-side null-serialization, addressed by client config (WhenWritingNull), not a model defect.
• Live-run scope: Storage operations exercised; Functions and Database clients generated and compiled but not exercised (live Functions streaming needs a deployed edge function — still open).

Recommendation — adapt, with NSwag in models-only mode, in two stages

• Stage 1 (now, no change to existing public API): generate wire DTOs with NSwag (remaining local fixes: WhenWritingNull serialization config, source-generated JsonSerializerContext for AOT, PascalCase naming — optional-field typing now comes from the spec); use generated models as the type source for newly implemented endpoints; stand up a drift monitor — generated models diffed against the existing public types, with a committed baseline, reported and triaged (divergence on implemented surface = defect/contract change; unimplemented surface = parity-backlog input), not a build gate; adopt HttpCompletionOption.ResponseHeadersRead in Core.
• Stage 2 (next major version): decide whether generated types replace the existing public model types, bundled with the breaking changes already planned for the feature-parity catch-up, conditional on the contract source having proven reliable through Stage-1 monitoring.

Scope of validity: these conclusions are specific to an SDK with owned transport, a published public API, and a parity backlog. For an SDK built from scratch, the same evaluation favours whole-client generation (Kiota is the strongest candidate assessed). Invariant in both settings: generated code is only as correct as the contract it is generated from, and a live contract-test harness is a required pipeline stage. Full rationale, cost analysis, and reversal criteria in codegen-comparison.md.

Upstream model gaps — status

1. ✅ List-response envelope — fixed in supabase/sdk#55, verified live.
2. ✅ Optional fields not marked nullable — fixed in supabase/sdk#55, verified live.
3. ✅ Streaming byte→binary; {wildcardPath+} label; build reproducibility — fixed in supabase/sdk#55.
4. ⏳ Auth unmodelled — needs models or a formal decision to scope Auth out as hand-written.
5. ⏳ Write ops model 200 only — the API can return 204; currently handled in hand-written code.

How to run the harnesses

supabase start && supabase status     # copy the API URL + service_role key
cd kiota-testrun   # or nswag-testrun
SUPABASE_URL=http://127.0.0.1:54321 SUPABASE_KEY=<service_role key> dotnet run

🤖 Generated with Claude Code