Skip to content

AGENTS.md

Latest commit

 

History

History
119 lines (76 loc) · 16.9 KB

AGENTS.md

File metadata and controls

119 lines (76 loc) · 16.9 KB
Raw

What this is

csharprepl is a cross-platform command-line C# REPL distributed as a .NET 10 global tool. It evaluates C# with Roslyn's scripting API and provides IntelliSense, syntax highlighting, NuGet/project/assembly references, object inspection, IL/lowered-C# disassembly (ILSpy), and provider-agnostic AI code completion (via Microsoft.Extensions.AI). The terminal UX (input, autocomplete menus, highlighting) comes from the separate PrettyPrompt library (same author, separate NuGet package/repo).

Everything targets net10.0. The SDK is pinned in global.json (10.0.0, latestMajor, prereleases allowed).

Build, run, test

dotnet build CSharpRepl.slnx                 # build the whole solution
dotnet run --project CSharpRepl              # run the REPL locally
dotnet test                                  # run the test suite (see gotchas below)

Tests use Microsoft.Testing.Platform (MTP), not VSTest

The test runner is Microsoft.Testing.Platform with the xUnit v3 runner (pinned in global.json under "test"). This changes the command line:

  • dotnet test (optionally targeting the project: dotnet test Tests/CSharpRepl.Tests/CSharpRepl.Tests.csproj) goes through the MTP front-end, which has its own option set — run dotnet test ... --help to see it; don't assume vstest.console/MSBuild-runner flags.

  • To run a subset, use MTP's *-wildcard filter options (wildcard works at the start and/or end of each pattern; repeat a flag to OR; the "simple" filters below can't be combined with the VSTest or query filter in one run):

    dotnet test Tests/CSharpRepl.Tests/CSharpRepl.Tests.csproj --filter-class "*ShellDetectorTests"
dotnet test Tests/CSharpRepl.Tests/CSharpRepl.Tests.csproj --filter-method "*ShellDetectorTests.DetectShell_IsBestEffortAndNeverThrows"
dotnet test Tests/CSharpRepl.Tests/CSharpRepl.Tests.csproj --filter-namespace "CSharpRepl.Tests"

    --filter-class (FQ type), --filter-method (FQ Type.Method), --filter-namespace, and --filter-trait "name=value"; each has a --filter-not-* exclude variant. There's also --filter "<VSTest syntax>" and a path-form query filter, but only one filter category per run.

Test-suite behavior to know about

  • Heavy Roslyn/integration tests share [Collection(nameof(RoslynServices))] and run serially on purpose: the loader's AssemblyLoadContext.Resolving hooks (attached to the process-global Default ALC by AssemblyLoadContextHook and never detached) are process-global, not per-RoslynServices. The full suite is ~2 minutes. (MSBuildLocator.RegisterDefaults() is also process-global, but a [ModuleInitializer] in TestAssemblyInitializer runs it once at assembly load, so it's no longer the reason for the collection — and tests that only needed MSBuildLocator, like NugetPackageInstallerTests, can now run in isolation.)
  • Some tests spawn dotnet build / MSBuild subprocesses (solution/project references) and a few touch the network (NuGet install). These are the slow ones and can occasionally be flaky.
  • The connect-feature integration tests (ConnectorRoundTripTests, ConnectorCancellationTests, RemoteEditorServicesTests, ConnectorServerProtocolTests, RemoteReadEvalPrintLoopTests, RemotePipedInputEvaluatorTests) launch a real hooked child process — the interactive PrettyPrompt loop itself cannot be driven without a TTY, so RemoteReadEvalPrintLoopTests stubs IPrompt (like ReadEvalPrintLoopTests) and everything below it is real. The non-interactive connect path (connect <pid> --eval/--eval-file/piped stdin) needs no TTY, so RemotePipedInputEvaluatorTests drives the real RemotePipedInputEvaluator against a real hooked child directly. Two more connect suites run in-process without a child: ConnectorEngineTests hosts the real engine + Roslyn inside the test process, and ConnectorTransportTests exercises the real OS pipe/socket transport (note: the Windows pipe uses zero-byte buffers, so a write rendezvouses with the peer's read — keep the read pending while writing).

Benchmarks

BenchmarkDotNet project at Tests/CSharpRepl.Benchmarks (in the solution; BenchmarkDotNet.Artifacts/ is gitignored):

dotnet run -c Release --project Tests/CSharpRepl.Benchmarks -- --filter *AllocationBreakdown*

Solution layout

  • CSharpRepl/ — the executable / global tool. Program.cs (CLI args, help, the read-eval-print loop), CommandLine.cs (argument parsing), CSharpReplPromptCallbacks.cs (wires PrettyPrompt callbacks to Roslyn services), ReadEvalPrintLoop.cs.
  • CSharpRepl.Services/ — the bulk of the logic. Roslyn/ (scripting + workspace, see below), Completion/, SyntaxHighlighting/, Theming/, Nuget/, SymbolExploration/ (Source Link), CodeTransformation/ (IL disassembly + ILSpy lowering), and Remote/ (controller side of the connect feature).
  • InjectedHook/ — the three projects for the "connect a running process" feature (see below).
  • Tests/ — CSharpRepl.Tests and CSharpRepl.Benchmarks.
  • ARCHITECTURE.md — the authoritative deep-dive on design, including sequence/class diagrams for the connect feature. Read it before substantial changes.
  • CSharpRepl.Services/Roslyn/References/AssemblyReferenceReadme.md — the authoritative deep-dive on the #r/NuGet/assembly-load-context machinery: compile-time-vs-run-time resolution, reference-vs-implementation assemblies, NuGet restore + version unification, and the dedicated load context. Read it before touching reference/NuGet/loading code.

Core architecture

csharprepl is an intermediary between Roslyn and PrettyPrompt. The single most important concept is that Roslyn is used through two separate APIs that must be kept in sync:

  1. Scripting world — the C# Scripting API (CSharpScript) actually executes code and holds the ScriptState chain. Each submission is ContinueWithAsync'd onto the previous, so locals, declared methods, and types persist line-to-line. Lives in Roslyn/Scripting/ScriptRunner.cs.
  2. Workspace world — the C# Workspaces API powers the editor features (highlighting, completion, tooltips, overloads, symbol lookup). The workspace is a linked list of projects, one per submitted line, each referencing the previous (Roslyn/WorkspaceManager.cs).

Roslyn/RoslynServices.cs is the façade over both. It manages Roslyn's slow initialization in the background (so the prompt stays responsive before init finishes) and, on each successful evaluation, advances the workspace with a new project/document so highlighting and completion see the latest state. When changing evaluation flow, preserve this scripting↔workspace consistency.

#r references (assemblies, NuGet packages, .csproj/.sln/.slnx) are resolved on two independent paths that must agree. At compile time, the Roslyn/MetadataResolvers/ chain (a MetadataReferenceResolver extension point) plus AssemblyReferenceService produce the MetadataReferences Roslyn binds against; this is also where shared-framework and implementation-vs-reference-assembly concerns live. At run time, Roslyn/References/ReplAssemblyLoader.cs loads the real implementation assemblies into a single dedicated AssemblyLoadContext ("CSharpReplLoadContext") through a name→path registry, with an AssemblyLoadContext.Resolving fallback that AssemblyLoadContextHook attaches to the relevant contexts. NuGet is restore-based: Nuget/NugetPackageInstaller.cs runs a real in-process NuGet.Commands.RestoreCommand over the whole accumulated #r "nuget:" set, so transitive versions unify exactly as a built app's restore would (one version per assembly). Most "compiles fine but throws at run time" bugs come from those two paths disagreeing — read Roslyn/References/AssemblyReferenceReadme.md before changing any of it.

Performance note (per-keystroke latency)

Per-keystroke latency is dominated by syntax highlighting, which historically scaled linearly with submission count. The fix lives in WorkspaceManager.UpdateCurrentDocumentAsync: after setting the current document it awaits GetCompilationAsync() and discards the result, forcing Roslyn's compilation tracker to its strongly-held final state so per-keystroke forks reuse it. Removing that line reintroduces an O(depth) regression — don't.

Connect-a-running-process feature (InjectedHook/ + CSharpRepl.Services/Remote/)

csharprepl can attach to a separate, already-running .NET app and evaluate C# inside it, reading/writing its live state with full local-REPL parity. CLI: csharprepl connect init (prints the env vars to launch the target with) then csharprepl connect <pid>. It is cooperative (a real Roslyn engine is injected via a DOTNET_STARTUP_HOOKS startup hook — not a debugger) and opt-in only.

Naming — read this before searching

The feature was renamed during development; names are inconsistent across layers, so search by the right token:

  • Folder / projects: InjectedHook/ containing CSharpRepl.InjectedHook, CSharpRepl.InjectedHook.Contracts, and CSharpRepl.InjectedHook.ScriptEngine.
    • Note: earlier planning docs use CSharpRepl.Connector.* or CSharpRepl.InjectedHook.Engine; those names no longer exist.
  • Classes: still prefixed Connector* (ConnectorServer, ConnectorEngine, IConnectorEngine, ConnectorClient, ConnectorTransport, ConnectorRoots, ConnectorGlobals).
  • User-facing verb: connect.

The three injected projects and their assembly-load-context (ALC) roles

This is the crux of the design — the target may already load its own Roslyn, so the injected Roslyn must be isolated:

  • CSharpRepl.InjectedHook (bootstrap) — injected into the target's default ALC. References no Roslyn. StartupHook.Initialize() (no namespace, public static void, must never throw and runs before the target's Main) installs an AssemblyLoadContext.Default.Resolving handler, creates the isolated engine ALC (EngineHost.cs), and starts the transport server (ConnectorServer.cs).
  • CSharpRepl.InjectedHook.ScriptEngine — loaded into a dedicated isolated ALC with its own Roslyn closure (Microsoft.CodeAnalysis.CSharp.Scripting). ConnectorEngine.cs hosts CSharpScript, builds compilation references lazily on first eval from the target's loaded assemblies (so submissions bind to the target's real live objects), and projects results into a serializable RemoteValue tree.
  • CSharpRepl.InjectedHook.Contracts — the shared boundary, loaded once in the default ALC and resolved to that same instance from the isolated ALC, so these types are type-identical across the boundary: IConnectorEngine, ConnectorGlobals/ConnectorRoots, RemoteValue, the WireMessages hierarchy, and ConnectorTransport/MessageChannel. Also references no Roslyn.

Controller side (CSharpRepl.Services/Remote/ + CSharpRepl/RemoteReadEvalPrintLoop.cs)

When attached, csharprepl is a thin controller: it compiles nothing for evaluation, sends code strings, and renders the returned RemoteValue through the same theme/formatting pipeline as local output (RemoteValueRenderer). The scripting world lives in the target (the engine), but the workspace world (completion/highlighting) stays in the controller against a second, remote-configured RoslynServices seeded with the target's assembly paths + ConnectorGlobals — so editor features need no per-keystroke round-trip. The controller advances that remote workspace only when an EvalResponse reports Committed == true.

Connect mode also runs non-interactively (so agents/scripts can use it without a TTY), mirroring the local REPL's PipedInputEvaluator: connect <pid> --eval/--eval-file or piped stdin route to RemotePipedInputEvaluator instead of the interactive RemoteReadEvalPrintLoop. It evaluates against the same RemoteSession, auto-prints the final value as plain, uncolored, unwrapped text on stdout (errors to stderr, nonzero exit), and honors the same #replace/#wrap/#patches/#revert commands (the command-result wording is shared via ConnectorCommandResultPrinter). It skips the editor-workspace seeding (completion/highlighting are interactive-only) and the connection chatter. Because the engine's state chain lives in the target, separate one-shot --eval invocations accumulate state across reconnects.

Wire protocol

A single duplex connection (named pipe on Windows, Unix domain socket elsewhere, current-user only). Every frame is a 4-byte little-endian length prefix + UTF-8 JSON body; messages are a System.Text.Json polymorphic WireMessage hierarchy keyed on a $kind discriminator (MessageChannel). Security model mirrors the .NET diagnostic port — OS access control, no secret. A connector-enabled process is RCE-equivalent for same-user code and must never run in production.

Live method replacement

#replace/#wrap/#patches/#revert (controller commands in RemoteReadEvalPrintLoop) detour a live method in the target to a REPL-defined delegate.

  • Engine side: ConnectorPatcher (in CSharpRepl.InjectedHook.ScriptEngine) over MonoMod.RuntimeDetour. Resolves the target by name, matches the overload from the delegate's signature, coerces a bare method group via a generated cast, applies a Hook, tracks it by id for revert.
  • Wire (protocol v2): ReplaceRequest/ReplaceResponse, PatchListRequest/PatchListResponse, RevertRequest/RevertResponse, plus three IConnectorEngine methods served in ConnectorServer.
  • Replacement shape: instance methods take the instance as the first parameter; #wrap prepends an orig delegate the body can call.
  • Detours repoint native code, so they cross the engine-ALC/target-ALC boundary: the replacement compiles in the engine ALC, the target method lives in the default ALC.
  • ref/out/in work in Replace mode: BuildCastDelegate emits a delegate type carrying the modifiers (Func can't), which the engine prepends to the probe submission before the method-group cast. Not supported: generic methods, pointer parameters, Wrap + by-ref (the orig delegate can't be a shared type).
  • Limits: JIT-inlined call sites keep old behavior. Patches persist after detach until reverted.
  • Tests: ConnectorEngineTests.ReplaceMethod_* (in-process engine) and RemoteReadEvalPrintLoopTests (cross-process, real hooked child).

Packaging

CSharpRepl.csproj's IncludeConnectorPayload target stages the full connector payload (bootstrap + contracts + engine + Roslyn closure + MonoMod.RuntimeDetour closure + .deps.json) into an connector/ subdirectory next to the tool (both on dotnet run and in the packed global tool), isolated so the engine's Roslyn never shadows the tool's. connect init points DOTNET_STARTUP_HOOKS at connector/CSharpRepl.InjectedHook.dll. The bootstrap is a ProjectReference with ReferenceOutputAssembly="false" (the tool must not link it).

Gotchas

  • System.CommandLine is v3 (3.0.0-preview.4), driven via RootCommand.Parse(...) + GetValue (never Invoke). Two traps in CommandLine.cs: (1) a command with subcommands but no action makes Parse emit a "Required command was not provided" error — give every such command a no-op SetAction(_ => 0); (2) GetValue(argument) throws on an unparseable token, so to validate yourself read the raw token (parseResult.GetResult(arg)?.Tokens) before int.TryParse. Recursive/global options use option.Recursive = true.
  • MSBuild assemblies must stay out of the output dir (MSBL001). Microsoft.Build.Framework / Microsoft.NET.StringTools are referenced with ExcludeAssets="runtime" PrivateAssets="all"; MSBuild loads from the SDK at runtime via MSBuildLocator. When targeting a new SDK major, bump NuGet.* references in CSharpRepl.Services to match the SDK's bundled NuGet version.
  • NuGet.* all resolve from the SDK at run time. In CSharpRepl.Services.csproj every NuGet.* package is PrivateAssets="all": the compile-time reference stays but the runtime DLL isn't shipped, so they bind to the SDK's bundled copies (same MSBuildLocator mechanism as the MSBL001 fix above — avoids version skew, and the reason NugetPackageInstaller's NuGet types only load after MSBuildLocator.RegisterDefaults() has run). Because nothing under NuGet.* ships, the R2R exclude list in CSharpRepl.csproj (active under -p:PackRidSpecific=true) needs only Microsoft.CodeAnalysis.Workspaces.MSBuild.dll (also SDK-loaded). Note: before the restore-based rework, NuGet.PackageManagement (and the NuGet.Resolver it dragged in) was shipped — the SDK doesn't bundle it — leaving an incomplete reference closure that broke R2R crossgen2, so both .dlls also had to be R2R-excluded. That dependency is gone now. Don't reintroduce a shipped NuGet.* runtime assembly without re-checking MSBL001 and the R2R exclude list (verify with an R2R publish — see Roslyn/References/AssemblyReferenceReadme.md §4).