Skip to content

Commit 5e8a2ec

Browse files
authored
Merge pull request #18 from KadenMc/feat/aexp-jupyter-introspection
feat: live Jupyter session introspection + CRLF/LF install fix
2 parents 84b9811 + 4e640f5 commit 5e8a2ec

17 files changed

Lines changed: 1961 additions & 42 deletions

.gitattributes

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Line-endings policy
2+
# ====================
3+
#
4+
# This package ships text files (slash commands, vendored docs, JSON snippets)
5+
# inside the wheel and copies them into consumer repos at `aexp install` time.
6+
# Wheels store file bytes verbatim — whatever line endings are in the working
7+
# tree at build time go into the wheel and out to every consumer.
8+
#
9+
# To guarantee LF everywhere regardless of which OS built the wheel (and to
10+
# stop git's auto-CRLF on Windows from re-introducing CRLF into the working
11+
# tree on checkout), we force LF for every text file kind we touch.
12+
#
13+
# After editing this file the first time, run:
14+
#
15+
# git add --renormalize . && git commit -m "normalize line endings to LF"
16+
#
17+
# to re-normalize every tracked text file in the working tree.
18+
19+
# Default: auto-detect text vs. binary, normalize text to LF in the object store
20+
* text=auto
21+
22+
# Force LF in the working tree for every text type we ship or author.
23+
# `eol=lf` overrides `core.autocrlf=true` on Windows, so Windows devs no longer
24+
# pollute commits with CRLF.
25+
*.py text eol=lf
26+
*.md text eol=lf
27+
*.json text eol=lf
28+
*.toml text eol=lf
29+
*.yaml text eol=lf
30+
*.yml text eol=lf
31+
*.txt text eol=lf
32+
*.rst text eol=lf
33+
*.csv text eol=lf
34+
*.cfg text eol=lf
35+
*.ini text eol=lf
36+
*.sh text eol=lf
37+
.gitignore text eol=lf
38+
.gitattributes text eol=lf
39+
CHANGELOG.md text eol=lf
40+
41+
# Explicitly binary (defensive; auto-detect would normally catch these).
42+
*.png binary
43+
*.jpg binary
44+
*.jpeg binary
45+
*.gif binary
46+
*.pdf binary
47+
*.ico binary
48+
*.zip binary
49+
*.tar.gz binary
50+
*.whl binary

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Claude Code runtime state — never commit
22
.claude/scheduled_tasks.lock
33
.claude/settings.local.json
4+
CLAUDE.local.md
45

56
# Byte-compiled / optimized / DLL files
67
__pycache__/

CHANGELOG.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,92 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [0.3.0]
11+
12+
### Release summary
13+
14+
`aexp.jupyter` — live introspection for the Jupyter MCP integration. An
15+
agent that has connected to *any* Jupyter the user is running can now
16+
answer "what am I sitting on, what else does the user have running, and
17+
what's happening inside each session?" without consulting any
18+
persistent registry — every fact is recovered from live state on
19+
demand.
20+
21+
The most acute use case is multi-job HPC: a researcher has two SLURM
22+
Jupyters on different ports reached via separate SSH tunnels, and the
23+
agent must never accidentally execute code on the wrong kernel. Today,
24+
no MCP tool exposes "which URL am I currently connected to" — the
25+
context evaporates across resumed sessions. A new PostToolUse hook
26+
nudges the agent to re-introspect after every `connect_to_jupyter`
27+
call so identity beliefs cannot go stale silently.
28+
29+
### Added — `aexp.jupyter` (live session introspection)
30+
31+
- `aexp.jupyter.init() -> SessionInfo` — composes every introspection
32+
probe (technical identity, SLURM context, attached notebooks, GPU
33+
residents, sibling Jupyters) into one Pydantic model. Side-effect
34+
free. Designed to be called from inside any kernel.
35+
- `aexp.jupyter.whoami()` — alias for `init()`.
36+
- `aexp.jupyter.discover_other_servers()` — enumerates every Jupyter
37+
visible from `list_running_servers`, excluding the current one. On
38+
shared-home HPC this enumerates cluster-wide.
39+
- `aexp.jupyter.describe_server(url, token)` — HTTP-only summary of a
40+
remote Jupyter's `/api/sessions` and `/api/kernels` (attached
41+
notebooks, kernel state). Never executes code on the remote kernel.
42+
- `aexp.jupyter.probe_slurm()` / `probe_gpu()` — extracted helpers,
43+
independently testable.
44+
45+
### Added — CLI
46+
47+
- `aexp jupyter` subcommand group: `init`, `whoami`, `discover`,
48+
`setup` (the existing extension-disable recipe). All support
49+
`--json` for scripting.
50+
- `aexp jupyter-setup` retained as a deprecated alias for one release
51+
(delegates to `aexp jupyter setup` with a warning).
52+
53+
### Added — MCP tools
54+
55+
- `jupyter_introspect_current` — returns the Python snippet the agent
56+
should dispatch via the Jupyter MCP's `execute_code` to introspect
57+
the connected kernel. Recipe-based (not direct cross-MCP coupling)
58+
so responsibility boundaries stay clean.
59+
- `jupyter_parse_introspection(raw_output)` — validates and parses the
60+
stdout into a structured `SessionInfo`.
61+
62+
### Added — Hooks
63+
64+
- `aexp.hooks.jupyter_connect_postuse` — PostToolUse matcher on
65+
`mcp__jupyter.*__connect_to_jupyter`. After every connection switch,
66+
emits a high-salience directive telling the agent to re-run
67+
`init()` immediately and verify SessionInfo before any further
68+
`execute_code` / `execute_cell` call. Registered automatically when
69+
`aexp install --with-jupyter` was used (sticky bit honored).
70+
71+
### Added — Slash commands
72+
73+
- `/aexp-jupyter-discover` — list every Jupyter the user is running
74+
with port, attached notebook, SLURM job, and GPU residents.
75+
- `/aexp-jupyter-connect <port-or-hint>` — switch the active Jupyter
76+
using a port number or SLURM-job hint; the PostToolUse hook handles
77+
re-init.
78+
- `/aexp-jupyter-iterate` updated — prepends a "Step 0: confirm
79+
identity" that runs the introspection recipe before touching cells.
80+
81+
### Decisions documented
82+
83+
- **No persistent registry.** Everything is recomputed from live
84+
introspection. Specifically: no `~/.aexp/sessions/`, no marker
85+
files, no policy files.
86+
- **No PreToolUse enforcement hook.** Without persistent policy state
87+
there's nothing to enforce against. Re-init is informed
88+
deterministically via the PostToolUse directive; compliance is by
89+
discipline (and the conversation-level statement of intent).
90+
- **Recipe-based MCP dispatch.** The `aexp` MCP returns code for the
91+
agent to dispatch via the Jupyter MCP, rather than the two servers
92+
talking to each other.
93+
94+
## [0.2.1]
95+
1096
### Release summary
1197

1298
Two new surfaces lifted out of the 2026-05-10 electricrag prompt-

docs/mcp.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,8 @@ teammates get the MCP server on clone.
9999
| `validate` | Compose KB + run-link + finding-citation checks ||
100100
| `sync_offline` | `wandb sync` every offline run in the store ||
101101
| `queue_stop` | Interrupt a running queued job; transitions to `"stopped"` | `job_id` |
102+
| `jupyter_introspect_current` | Returns the recipe for live-introspecting the connected kernel via the Jupyter MCP's `execute_code`. Pair with `jupyter_parse_introspection`. ||
103+
| `jupyter_parse_introspection` | Parses the stdout of an `aexp.jupyter.init()` dispatch into a structured `SessionInfo`. | `raw_output` |
102104

103105
All return JSON-serializable dicts. Errors surface either as
104106
`{"error": ..., "code": ...}` in the return value or as MCP error

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "agentic-experiments"
3-
version = "0.2.1"
3+
version = "0.3.0"
44
description = "Git-first, hypothesis-forcing experiment tracking for agent-driven ML research. Vendors Limina as the research harness, uses signac for local execution/run state, and bridges to W&B for remote observability."
55
authors = [
66
{name = "Kaden McKeen", email = "mckeenkaden@gmail.com"}

src/aexp/cli.py

Lines changed: 173 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -903,8 +903,61 @@ def sync_offline_cmd(
903903
_exit(1)
904904

905905

906-
@app.command("jupyter-setup")
907-
def jupyter_setup(
906+
# ---------------------------------------------------------------------------
907+
# jupyter subcommand group — live session introspection + extension setup
908+
# ---------------------------------------------------------------------------
909+
910+
911+
jupyter_app = typer.Typer(
912+
help=(
913+
"Live Jupyter session introspection (whoami, discover) plus the "
914+
"extension-disable/enable recipe needed before the Jupyter MCP can "
915+
"talk to a cluster JupyterLab."
916+
),
917+
no_args_is_help=True,
918+
)
919+
app.add_typer(jupyter_app, name="jupyter")
920+
921+
922+
def _do_jupyter_setup(dry_run: bool) -> None:
923+
"""Idempotent Jupyter extension reconfiguration.
924+
925+
Body shared between ``aexp jupyter setup`` (the canonical verb) and the
926+
deprecated flat ``aexp jupyter-setup`` alias.
927+
"""
928+
import subprocess
929+
import sys as _sys
930+
931+
cmds: list[list[str]] = [
932+
[_sys.executable, "-m", "jupyter", "server", "extension", "disable", "jupyter_server_documents"],
933+
[_sys.executable, "-m", "jupyter", "server", "extension", "enable", "jupyter_server_ydoc"],
934+
[_sys.executable, "-m", "jupyter", "server", "extension", "enable", "jupyter_server_nbmodel"],
935+
[_sys.executable, "-m", "jupyter", "labextension", "disable", "@jupyter-ai-contrib/server-documents"],
936+
]
937+
938+
for cmd in cmds:
939+
printable = " ".join(cmd)
940+
if dry_run:
941+
console.print(f"[cyan][dry-run][/cyan] {printable}")
942+
continue
943+
console.print(f"[dim]$[/dim] {printable}")
944+
result = subprocess.run(cmd, capture_output=True, text=True)
945+
if result.stdout:
946+
console.print(result.stdout.rstrip())
947+
if result.returncode != 0:
948+
console.print(f"[red]FAILED ({result.returncode})[/red] {result.stderr.rstrip()}")
949+
_exit(1)
950+
return
951+
952+
if dry_run:
953+
console.print("\n[cyan]dry-run complete[/cyan] — no changes applied.")
954+
return
955+
console.print("\n[green]✓[/green] Jupyter extension state configured.")
956+
console.print(" Restart your JupyterLab process to pick up the changes.")
957+
958+
959+
@jupyter_app.command("setup")
960+
def jupyter_setup_cmd(
908961
dry_run: bool = typer.Option(
909962
False,
910963
"--dry-run",
@@ -940,35 +993,129 @@ def jupyter_setup(
940993
See `docs/setup/jupyter-mcp.md` "Investigation log" §1-3 for the full
941994
rationale. Restart your JupyterLab process to pick up the changes.
942995
"""
943-
import subprocess
944-
import sys as _sys
996+
_do_jupyter_setup(dry_run)
945997

946-
cmds: list[list[str]] = [
947-
[_sys.executable, "-m", "jupyter", "server", "extension", "disable", "jupyter_server_documents"],
948-
[_sys.executable, "-m", "jupyter", "server", "extension", "enable", "jupyter_server_ydoc"],
949-
[_sys.executable, "-m", "jupyter", "server", "extension", "enable", "jupyter_server_nbmodel"],
950-
[_sys.executable, "-m", "jupyter", "labextension", "disable", "@jupyter-ai-contrib/server-documents"],
951-
]
952998

953-
for cmd in cmds:
954-
printable = " ".join(cmd)
955-
if dry_run:
956-
console.print(f"[cyan][dry-run][/cyan] {printable}")
957-
continue
958-
console.print(f"[dim]$[/dim] {printable}")
959-
result = subprocess.run(cmd, capture_output=True, text=True)
960-
if result.stdout:
961-
console.print(result.stdout.rstrip())
962-
if result.returncode != 0:
963-
console.print(f"[red]FAILED ({result.returncode})[/red] {result.stderr.rstrip()}")
964-
_exit(1)
999+
@app.command("jupyter-setup", hidden=True)
1000+
def jupyter_setup_deprecated_alias(
1001+
dry_run: bool = typer.Option(
1002+
False,
1003+
"--dry-run",
1004+
"-n",
1005+
help="Print the commands that would be run without executing them.",
1006+
),
1007+
) -> None:
1008+
"""Deprecated alias for ``aexp jupyter setup``. Kept for one release."""
1009+
console.print(
1010+
"[yellow]warning[/yellow] `aexp jupyter-setup` is deprecated; use "
1011+
"`aexp jupyter setup` (subcommand) instead."
1012+
)
1013+
_do_jupyter_setup(dry_run)
1014+
1015+
1016+
def _print_session_info(json_out: bool) -> None:
1017+
"""Shared body for ``aexp jupyter init`` / ``aexp jupyter whoami``."""
1018+
try:
1019+
from aexp.jupyter import _print_info_human, init
1020+
except ImportError as exc:
1021+
console.print(f"[red]{exc}[/red]")
1022+
_exit(1)
1023+
return
1024+
info = init()
1025+
if json_out:
1026+
# model_dump_json is the wire format the MCP recipe also produces.
1027+
print(info.model_dump_json(indent=2))
1028+
return
1029+
_print_info_human(info)
1030+
1031+
1032+
@jupyter_app.command("init")
1033+
def jupyter_init_cmd(
1034+
json_out: bool = typer.Option(
1035+
False, "--json", help="Emit SessionInfo as JSON instead of human-readable text."
1036+
),
1037+
) -> None:
1038+
"""Introspect the current Jupyter session and print the result.
1039+
1040+
Most useful when run from inside a Jupyter terminal or a notebook cell
1041+
(``!aexp jupyter init``) — populates SLURM context, attached
1042+
notebooks, GPU residents, and sibling Jupyters from live state.
1043+
"""
1044+
_print_session_info(json_out)
1045+
1046+
1047+
@jupyter_app.command("whoami")
1048+
def jupyter_whoami_cmd(
1049+
json_out: bool = typer.Option(
1050+
False, "--json", help="Emit SessionInfo as JSON instead of human-readable text."
1051+
),
1052+
) -> None:
1053+
"""Alias for ``aexp jupyter init`` — identical output, friendlier verb."""
1054+
_print_session_info(json_out)
1055+
1056+
1057+
@jupyter_app.command("discover")
1058+
def jupyter_discover_cmd(
1059+
json_out: bool = typer.Option(
1060+
False, "--json", help="Emit the sibling list as JSON instead of a table."
1061+
),
1062+
describe: bool = typer.Option(
1063+
False,
1064+
"--describe",
1065+
help=(
1066+
"Probe each sibling's /api/sessions and /api/kernels for attached "
1067+
"notebook paths + kernel state (one HTTP roundtrip per server)."
1068+
),
1069+
),
1070+
) -> None:
1071+
"""List every Jupyter the user has running, excluding the current one."""
1072+
try:
1073+
from aexp.jupyter import describe_server, discover_other_servers
1074+
except ImportError as exc:
1075+
console.print(f"[red]{exc}[/red]")
1076+
_exit(1)
1077+
return
1078+
siblings = discover_other_servers()
1079+
if describe:
1080+
rendered = []
1081+
for s in siblings:
1082+
entry = s.model_dump()
1083+
entry["describe"] = describe_server(s.url, s.token)
1084+
rendered.append(entry)
1085+
if json_out:
1086+
import json as _json
1087+
print(_json.dumps(rendered, indent=2, default=str))
9651088
return
1089+
for entry in rendered:
1090+
console.print(
1091+
f"[cyan]{entry['url']}[/cyan] port={entry['port']} "
1092+
f"pid={entry['pid']} host={entry['hostname']}"
1093+
)
1094+
d = entry["describe"]
1095+
if d.get("attached_notebooks"):
1096+
console.print(" notebooks:")
1097+
for nb in d["attached_notebooks"]:
1098+
console.print(f" - {nb}")
1099+
for k in d.get("kernels", []):
1100+
console.print(
1101+
f" kernel: id={k.get('id')} state={k.get('execution_state')} "
1102+
f"last={k.get('last_activity')}"
1103+
)
1104+
if not rendered:
1105+
console.print("[dim]no other Jupyter servers visible[/dim]")
1106+
return
9661107

967-
if dry_run:
968-
console.print("\n[cyan]dry-run complete[/cyan] — no changes applied.")
1108+
if json_out:
1109+
import json as _json
1110+
print(_json.dumps([s.model_dump() for s in siblings], indent=2, default=str))
9691111
return
970-
console.print("\n[green]✓[/green] Jupyter extension state configured.")
971-
console.print(" Restart your JupyterLab process to pick up the changes.")
1112+
if not siblings:
1113+
console.print("[dim]no other Jupyter servers visible[/dim]")
1114+
return
1115+
for s in siblings:
1116+
console.print(
1117+
f"[cyan]{s.url}[/cyan] port={s.port} pid={s.pid} host={s.hostname}"
1118+
)
9721119

9731120

9741121
@app.command("install-slash-commands")

0 commit comments

Comments
 (0)