████████╗██████╗ ██████╗ ██╗██╗      ██████╗ ████████╗
╚══██╔══╝██╔══██╗██╔══██╗██║██║     ██╔═══██╗╚══██╔══╝
   ██║   ██║  ██║██████╔╝██║██║     ██║   ██║   ██║
   ██║   ██║  ██║██╔═══╝ ██║██║     ██║   ██║   ██║
   ██║   ██████╔╝██║     ██║███████╗╚██████╔╝   ██║
   ╚═╝   ╚═════╝ ╚═╝     ╚═╝╚══════╝ ╚═════╝    ╚═╝

TDPilot Runtime v2.4.0

Build, debug, and verify TouchDesigner networks by talking to your AI.

TDPilot gives Claude (or Cursor, Codex, any MCP client) live control of a running TouchDesigner session. The agent creates and wires operators, sets real parameter values, screenshots the render to check its own work, and can roll back anything it did — transactionally, with snapshots and a full activity log. It knows TouchDesigner: every one of TD's operators is covered by a reviewed knowledge atlas grounded in official Derivative docs.

Try these, verbatim, once installed:

/td-first-wow

Builds a moving feedback visual in your project, verifies it error-free, and shows you the screenshot — in about two minutes.

/td-concept A luminous particle tunnel with deep fog and slow camera drift

Grounds the complete artistic request, reviews a concrete TD architecture, and executes it only when every required facet is covered.

Make my project audio-reactive: drive the main TOP chain from the kick drum of audio_in.

Plans the CHOP analysis chain, wires it to your visuals, and proves the binding works before calling it done.

Why is my render black? Fix it.

Reads errors recursively, inspects the render chain state, finds the broken link, fixes it, and screenshots the repaired output.

Why artists trust it on real projects: every mutation can be snapshot, validated, and rolled back; risky parameter writes are gated by an operator-semantics contract; and the whole session is auditable. No signup, no API key, no hosted service — your work never leaves your machine.

Install — Claude Code plugin (recommended)

Easiest: paste these two slash commands into any Claude Code session.

/plugin marketplace add dreamrec/TDPilot
/plugin install tdpilot@dreamrec-TDPilot

That installs the full MCP tool surface, the TDPilot skills (tdpilot-core, tdpilot-production, popx-touchdesigner, plus the public brain skills), the brain agents, the guided slash commands (/td-concept, /td-first-wow, /td-audio-reactive, /td-explain-patch, /td-check, /td-snapshot), and the TD-side .tox component — one command, no Python setup required.

Shell one-liner alternative:

curl -fsSL https://raw.githubusercontent.com/dreamrec/TDPilot/main/scripts/install_claude_plugin.sh | bash

Or via npx:

npx tdpilot plugin-install

TouchDesigner side (once, after install)

Drag ~/.claude/plugins/cache/dreamrec-TDPilot/tdpilot/<version>/td_component/tdpilot.tox into your TD /local container. Or paste the auto-setup Python block from docs/INSTALL_CLAUDE_PLUGIN.md into the Textport (auto-detects the latest installed version).

Using Claude Desktop, Cursor, Codex, or another MCP client? See docs/INSTALL_OTHER_CLIENTS.md — one page per client, including the one-click .mcpb bundle for Claude Desktop. (Don't mix the Desktop and Claude Code flows on one machine.)

Documentation

Which doc do I read? Installing → INSTALL_CLAUDE_PLUGIN.md. First hour → GETTING_STARTED.md. Daily driving → USER_GUIDE.md. Running a show → MANUAL.md. Looking up a tool → API_REFERENCE.md. Something broke → TROUBLESHOOTING.md.

• Install (Claude Code plugin): docs/INSTALL_CLAUDE_PLUGIN.md
• Install (Claude Desktop / Cursor / Codex / generic MCP): docs/INSTALL_OTHER_CLIENTS.md
• Getting started: docs/GETTING_STARTED.md
• User guide: docs/USER_GUIDE.md
• Memory guide: docs/MEMORY_GUIDE.md
• Production manual: docs/MANUAL.md
• Practical Intelligence v2.4 architecture and guarantees: docs/PRACTICAL_INTELLIGENCE.md
• API reference: docs/API_REFERENCE.md
• Security model: docs/SECURITY.md
• Troubleshooting: docs/TROUBLESHOOTING.md
• Contributing: CONTRIBUTING.md
• Release notes: CHANGELOG.md
• Architecture deep-dives: docs/TDPILOT_CONCEPT_TO_NODE_MASTER_PLAN.md, docs/TDPILOT_EFFECTIVENESS_ROADMAP.md, docs/MCP_1_1_SURFACE.md (historical)

What This Is

• A practical control layer between AI agents and TouchDesigner — built for iterative patch development, not one-shot guessing.
• A planner that grounds ideas in real operators: every TD operator family (CHOP, COMP, DAT, MAT, POP, SOP, TOP) is covered by a reviewed knowledge atlas sourced from official Derivative docs, with key parameters and gotchas per operator.
• A safety layer professionals can rely on: transactional apply, preflight checks, snapshots, verified rollback, parameter-semantics gating on risky writes, and an activity log of everything the agent touched.
• A technique memory that learns validated patterns from your projects and reuses them — exportable, importable, shareable.
• Diagnostics that close the loop: error scans, cook-time profiling, and screenshot verification so the agent proves its work instead of declaring it.

Packaged Add-ons

• Reviewed operator atlas — The core local add-on for translating abstract ideas into real TD operators. Agents can use the 656-card reviewed operator atlas, Official Derivative source URLs, params, concepts, and gotchas to choose smaller, safer operator chains.
• Brain skills and agents — Codex and Claude Code ship explorer, builder, validator, and recovery workflows. Exact validated patterns use deterministic planning; artistic and multi-domain work uses ground -> author -> propose before transactional execution.
• Scoped transaction hook — The shipped hook runs only after brain/transaction mutations to reinforce final validation. Source-repository release guards check mirror drift, personal paths, and stale artifacts but are not shipped to user projects.
• Optional local knowledge packs — POPX and future specialty packs stay local and user-owned; they extend planning context without adding hosted-service dependencies.

Start Here: Core Workflow

You don't need all 114 tools. Start with these and expand as needed:

Step	Tools	What You're Doing
Route a pattern	td_brain_plan	Compile an exact validated topology, then require complete intent coverage
Author a concept	td_brain_ground, td_brain_propose	Ground artistic, multi-domain, spatial, or implicit architecture and review the authored graph
Execute	td_brain_execute, td_transaction_apply	Apply only a valid BrainPlan or PatchPlan with preflight, snapshots, rollback, and validation
Inspect	td_get_info, td_get_nodes, td_get_params, td_get_errors	Understand current state before touching anything
Check memory	td_memory_recall	See if a reusable technique already exists
Build	td_create_node, td_connect_nodes, td_set_params	Make changes in small, reversible steps
Verify	td_get_errors, td_cooking_info, td_screenshot	Prove the change worked
Protect	td_snapshot_scene, td_restore_snapshot	Save milestones, roll back if needed
Remember	td_memory_learn, td_memory_save	Save successful patterns for reuse

The loop: Inspect -> Route -> Plan or Ground/Propose -> Execute transactionally -> Validate -> Learn only if validated.

Everything else (vision, streaming, optimization, planning, TD2025 inspection) builds on top of this core.

Latest Release

v2.4.0 — Practical Intelligence: truthful pattern-versus-concept routing, server-derived intent coverage, explicit CHOP→levelTOP bindings, cached grounding and recall, 15 deterministic compiler-backed techniques, live task-specific validation, bounded repairs, show-safe staging/route swaps, and ownership-guarded partial rebuilds. The public surface remains 114 tools and local-first. Read the Practical Intelligence guide or the full CHANGELOG.md.

Core Thinking Model (How To Think With This MCP)

Use this loop for every non-trivial task:

1. Inspect the relevant scope — Read focus, target nodes, direct connections, and parameters before touching anything. Batch independent reads when that saves a roundtrip.

2. Check memory — Before building from scratch, use td_memory_recall to check if a similar technique already exists in the library.

3. Choose the route — Use td_brain_plan for an exact validated pattern. Use td_brain_ground -> author -> td_brain_propose for artistic, multi-domain, spatial, or implicit architecture.

4. Execute safely — Execute only complete accepted plans, or use a small typed direct edit for one proven change. Validate at logical checkpoints rather than after every primitive operation.

5. Validate the request — Check graph, runtime, bindings/references, and the requested visual/temporal behavior. Zero TD errors alone is not visual proof.

6. Learn and control cost — Save only validated reusable work. Prefer metadata and one low-quality proof thumbnail; ask before repeated images or streaming.

Tool Map (114 Tools)

0) Brain Planning + Transactions

Use for non-trivial visual programming tasks where complete intent coverage, rollback, and validation matter.

• td_brain_plan, td_brain_ground, td_brain_propose, td_brain_execute, td_transaction_apply, td_cockpit_render

1) Scene + Timeline + Project Lifecycle

Use for global context, playback control, save/load, and undo operations.

• td_get_info, td_list_families, td_timeline, td_timeline_set, td_project_lifecycle

2) Network Build + Wiring

Use for creating, moving, renaming, connecting, and pruning structure.

• td_get_nodes, td_get_node_detail, td_search_nodes
• td_create_node, td_delete_node, td_copy_node, td_rename_node
• td_connect_nodes, td_disconnect, td_get_connections

3) Parameters + DAT Content

Use for patch logic, expressions, config tables, scripts, and trigger pulses.

• td_get_params, td_set_params, td_pulse_param
• td_get_content, td_set_content, td_custom_parameters

4) Diagnostics + Capture

Use for proving behavior instead of assuming behavior.

• td_screenshot, td_chop_data, td_geometry_data, td_pop_inspect
• td_cooking_info, td_get_errors
• td_exec_python, td_python_help, td_python_classes

Structured exec note: td_exec_python now returns JSON-safe result, result_type, and result_is_structured fields. Use it for lightweight structured probes before reaching for stdout parsing.

5) Events + Streaming

Use for reactive and continuous workflows.

• td_subscribe, td_unsubscribe, td_get_events
• td_capture_and_analyze
• td_monitor_visual, td_stop_monitor_visual
• td_stream_top, td_stop_stream_top

Token guidance: start with include_image=false for monitors/streams. Use image payloads only when visual detail is explicitly required. Prefer td_screenshot for single checks.

6) Optimization + Dynamics

Use for quality passes and temporal behavior analysis.

• td_optimize_visual — now accepts direct objective_weights (e.g. {"stability": 0.8, "complexity": 0.2})
• td_describe_dynamics

7) Safety + Recovery

Use for guardrails, emergency control, and rollback confidence.

• td_set_param_bounds, td_clear_param_bounds
• td_detect_instability, td_emergency_stabilize
• td_snapshot_scene, td_list_snapshots, td_diff_snapshots, td_restore_snapshot
• td_get_state_vector, td_get_timescale_state

8) Technique Memory & User Knowledge Store

Two parallel persistence surfaces — technique memory for replayable network recipes, knowledge store (new in v1.5.3) for free-form markdown reference essays (prose + math).

Technique memory — learning, saving, and replaying reusable network patterns:

• td_memory_learn — Analyze a live network subtree and extract a portable recipe. Auto-detects complexity: small/medium networks get full recipes with all params and expressions; large networks get structure summaries + key params.
• td_memory_save — Persist a technique to the project or global library.
• td_memory_recall — Search the library by text query and/or tags. Returns summaries.
• td_memory_replay — Rebuild a saved technique in a new location. Creates nodes, sets parameters and expressions, wires connections.
• td_memory_list — List all saved techniques with optional filtering.
• td_memory_favorite — Mark techniques as favorites and rate them (0-5).
• td_memory_promote — Copy a project-level technique to the global library for use across all projects.
• td_memory_export — Export the technique library as a portable JSON object for sharing or backup.
• td_memory_import — Import techniques from an exported library (from td_memory_export).
• td_memory_preferences — Get/set user preferences (color palettes, default resolutions, naming conventions, etc.)

User knowledge store (new in v1.5.3) — free-form markdown reference essays for prose-with-math reference content (BZ reaction equations, feedback recipes, "why this approach works" essays):

• td_knowledge_save — Persist a markdown body with name/description/tags/source/notes. Project- or global-scoped. Body capped at 200 KB; split larger writeups into linked entries.
• td_knowledge_recall — Search by free-text query and/or tags across name/description/tags/source/notes. Optional full_text=true also reads bodies (slower but more thorough).
• td_knowledge_get — Fetch full markdown body + metadata for one entry by id.
• td_knowledge_list — List entry summaries newest-first with optional filtering.

Storage lives at ~/.tdpilot/{memory,knowledge}/ with per-project and global scopes:

~/.tdpilot/
  memory/
    global/
      techniques.json
      preferences.json
    projects/{project_name}/
      techniques.json
      preferences.json
  knowledge/
    global/
      index.json
      entries/<uuid>.md
    projects/{project_name}/
      index.json
      entries/<uuid>.md

Starter technique recipes — the repo ships canonical live-visual recipes under data/techniques_starter/ (one td_memory_import-format JSON file per technique: plain feedback loop, feedback displacement bloom, audio-reactive level pulse, beat-detected flash, kaleidoscope mirror, noise-displace flow, particle GPU sprite trail, camera post chain, LFO param wobble, edge-glow composite). Load one with a single call — pass the file's parsed JSON as the data argument:

td_memory_import(data=<contents of data/techniques_starter/plain_feedback_loop.json>, scope="global")

Parameter names are verified against the operator atlas cards (a test enforces this), but every entry ships state: "candidate" with verified_on: null — they have not yet been replayed against a live TD build. Replay one with td_memory_replay, and promote it once it validates.

9. Macros & Planning (7)

Tool	Purpose
td_create_macro	Create a reusable macro from a template
td_list_macros	List available macros
td_get_macro_params	Get macro parameter schema
td_plan_patch	Plan a multi-step network patch
td_preflight_patch	Pre-validate a patch plan
td_validate_recipe	Validate a technique recipe
td_audit_project	Audit project subtree

10. Vision & Streaming (7)

Tool	Purpose
td_capture_frame	Capture a single frame from a TOP
td_analyze_frame	Analyze frame content (colors, regions)
td_monitor_visual	Start continuous visual monitoring
td_stop_monitor_visual	Stop visual monitoring
td_stream_top	Stream TOP output via WebSocket
td_stop_stream_top	Stop TOP streaming
td_optimize_visual	Get optimization suggestions for visuals

11. Knowledge Corpus (7)

Tool	Purpose
td_search_official_docs	Search official TD documentation
td_get_operator_doc	Get detailed operator documentation
td_get_param_help	Get parameter-level help
td_lookup_snippets	Find code snippets by topic
td_lookup_palette_component	Look up Palette component info
td_get_release_delta	Get changes between TD builds
td_get_build_compatibility	Check operator build compatibility

12. Server Introspection (3)

Tool	Purpose
td_get_capabilities	Report server capabilities
td_get_server_metrics	Get server performance metrics
td_describe_surface	Describe the full tool surface

13. Recommendations (3)

Tool	Purpose
td_recommend_official_component	Suggest official components
td_find_official_example	Find relevant official examples
td_explain_better_way	Suggest better approaches

14. TD 2025 Native (6)

Tool	Purpose
td_python_env_status	Inspect Python environment in TD
td_threading_status	Check threading configuration
td_logger_status	Inspect TD logger state
td_tdresources_inspect	Inspect TDResources categories
td_component_standardize	Audit/fix COMP standards
td_color_pipeline	Inspect color management pipeline

How To Use It (Practical Workflow)

1. Connect MCP client to TDPilot.
2. Ask for current project state.
3. Request a scoped patch goal.
4. Let agent apply changes in batches.
5. Require end-of-task td_get_errors check.
6. Save snapshot at stable milestone.
7. When you find something worth keeping: learn it, save it, rate it.

What It Is Good At

• Building and refactoring operator networks quickly.
• Inspecting modern POP systems with attribute-aware reads.
• Converting high-level creative goals into concrete TD graph operations.
• Audio-reactive/control-system patch scaffolding.
• Automated cleanup, relayout, and consistency passes.
• Diagnosing wiring/parameter/runtime errors with direct evidence.
• Remembering what works and reusing it across projects.

What It Is Not Good At

• Replacing artistic direction by itself.
• High-level show design without iterative user feedback.
• Unlimited always-on image streaming without token impact.
• Ignoring TD-specific context (operator families, cook behavior, timing model).
• "One shot perfect patch" generation in complex scenes.

Network Design Protocol (Default Aesthetic Rules)

When generating or reorganizing networks: use color coding by role, keep clean spacing and avoid overlaps, group nodes into functional clusters, preserve clear flow direction, name nodes by purpose, and run td_get_errors after edits.

Quick Setup

Recommended runtime (no manual Python setup in client config):

npx -y tdpilot

Local development runtime:

git clone https://github.com/dreamrec/TDPilot.git
cd TDPilot
uv sync
uv run tdpilot

TouchDesigner Side

Run the setup script once inside the TD Textport:

exec(open("/path/to/TDPilot/setup_mcp_in_td.py").read(), globals(), globals())

This installs the MCP component into /local/mcp_server by default, which means it persists across project opens within the same TD session. You only need to run this once — every project you open afterward will already have TDPilot available.

To install into a specific project instead: os.environ["TD_MCP_PARENT_PATH"] = "/project1" before running.

Alternatively, drag-and-drop td_component/tdpilot.tox into /local manually.

One-command setup helpers: macOS ./install.sh, Windows ./install.ps1

MCP Bundle (Standardized)

TDPilot ships a standard bundle in-repo:

• mcp/manifest.json
• mcp/profiles/claude-desktop.json, cursor.json, generic.json

Auto-generate client config:

tdpilot init --client claude-desktop
tdpilot init --client cursor --output ./cursor_mcp_config.json
tdpilot init --client generic --print-only

Doctor Command

Run a final environment/runtime check:

tdpilot doctor
tdpilot doctor --json

Environment Variables

• TD_MCP_HOST (default 127.0.0.1 — supports hostnames like desktop-3lurf0p.tail88651a.ts.net)
• TD_MCP_PORT (default 9981)
• TD_MCP_SCHEME (default http — set to https for Tailscale HTTPS or TLS-enabled setups)
• TD_MCP_WS_PORT (default 9982)
• TD_MCP_TRANSPORT (stdio or streamable_http)
• TD_MCP_HTTP_PORT (default 8765)
• TD_MCP_CAPTURE_QUALITY (default 0.3)
• TD_MCP_STREAM_MAX_FPS (default 15.0)
• TD_MCP_EXEC_MODE (off, restricted, standard, full)
• TDPILOT_PROJECT_NAME (set to enable per-project technique memory)
• TDPILOT_MEMORY_DIR (override default ~/.tdpilot/memory/ path)

Test Suite

Run the test suite:

uv run --extra dev pytest tests/ -v

Planning quality is measured, not asserted: the brain is gated by a 50+ case concept-to-node golden eval corpus (scripts/eval_brain_golden.py) plus schema-snapshot, docs-truth, and packaging-mirror checks — all in CI.

Reliability Habit

Treat this as mandatory for every meaningful task: before edits inspect, during edits take small reversible steps, after edits run td_get_errors, before risky changes snapshot.

Community

• Show what you built / ask anything — GitHub Discussions
• Share a working technique — the technique-share issue template turns your patch (with real parameter values + a screenshot) into bundled TDPilot knowledge, with credit
• Report a bug — issue templates (the bug form asks for tdpilot doctor --json)
• Contribute — CONTRIBUTING.md; techniques and hint packs land fastest
• Security — private disclosure via GHSA, see docs/SECURITY.md

License

MIT

┌─────────────────────────────────────────────────────────────────────┐
│ dreamrec // TDPilot // live laugh love                             │
└─────────────────────────────────────────────────────────────────────┘