Skip to content

refactor(agent-service): redesign sync-execution result and error model#7

Closed
bobbai00 wants to merge 1 commit into
refactor/agent-service-foundationfrom
refactor/agent-service-execution-model
Closed

refactor(agent-service): redesign sync-execution result and error model#7
bobbai00 wants to merge 1 commit into
refactor/agent-service-foundationfrom
refactor/agent-service-execution-model

Conversation

@bobbai00

Copy link
Copy Markdown
Owner

What changes were proposed in this PR?

Stacked on apache#5751 (foundation). This is the second of the split: it carries the sync-execution result/error model redesign (no overlap with the foundation PR — types/execution.ts is untouched there).

  • Replace the flat OperatorInfo with OperatorExecutionSummary (orthogonal sub-summaries: state, errorMessages, resultSummary?, consoleLogsSummary?); rename SyncExecutionResultWorkflowExecutionSummary.
  • resultSummary.sampleTuples is now SampleRow[] ({ rowIndex, tuple }) instead of JSON rows with an embedded __row_index__; drop the table-shape types (the agent derives input-port shapes from the DAG).
  • Reuse the engine's WorkflowFatalError for per-operator errors (the same type the compiling service returns), replacing the bespoke OperatorError so compile and execution errors share one wire shape.
  • errorMessages / errors are non-optional (empty = none); drop compilationErrors; collapse the console-message types and derive warnings from WARNING:-titled messages.
  • The WS operatorResults payload carries the canonical OperatorExecutionSummary; the frontend maps it to its flat display type.

Touches the Scala producer (SyncExecutionResource), the agent-service consumers (result-formatting, workflow-execution-tools, workflow-result-state, server), and the frontend WS mapping. Representation/type-level; behavior preserved (input-port shape lines are now derived rather than explicitly rendered).

Any related issues, documentation, discussions?

Part of apache#5747. Depends on apache#5751 — review/merge that first. (This is a draft stacked on the foundation branch; it will be retargeted to main once apache#5751 lands.)

How was this PR tested?

  • bunx tsc --noEmit, bun test (121 pass / 0 fail), prettier --check in agent-service; sbt WorkflowExecutionService/compile for amber.
  • End-to-end on the full local stack with a Claude Haiku 4.5 agent: it built and executed a CSV workflow; /operator-results returned the new shape — resultSummary.sampleTuples: [{ rowIndex, tuple }], errorMessages: [] — and the agent rendered the rows correctly.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8 (1M context)

Restructure the per-operator summary the sync-execution backend returns and the
agent-service/frontend consume, for a leaner and consistent wire contract:

- Replace flat OperatorInfo with OperatorExecutionSummary: state, errorMessages,
  resultSummary?, consoleLogsSummary? (orthogonal sub-summaries; no shape stats).
- Rename SyncExecutionResult -> WorkflowExecutionSummary; drop compilationErrors
  (folded into errors). errors and per-op errorMessages are non-optional (empty
  means none).
- OperatorResultSummary.sampleTuples is now List[SampleRow] ({rowIndex, tuple})
  instead of a JSON array with an embedded __row_index__. Drop the table-shape
  types (TableShape/InputPortTableShape): the agent derives input-port shapes
  from the DAG + each upstream's output shape; output shape comes from the result
  summary.
- Reuse the engine's WorkflowFatalError for per-operator errors (the same type
  the compiling service returns for compilation errors), replacing the bespoke
  OperatorError so compile and execution errors share one wire shape.
- Collapse console messages onto one type; derive warnings from WARNING-titled
  messages rather than a separate field.
- Replace OperatorResultSummaryWs: the WS operatorResults payload now carries the
  canonical OperatorExecutionSummary; the frontend maps it to its flat display
  type (re-flattening sampleTuples to keep the display components unchanged).

Touches the Scala producer (SyncExecutionResource), the agent-service consumers
(result-formatting, workflow-execution-tools, workflow-result-state, server) and
the frontend WS mapping. Representation/type-level change; behavior preserved,
except input-port shape lines are now derived rather than explicitly rendered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qFkyrpTd5PrkNBPcBeo4Q
@github-actions

Copy link
Copy Markdown

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Ma77Ball
    You can notify them by mentioning @Ma77Ball in a comment.

@bobbai00

Copy link
Copy Markdown
Owner Author

Superseded by apache#5927 (opened against apache:main).

@bobbai00 bobbai00 closed this Jun 24, 2026
@github-actions

Copy link
Copy Markdown

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 5 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 2bae592 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 385 0.235 25,109/42,262/42,262 us 🔴 +25.4% / ⚪ within ±5%
🟢 bs=100 sw=10 sl=64 835 0.51 118,572/140,438/140,438 us 🟢 -7.7% / ⚪ within ±5%
bs=1000 sw=10 sl=64 949 0.58 1,057,487/1,096,278/1,096,278 us ⚪ within ±5% / ⚪ within ±5%
Baseline details

Latest main 2bae592 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 385 tuples/sec 416 tuples/sec n/a -7.5% n/a
bs=10 sw=10 sl=64 MB/s 0.235 MB/s 0.254 MB/s n/a -7.5% n/a
bs=10 sw=10 sl=64 p50 25,109 us 23,847 us n/a +5.3% n/a
bs=10 sw=10 sl=64 p95 42,262 us 33,714 us n/a +25.4% n/a
bs=10 sw=10 sl=64 p99 42,262 us 33,714 us n/a +25.4% n/a
bs=100 sw=10 sl=64 throughput 835 tuples/sec 849 tuples/sec n/a -1.6% n/a
bs=100 sw=10 sl=64 MB/s 0.51 MB/s 0.518 MB/s n/a -1.5% n/a
bs=100 sw=10 sl=64 p50 118,572 us 115,733 us n/a +2.5% n/a
bs=100 sw=10 sl=64 p95 140,438 us 152,184 us n/a -7.7% n/a
bs=100 sw=10 sl=64 p99 140,438 us 152,184 us n/a -7.7% n/a
bs=1000 sw=10 sl=64 throughput 949 tuples/sec 948 tuples/sec n/a +0.1% n/a
bs=1000 sw=10 sl=64 MB/s 0.58 MB/s 0.579 MB/s n/a +0.2% n/a
bs=1000 sw=10 sl=64 p50 1,057,487 us 1,053,859 us n/a +0.3% n/a
bs=1000 sw=10 sl=64 p95 1,096,278 us 1,088,189 us n/a +0.7% n/a
bs=1000 sw=10 sl=64 p99 1,096,278 us 1,088,189 us n/a +0.7% n/a
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,519.41,200,128000,385,0.235,25108.90,42261.57,42261.57
1,100,10,64,20,2395.00,2000,1280000,835,0.510,118572.45,140438.42,140438.42
2,1000,10,64,20,21064.07,20000,12800000,949,0.580,1057487.22,1096277.97,1096277.97

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant