test(workflow-operator): fill uncovered branches in Java/R/Python UDF descriptors#6085
Merged
aglinxinyuan merged 2 commits intoJul 4, 2026
Merged
Conversation
… descriptors Extends the existing JavaUDFOpDescSpec, RUDFOpDescSpec, and PythonUDFOpDescV2Spec to cover the Codecov-missed branches: workers>1 parallelizable getPhysicalOp path, custom inputPorts/outputPorts in operatorInfo and partitionRequirement, r-tuple language selection, trimmed pveName, and runtimeReconfiguration.
Contributor
Automated Reviewer SuggestionsBased on the
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends unit-test coverage in common/workflow-operator for the Java, R, and Python UDF operator descriptors, targeting previously uncovered branches (primarily workers > 1, custom port descriptors/partition requirements, environment-name trimming, and runtime reconfiguration) without changing production logic.
Changes:
- Add/extend specs to exercise
getPhysicalOpbranches for multi-worker configurations and language selection. - Add tests for custom
inputPorts/outputPortshandling (including dependencies) and partition-requirement mapping. - Add tests covering
runtimeReconfigurationreturning a new physical op with no state transfer, plus Python venv-name trimming behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/udf/java/JavaUDFOpDescSpec.scala | Adds tests for workers>1, custom ports, and runtime reconfiguration branches in Java UDF descriptor. |
| common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/udf/r/RUDFOpDescSpec.scala | Adds tests for r-tuple selection, custom port partition requirements, and runtime reconfiguration branches in R UDF descriptor. |
| common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/udf/python/PythonUDFOpDescV2Spec.scala | Adds tests for workers>1, trimmed venv name, custom ports/dependencies, partition requirements, and runtime reconfiguration branches in Python UDF V2 descriptor. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6085 +/- ##
============================================
+ Coverage 57.64% 57.76% +0.12%
- Complexity 3142 3161 +19
============================================
Files 1130 1130
Lines 43832 43832
Branches 4747 4747
============================================
+ Hits 25268 25321 +53
+ Misses 17122 17083 -39
+ Partials 1442 1428 -14
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 372 | 0.227 | 26,734/36,216/36,216 us | 🔴 +27.8% / 🔴 +136.3% |
| 🟢 | bs=100 sw=10 sl=64 | 804 | 0.491 | 124,103/136,699/136,699 us | 🟢 -8.9% / 🔴 +26.2% |
| ⚪ | bs=1000 sw=10 sl=64 | 913 | 0.557 | 1,097,864/1,122,013/1,122,013 us | ⚪ within ±5% / 🔴 +10.5% |
Baseline details
Latest main 3dd4c2b from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 372 tuples/sec | 428 tuples/sec | 772.08 tuples/sec | -13.1% | -51.8% |
| bs=10 sw=10 sl=64 | MB/s | 0.227 MB/s | 0.261 MB/s | 0.471 MB/s | -13.0% | -51.8% |
| bs=10 sw=10 sl=64 | p50 | 26,734 us | 20,926 us | 12,745 us | +27.8% | +109.8% |
| bs=10 sw=10 sl=64 | p95 | 36,216 us | 36,279 us | 15,330 us | -0.2% | +136.3% |
| bs=10 sw=10 sl=64 | p99 | 36,216 us | 36,279 us | 19,054 us | -0.2% | +90.1% |
| bs=100 sw=10 sl=64 | throughput | 804 tuples/sec | 807 tuples/sec | 982.64 tuples/sec | -0.4% | -18.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.491 MB/s | 0.492 MB/s | 0.6 MB/s | -0.2% | -18.1% |
| bs=100 sw=10 sl=64 | p50 | 124,103 us | 119,781 us | 101,961 us | +3.6% | +21.7% |
| bs=100 sw=10 sl=64 | p95 | 136,699 us | 150,131 us | 108,335 us | -8.9% | +26.2% |
| bs=100 sw=10 sl=64 | p99 | 136,699 us | 150,131 us | 114,379 us | -8.9% | +19.5% |
| bs=1000 sw=10 sl=64 | throughput | 913 tuples/sec | 918 tuples/sec | 1,013 tuples/sec | -0.5% | -9.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.557 MB/s | 0.561 MB/s | 0.618 MB/s | -0.7% | -9.9% |
| bs=1000 sw=10 sl=64 | p50 | 1,097,864 us | 1,094,214 us | 993,573 us | +0.3% | +10.5% |
| bs=1000 sw=10 sl=64 | p95 | 1,122,013 us | 1,128,893 us | 1,032,489 us | -0.6% | +8.7% |
| bs=1000 sw=10 sl=64 | p99 | 1,122,013 us | 1,128,893 us | 1,065,526 us | -0.6% | +5.3% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,538.31,200,128000,372,0.227,26733.85,36216.24,36216.24
1,100,10,64,20,2488.57,2000,1280000,804,0.491,124103.20,136698.52,136698.52
2,1000,10,64,20,21900.54,20000,12800000,913,0.557,1097864.20,1122013.19,1122013.19…c port tests Replace partitionRequirement = null with UnknownPartition() (a non-optional PartitionInfo) in the port-description test data and assert the resulting physical op's partitionRequirement mapping, so the test matches real operator configuration and won't mask null-handling issues.
Yicong-Huang
approved these changes
Jul 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Fill uncovered branches in the Java/R/Python UDF descriptors, selected from the Codecov report. No production-code changes — extends the existing specs (nothing duplicated).
JavaUDFOpDesc.scalagetPhysicalOpworkers>1 (parallelizable, one-to-many, suggested workers); custominputPorts/outputPortsinoperatorInfo+ partitionRequirement;runtimeReconfigurationRUDFOpDesc.scalar-tuplelanguage selection + workers>1 branch; custom-portoperatorInfo+ partitionRequirement;runtimeReconfigurationPythonUDFOpDescV2.scalapveName; custominputPorts/outputPorts(incl. dependencies);runtimeReconfigurationThe existing specs only exercised the default
workers=1/ null-inputPortspaths, so these are the untested branch sides.Any related issues, documentation, discussions?
Follow-up to the review feedback on #6043: prioritize tests that fill uncovered code paths.
How was this PR tested?
sbt "WorkflowOperator/testOnly *JavaUDFOpDescSpec *RUDFOpDescSpec *PythonUDFOpDescV2Spec"— all greensbt "WorkflowOperator/Test/scalafmtCheck"andsbt "WorkflowOperator/scalafixAll --check"— cleanWas this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8 [1M context])