Skip to content

Layer checking as products pass over edges#704

Open
beojan wants to merge 2 commits into
Framework-R-D:mainfrom
beojan:layer-checking
Open

Layer checking as products pass over edges#704
beojan wants to merge 2 commits into
Framework-R-D:mainfrom
beojan:layer-checking

Conversation

@beojan

@beojan beojan commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

As discussed, layer checking happens dynamically as products pass over edges. A product selector without a layer specification would (in theory) accept products from any layer and place outputs appropriately so the layer a product belongs to need not be constant between iterations.

Layer specifications remain mandatory in selectors for now. Ironically until now they weren't actually being checked at all except when matching to products from providers.


  • Code

    • Added graph access to declared node types (fold, observer, predicate, transform, unfold) by storing the owning tbb::flow::graph and exposing it via graph() const.
    • Updated producer lookup to return all matching producers instead of a single match.
    • Reworked edge construction to connect a consumer to multiple matching producers when applicable.
    • Added layer-checking flow nodes in products_consumer, so product layer validation now happens as products traverse edges rather than only during producer matching.
    • Adjusted producer ambiguity handling to provide richer disambiguation/error reporting using suffix, creator, and output type signals.
  • Tests

    • Added coverage for selecting products by layer only, verifying the new layer-based traversal behavior with a job-layer transform.
  • Behavior

    • Layer validation is now enforced dynamically during graph traversal, enabling more flexible product flow while preserving current selector requirements.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b9050ad8-480f-427d-aa92-e7ef39b43b9e

📥 Commits

Reviewing files that changed from the base of the PR and between 4e8e474 and fb71620.

📒 Files selected for processing (11)
  • phlex/core/declared_fold.hpp
  • phlex/core/declared_observer.hpp
  • phlex/core/declared_predicate.hpp
  • phlex/core/declared_transform.hpp
  • phlex/core/declared_unfold.hpp
  • phlex/core/make_computational_edges.cpp
  • phlex/core/producer_catalog.cpp
  • phlex/core/producer_catalog.hpp
  • phlex/core/products_consumer.cpp
  • phlex/core/products_consumer.hpp
  • test/product_selecting.cpp
📜 Recent review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: Analyze cpp with CodeQL
  • GitHub Check: build (gcc, none)
  • GitHub Check: clang-tidy-check
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{cpp,cc,cxx,h,hpp}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.{cpp,cc,cxx,h,hpp}: Use clang-format tool for all C++ code formatting (VS Code auto-formats on save); configuration defined in .clang-format with 100-character line limit and 2-space indentation
Follow clang-tidy recommendations defined in .clang-tidy

Files:

  • phlex/core/make_computational_edges.cpp
  • phlex/core/producer_catalog.hpp
  • test/product_selecting.cpp
  • phlex/core/declared_predicate.hpp
  • phlex/core/products_consumer.cpp
  • phlex/core/products_consumer.hpp
  • phlex/core/producer_catalog.cpp
  • phlex/core/declared_fold.hpp
  • phlex/core/declared_observer.hpp
  • phlex/core/declared_transform.hpp
  • phlex/core/declared_unfold.hpp
**/*.{hpp,cpp}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{hpp,cpp}: Use .hpp for header files, .cpp for implementation, and *_test.cpp for test files in C++
Enforce 100-character line limit and 2-space indentation in C++ code via .clang-format
Use QualifierAlignment: Right (east-const) style: int const x not const int x in C++
Use PointerAlignment: Left in C++ (pointer * attached to type, not variable name)
All C++ identifiers must use lower_case naming: namespaces, classes, structs, enums, functions, variables, parameters, members, and constants
Exception to C++ naming: template parameters use CamelCase
Exception to C++ naming: macros use UPPER_CASE
Private, protected, and constant members in C++ must have a trailing underscore (_), no trailing underscore on anything else
Use enum class preferred over plain enum in C++
Use std::shared_ptr for shared ownership, std::unique_ptr for exclusive ownership, raw pointers for non-owning references only in C++
Use functors with agent-noun pattern: ModelEvaluator evaluate_model(...) in C++
Apply .clang-tidy checks for bugprone, cert, clang-analyzer, concurrency, cppcoreguidelines, misc, modernize, performance, portability, and readability as defined in the .clang-tidy configuration file
Use phlex:: namespace for core code, phlex::experimental:: for experimental features in C++

Files:

  • phlex/core/make_computational_edges.cpp
  • phlex/core/producer_catalog.hpp
  • test/product_selecting.cpp
  • phlex/core/declared_predicate.hpp
  • phlex/core/products_consumer.cpp
  • phlex/core/products_consumer.hpp
  • phlex/core/producer_catalog.cpp
  • phlex/core/declared_fold.hpp
  • phlex/core/declared_observer.hpp
  • phlex/core/declared_transform.hpp
  • phlex/core/declared_unfold.hpp
**/*.hpp

📄 CodeRabbit inference engine (AGENTS.md)

Avoid boolean parameters in C++ interfaces; prefer enumerations instead

Files:

  • phlex/core/producer_catalog.hpp
  • phlex/core/declared_predicate.hpp
  • phlex/core/products_consumer.hpp
  • phlex/core/declared_fold.hpp
  • phlex/core/declared_observer.hpp
  • phlex/core/declared_transform.hpp
  • phlex/core/declared_unfold.hpp
🔇 Additional comments (12)
phlex/core/declared_fold.hpp (1)

117-118: LGTM!

Also applies to: 210-211

phlex/core/declared_observer.hpp (1)

58-74: LGTM!

Also applies to: 111-112

phlex/core/declared_predicate.hpp (1)

80-81: LGTM!

Also applies to: 119-120

phlex/core/declared_transform.hpp (1)

99-100: LGTM!

Also applies to: 153-155

phlex/core/producer_catalog.cpp (1)

92-127: 🎯 Functional Correctness

Multi-candidate disambiguation logic is sound.

The three-map approach (suffixes, creators, types) correctly identifies when multiple candidates differ only by layer. The error reporting that logs which dimension(s) distinguish candidates before throwing is helpful for debugging. The static locals for port_to_node and deref_view are a reasonable micro-optimization — the generic lambda in deref_view correctly handles different value types across the three maps.

One observation: the disambiguation maps use hash values as keys. If two distinct suffixes/creators/types happen to share a hash, std::map::emplace silently drops the second entry, which could cause size() == 1 when it should be > 1. This is a very low-probability concern and the trade-off (avoiding copies of potentially expensive identifiers) is reasonable, but worth being aware of if you ever see a mysterious false-permit in production.

phlex/core/make_computational_edges.cpp (1)

150-159: LGTM!

phlex/core/declared_unfold.hpp (1)

120-121: LGTM!

Also applies to: 204-205

phlex/core/products_consumer.hpp (2)

49-52: LGTM!


21-22: 🩺 Stability & Availability

1UZ is fine here. The project targets C++23 (CMAKE_CXX_STANDARD=23), so this literal is compatible.

			> Likely an incorrect or invalid review comment.
test/product_selecting.cpp (1)

102-120: LGTM!

phlex/core/producer_catalog.hpp (1)

31-31: LGTM!

phlex/core/products_consumer.cpp (1)

33-46: 🩺 Stability & Availability

No dangling reference hereproduct_selector::layer owns an experimental::identifier, and its conversion operator returns experimental::identifier const&. The capture binds to the selector’s stored identifier, and input_product comes from input_products_, so the lambda doesn’t end up holding a dangling reference.

			> Likely an incorrect or invalid review comment.

📝 Walkthrough

Walkthrough

Declared nodes now expose their associated TBB flow graph. Producer resolution returns multiple matches with ambiguity checks, computational edges connect all matches, and consumer ports filter messages by layer. A product-selection test covers layer-only selection.

Changes

Product selection flow

Layer / File(s) Summary
Graph-aware layer filtering
phlex/core/declared_*.hpp, phlex/core/products_consumer.*, test/product_selecting.cpp
Declared nodes retain and expose their TBB graph; products_consumer adds layer-check nodes that forward matching messages, and the test validates job-layer-only selection.
Multi-producer edge resolution
phlex/core/producer_catalog.*, phlex/core/make_computational_edges.cpp
Producer lookup returns matching vectors, applies ambiguity checks, and creates edges from every matching producer to the consumer input.

Estimated code review effort: 4 (Complex) | ~45 minutes

Suggested reviewers: marcpaterno

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: validating layers dynamically as products traverse edges.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@beojan beojan requested review from knoepfel and marcpaterno July 9, 2026 22:11
@greenc-FNAL

Copy link
Copy Markdown
Contributor

21 fixed, 0 new since branch point (4e8e474)
21 fixed, 0 new since previous report on PR (c4e4a06)

✅ 21 CodeQL alerts resolved since the previous PR commit

  • Warning # 196 actions/untrusted-checkout-toctou/critical at .github/workflows/clang-tidy-fix.yaml:109:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 227 actions/untrusted-checkout-toctou/high at .github/workflows/clang-format-fix.yaml:94:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 228 actions/untrusted-checkout-toctou/high at .github/workflows/python-fix.yaml:94:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 229 actions/untrusted-checkout/high at .github/workflows/clang-format-fix.yaml:94:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 230 actions/untrusted-checkout/high at .github/workflows/python-fix.yaml:94:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 231 actions/untrusted-checkout-toctou/high at .github/workflows/cmake-format-fix.yaml:94:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 232 actions/untrusted-checkout-toctou/high at .github/workflows/jsonnet-format-fix.yaml:95:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 233 actions/untrusted-checkout/high at .github/workflows/cmake-format-fix.yaml:94:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 234 actions/untrusted-checkout/high at .github/workflows/jsonnet-format-fix.yaml:95:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 235 actions/untrusted-checkout/medium at .github/workflows/clang-format-check.yaml:82:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 236 actions/untrusted-checkout/medium at .github/workflows/actionlint-check.yaml:86:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 237 actions/untrusted-checkout/medium at .github/workflows/clang-tidy-check.yaml:59:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 238 actions/untrusted-checkout/medium at .github/workflows/cmake-format-check.yaml:79:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 239 actions/untrusted-checkout/medium at .github/workflows/cmake-build.yaml:159:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 240 actions/untrusted-checkout/medium at .github/workflows/header-guards-check.yaml:82:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 241 actions/untrusted-checkout/medium at .github/workflows/jsonnet-format-check.yaml:79:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 242 actions/untrusted-checkout/medium at .github/workflows/markdown-check.yaml:82:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 243 actions/untrusted-checkout/medium at .github/workflows/python-check.yaml:84:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 244 actions/untrusted-checkout/medium at .github/workflows/yaml-check.yaml:76:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 245 actions/untrusted-checkout-toctou/high at .github/workflows/coverage.yaml:386:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • ✅ …and 1 more alerts (see Code Scanning for the full list).

✅ 21 CodeQL alerts resolved since the branch point

  • Warning # 196 actions/untrusted-checkout-toctou/critical at .github/workflows/clang-tidy-fix.yaml:109:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 227 actions/untrusted-checkout-toctou/high at .github/workflows/clang-format-fix.yaml:94:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 228 actions/untrusted-checkout-toctou/high at .github/workflows/python-fix.yaml:94:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 229 actions/untrusted-checkout/high at .github/workflows/clang-format-fix.yaml:94:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 230 actions/untrusted-checkout/high at .github/workflows/python-fix.yaml:94:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 231 actions/untrusted-checkout-toctou/high at .github/workflows/cmake-format-fix.yaml:94:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 232 actions/untrusted-checkout-toctou/high at .github/workflows/jsonnet-format-fix.yaml:95:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
    Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • Warning # 233 actions/untrusted-checkout/high at .github/workflows/cmake-format-fix.yaml:94:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 234 actions/untrusted-checkout/high at .github/workflows/jsonnet-format-fix.yaml:95:9 — Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
    Checkout of untrusted code in a privileged workflow with later potential execution (event trigger: issue_comment).
  • Warning # 235 actions/untrusted-checkout/medium at .github/workflows/clang-format-check.yaml:82:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 236 actions/untrusted-checkout/medium at .github/workflows/actionlint-check.yaml:86:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 237 actions/untrusted-checkout/medium at .github/workflows/clang-tidy-check.yaml:59:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 238 actions/untrusted-checkout/medium at .github/workflows/cmake-format-check.yaml:79:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 239 actions/untrusted-checkout/medium at .github/workflows/cmake-build.yaml:159:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 240 actions/untrusted-checkout/medium at .github/workflows/header-guards-check.yaml:82:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 241 actions/untrusted-checkout/medium at .github/workflows/jsonnet-format-check.yaml:79:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 242 actions/untrusted-checkout/medium at .github/workflows/markdown-check.yaml:82:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 243 actions/untrusted-checkout/medium at .github/workflows/python-check.yaml:84:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 244 actions/untrusted-checkout/medium at .github/workflows/yaml-check.yaml:76:9 — Potential unsafe checkout of untrusted pull request on non-privileged workflow.
  • Warning # 245 actions/untrusted-checkout-toctou/high at .github/workflows/coverage.yaml:386:9 — Insufficient protection against execution of untrusted code on a privileged workflow (issue_comment).
  • ✅ …and 1 more alerts (see Code Scanning for the full list).

Review the full CodeQL report for details.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Clang-Tidy Check Results

Found 5788 issue(s); none are newly introduced by this patch.

All issues by check:

  • readability-identifier-naming: 2132
  • readability-redundant-member-init: 1361
  • portability-template-virtual-member-function: 575
  • performance-unnecessary-value-param: 532
  • readability-avoid-const-params-in-decls: 278
  • modernize-pass-by-value: 215
  • readability-braces-around-statements: 123
  • modernize-use-designated-initializers: 93
  • readability-convert-member-functions-to-static: 85
  • readability-const-return-type: 50
  • modernize-use-auto: 31
  • performance-move-const-arg: 29
  • performance-enum-size: 24
  • readability-qualified-auto: 23
  • readability-redundant-control-flow: 23
  • modernize-use-using: 19
  • modernize-concat-nested-namespaces: 17
  • modernize-use-equals-default: 17
  • readability-math-missing-parentheses: 16
  • readability-function-size: 15
  • modernize-avoid-c-arrays: 15
  • readability-static-definition-in-anonymous-namespace: 14
  • bugprone-branch-clone: 12
  • modernize-return-braced-init-list: 10
  • readability-isolate-declaration: 8
  • modernize-use-starts-ends-with: 8
  • cppcoreguidelines-special-member-functions: 7
  • readability-redundant-access-specifiers: 7
  • modernize-use-ranges: 6
  • readability-inconsistent-declaration-parameter-name: 6
  • bugprone-throw-keyword-missing: 6
  • readability-container-contains: 5
  • readability-redundant-casting: 5
  • readability-container-size-empty: 5
  • modernize-use-std-numbers: 3
  • readability-else-after-return: 2
  • modernize-use-integer-sign-comparison: 2
  • readability-use-anyofallof: 2
  • cppcoreguidelines-rvalue-reference-param-not-moved: 1
  • readability-redundant-string-init: 1
  • modernize-redundant-void-arg: 1
  • modernize-make-shared: 1
  • modernize-use-emplace: 1
  • readability-simplify-boolean-expr: 1
  • performance-avoid-endl: 1

See inline comments for details. Comment @phlexbot tidy-fix to auto-fix.

@beojan

beojan commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb71620bf9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +157 to +158
for (auto* producer : producer_ports) {
make_edge(*producer->output_port, *receiver_port);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve layer checks for filtered consumers

When a consumer has experimental_when, receiver_port is the filter's collector, so this loop connects every matching producer directly to the filter and never calls node->port(query). The new layer-checking node is only created inside products_consumer::port(), while filter forwards to consumer.ports() (the raw downstream ports), so a filtered node whose selector asks for job can still receive matching event producer messages whenever the predicate accepts those message ids.

Useful? React with 👍 / 👎.

if (suffixes.size() == 1 && creators.size() == 1 && types.size() == 1) {
spdlog::info(msg);
spdlog::info("This is permitted -- layers may differ");
return candidates;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep same-layer duplicate producers ambiguous

This branch permits multiple producers solely because suffix, creator, and type are identical, but the catalog has not checked that the candidates actually differ by layer. In configurations like a transform and a fold registered with the same name and suffix that both emit on the selector's layer, all candidates are returned and connected; the downstream layer checker accepts both, causing duplicate executions instead of the previous ambiguity error.

Useful? React with 👍 / 👎.

Comment on lines +151 to 153
if (producer_ports.empty()) {
// Is there a way to detect mis-specified product dependencies?
result[node_name].push_back({query, receiver_port});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall back when only wrong-layer producers match

Because find_producers() is not layer-aware, a producer on a different layer still makes this branch treat the input as satisfied. In a graph with, for example, an event transform producing suffix number and a job provider for the same suffix/type, a job selector reaches this path, the provider lookup is skipped, and the new layer checker drops the event messages, leaving the consumer with no input instead of using the matching provider or reporting the product as unresolved.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants