Skip to content

Add ScalarUDF preserves_lex_ordering FFI support#23069

Open
Amogh-2404 wants to merge 1 commit into
apache:mainfrom
Amogh-2404:fix/issue-22330-ffi-preserves-lex-ordering
Open

Add ScalarUDF preserves_lex_ordering FFI support#23069
Amogh-2404 wants to merge 1 commit into
apache:mainfrom
Amogh-2404:fix/issue-22330-ffi-preserves-lex-ordering

Conversation

@Amogh-2404

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

FFI_ScalarUDF did not expose ScalarUDFImpl::preserves_lex_ordering, so a producer override was lost when the UDF crossed the FFI boundary.

What changes are included in this PR?

  • Adds preserves_lex_ordering to FFI_ScalarUDF.
  • Converts ExprProperties through the existing FFI representation.
  • Adds unit and cross-library integration coverage for the foreign UDF path.

Are these changes tested?

  • cargo fmt --all
  • cargo test -p datafusion-ffi
  • cargo test -p datafusion-ffi --features integration-tests
  • cargo clippy -p datafusion-ffi --all-targets --all-features -- -D warnings
  • Pre-push gate, including workspace clippy and sqllogictests

Are there any user-facing changes?

No end-user behavior change. This updates the DataFusion FFI ABI.

@Amogh-2404

Copy link
Copy Markdown
Contributor Author

This changes the FFI_ScalarUDF layout, so it needs the api change label.

@github-actions github-actions Bot added the ffi Changes to the ffi crate label Jun 21, 2026
@github-actions

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details
     Cloning apache/main
    Building datafusion-ffi v54.0.0 (current)
       Built [  38.162s] (current)
     Parsing datafusion-ffi v54.0.0 (current)
      Parsed [   0.043s] (current)
    Building datafusion-ffi v54.0.0 (baseline)
       Built [  38.704s] (baseline)
     Parsing datafusion-ffi v54.0.0 (baseline)
      Parsed [   0.070s] (baseline)
    Checking datafusion-ffi v54.0.0 -> v54.0.0 (no change; assume patch)
     Checked [   0.180s] 223 checks: 221 pass, 1 fail, 1 warn, 30 skip

--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field ForeignLibraryModule.create_lex_ordering_udf in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:95
  field FFI_ScalarUDF.preserves_lex_ordering in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/udf/mod.rs:106

--- warning repr_c_plain_struct_fields_reordered: struct fields reordered in repr(C) struct ---

Description:
A public repr(C) struct had its fields reordered. This can change the struct's memory layout, possibly breaking FFI use cases that depend on field position and order.
        ref: https://doc.rust-lang.org/reference/type-layout.html#reprc-structs
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/repr_c_plain_struct_fields_reordered.ron

Failed in:
  ForeignLibraryModule.create_table_function moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:97
  ForeignLibraryModule.create_sum_udaf moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:101
  ForeignLibraryModule.create_stddev_udaf moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:104
  ForeignLibraryModule.create_rank_udwf moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:106
  ForeignLibraryModule.create_extension_options moved from position 13 to 14, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:109
  ForeignLibraryModule.create_empty_exec moved from position 14 to 15, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:111
  ForeignLibraryModule.create_exec_with_statistics moved from position 15 to 16, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:113
  ForeignLibraryModule.create_table_with_statistics moved from position 16 to 17, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:115
  ForeignLibraryModule.create_physical_optimizer_rule moved from position 17 to 18, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:118
  ForeignLibraryModule.create_context_aware_optimizer_rule moved from position 18 to 19, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:120
  ForeignLibraryModule.version moved from position 19 to 20, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:122
  FFI_ScalarUDF.clone moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/udf/mod.rs:113
  FFI_ScalarUDF.release moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/udf/mod.rs:116
  FFI_ScalarUDF.private_data moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/udf/mod.rs:120
  FFI_ScalarUDF.library_marker_id moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/udf/mod.rs:125

     Summary semver requires new major version: 1 major and 0 minor checks failed
     Warning produced 1 major and 0 minor level warnings
    Finished [  79.456s] datafusion-ffi

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto detected api change Auto detected API change ffi Changes to the ffi crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FFI: FFI_ScalarUDF silently drops producer overrides of defaulted trait methods

1 participant