Skip to content

Add stable QUBO C ABI wrapper - #10

Merged
bernalde merged 5 commits into
masterfrom
fix/issue-9-c-abi
May 27, 2026
Merged

Add stable QUBO C ABI wrapper#10
bernalde merged 5 commits into
masterfrom
fix/issue-9-c-abi

Conversation

@bernalde

Copy link
Copy Markdown
Member

Summary

  • Adds a documented C ABI for solving QUBO instances through upstream MQLib internals without invoking the CLI executable.
  • Adds a C++ wrapper implementation that accepts caller-owned plain arrays, validates ABI inputs, runs explicit heuristics or the hyperheuristic path, and returns stable status codes, objective, solution, runtime, selected heuristic, and optional history data.
  • Adds a small native C example plus Julia tests for the ABI contract and optional upstream-backed syntax checking.

Tests run

  • cc -std=c99 -Ic_api/include -fsyntax-only c_api/examples/solve_qubo.c - passed.
  • c++ -std=c++11 -Ic_api/include -I/tmp/MQLib-upstream/include -fsyntax-only c_api/src/mqlib_c_api.cpp - passed.
  • c++ -std=c++11 -fPIC -Ic_api/include -I/tmp/MQLib-upstream/include $(find /tmp/MQLib-upstream/src -name '*.cpp' ! -name main.cpp) c_api/src/mqlib_c_api.cpp -shared -o /tmp/mqlib-c-api-build/libmqlib_c_api.so -lm - passed.
  • cc -std=c99 -Ic_api/include c_api/examples/solve_qubo.c -L/tmp/mqlib-c-api-build -lmqlib_c_api -Wl,-rpath,/tmp/mqlib-c-api-build -o /tmp/mqlib-c-api-build/solve_qubo - passed.
  • /tmp/mqlib-c-api-build/solve_qubo - passed:
    • ALKHAMIS1998 objective 6 solution 1 0 1 runtime 0.010025 seconds
    • HH_BEASLEY1998TS objective 0 solution 0 0 0 runtime 0.507517 seconds
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 9/9.
  • julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 8/8; upstream source syntax check skipped because MQLIB_UPSTREAM_DIR was not set.

Notes

  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. -e 'import Pkg; Pkg.test()' did not execute tests locally because the current manifest fixes local QUBOTools at 0.12.0 while Project.toml restricts QUBOTools to 0.10.1, 0.11.
  • The Julia wrapper still uses the executable path. Using this ABI from Julia is the follow-up shared-library integration work.

Closes #9

@bernalde bernalde left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review found two blocking issues. I attempted to submit this as REQUEST_CHANGES, but GitHub rejects request-changes reviews from the PR author. The inline comments below should be treated as blocking before merge.

Comment thread c_api/src/mqlib_c_api.cpp Outdated
} else {
mi.reset(new MaxCutInstance(qi));
std::string hyperheuristic_choice;
maxcut_heuristic.reset(new MaxCutHyperheuristic(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: The hyperheuristic path depends on upstream MaxCutHyperheuristic finding hhdata/*.rf relative to the process CWD, but this wrapper neither provides nor validates that data path. Running the new example from the package checkout instead of the upstream MQLib checkout returns MQLIB_STATUS_OK with selected_heuristic == "HH_", objective 0, and the all-zero solution, so shared-library consumers get a successful no-op whenever their CWD lacks hhdata. This breaks the issue requirement that the C ABI can run the hyperheuristic path. Please make the model directory explicit or robust for the shared library, and return a non-OK status if no hyperheuristic model is selected; add a regression test that runs from a CWD without hhdata.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in f175e52. The wrapper now selects hyperheuristic models from MQLibCQUBOInput.hyperheuristic_data_dir, returns MQLIB_STATUS_HYPERHEURISTIC_DATA_NOT_FOUND when no model files are found, and test/c_api_smoke.c covers running from a CWD without hhdata.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in f175e52: hyperheuristic_data_dir makes the model directory explicit, missing model data returns MQLIB_STATUS_HYPERHEURISTIC_DATA_NOT_FOUND, and the smoke test runs from a CWD without hhdata. The follow-up commit 8c1249e keeps that coverage passing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed previously in f175e52: the model directory is explicit via hyperheuristic_data_dir, missing model data returns MQLIB_STATUS_HYPERHEURISTIC_DATA_NOT_FOUND, and the smoke test runs from a CWD without hhdata. No additional change was needed for this thread in a446018.

Comment thread test/runtests.jl

upstream_dir = get(ENV, "MQLIB_UPSTREAM_DIR", "")
cxx = first_available_tool(["c++", "g++", "clang++"])
if isempty(upstream_dir) || !isdir(joinpath(upstream_dir, "include"))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: In normal CI MQLIB_UPSTREAM_DIR is not set, so this branch skips the only check that compiles c_api/src/mqlib_c_api.cpp; it also never links or runs c_api/examples/solve_qubo.c. The changed behavior is the native ABI implementation, but future C++ breakage, link errors, and runtime bugs still pass the default suite. Please add a CI-covered native smoke test/build target that compiles the wrapper against upstream MQLib sources and runs at least the explicit heuristic and hyperheuristic example, or otherwise fail when the implementation cannot be checked.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in f175e52. Added test/native_c_api_smoke.sh plus test/c_api_smoke.c, and wired the smoke target into Linux CI so the wrapper is compiled, linked, and run for explicit heuristic, hyperheuristic, and missing-hhdata cases.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in f175e52: Linux CI now builds and runs the native C ABI smoke test for explicit heuristic, hyperheuristic, and missing-hhdata cases. The follow-up commit 8c1249e keeps that path and pins the upstream dependency used by the smoke test.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed previously in f175e52 and kept covered by later commits: Linux CI builds and runs the native C ABI smoke test for explicit heuristic, hyperheuristic, and missing-hhdata cases. No additional change was needed for this thread in a446018.

@bernalde

Copy link
Copy Markdown
Member Author

Pushed follow-up commit f175e52 (Address C ABI review feedback (#9)).

Main changes:

  • Replaced the hyperheuristic wrapper's implicit CWD-dependent hhdata lookup with explicit model selection from MQLibCQUBOInput.hyperheuristic_data_dir.
  • Added MQLIB_STATUS_HYPERHEURISTIC_DATA_NOT_FOUND so missing hyperheuristic model data returns a non-OK status instead of HH_ with a no-op result.
  • Preserved upstream hyperheuristic behavior for QUBO-selected models by running them on the Max-Cut-derived QUBO and converting the result back to the original QUBO solution shape.
  • Added test/native_c_api_smoke.sh and test/c_api_smoke.c, and wired the native smoke test into Linux CI. The smoke test builds the shared library from upstream MQLib sources, runs the C example, checks explicit heuristic success, checks hyperheuristic success with an explicit hhdata path, and checks missing data returns the new status.
  • Updated the C example and C ABI docs for the explicit hhdata argument.

Tests run:

  • cc -std=c99 -Ic_api/include -fsyntax-only c_api/examples/solve_qubo.c - passed.
  • cc -std=c99 -Ic_api/include -fsyntax-only test/c_api_smoke.c - passed.
  • c++ -std=c++11 -Ic_api/include -I/tmp/MQLib-upstream/include -fsyntax-only c_api/src/mqlib_c_api.cpp - passed.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream bash test/native_c_api_smoke.sh - passed.
  • julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 9/9.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 10/10.
  • julia +1.12 --project=. -e 'import Pkg; Pkg.test()' - still fails before tests due the existing manifest/compat conflict: local QUBOTools is fixed at 0.12.0 while Project.toml allows 0.10.1, 0.11.
  • PR checks after push: Ubuntu and Windows CI passed.

Comments intentionally not addressed:

  • None.

Remaining risks / follow-up:

  • The native smoke test is CI-covered on Linux. Windows shared-library packaging remains part of the follow-up MQLib_jll work.

Comment thread c_api/src/mqlib_c_api.cpp Outdated
return MQLIB_STATUS_INTERNAL_ERROR;
}

result->runtime_seconds = heuristic->Runtime();

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: This reports only the selected heuristic runtime. In the hyperheuristic path, graph metric computation and random-forest model selection happen before the selected heuristic is constructed, so heuristic->Runtime() omits selection overhead. That diverges from the upstream hyperheuristic wrapper/CLI semantics and underreports the C ABI runtime_seconds for HH_* runs. Please time the full hyperheuristic path, or explicitly document that the ABI reports selected-heuristic runtime only, and add a focused test for the intended semantics.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8c1249e: hyperheuristic runtime_seconds now measures model selection plus the selected heuristic run, and test/c_api_smoke.c checks that the reported runtime does not omit elapsed hyperheuristic work.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8c1249e and refined in a446018: runtime_seconds now reports total hyperheuristic elapsed time, including model selection and the selected heuristic run.

UPSTREAM_DIR="$MQLIB_UPSTREAM_DIR"
else
UPSTREAM_DIR="$BUILD_DIR/MQLib-upstream"
git clone --depth 1 https://github.com/MQLib/MQLib.git "$UPSTREAM_DIR"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocking: This clones upstream MQLib at the current default branch. That keeps smoke coverage fresh, but it makes CI nondeterministic: a future upstream change can break this PR without any local change. Consider adding an optional MQLIB_UPSTREAM_REF and defaulting CI to a known vetted commit or tag, while retaining HEAD as an opt-in local mode.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8c1249e: the smoke script now defaults to upstream MQLib commit 585496274af5abb0849d0d47e135496b4688680b, CI passes that ref explicitly, and MQLIB_UPSTREAM_REF=HEAD remains available for opt-in current-upstream testing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed previously in 8c1249e: the smoke script defaults to upstream MQLib commit 585496274af5abb0849d0d47e135496b4688680b, CI passes that ref explicitly, and MQLIB_UPSTREAM_REF=HEAD remains available for opt-in testing. No additional change was needed for this thread in a446018.

@bernalde

Copy link
Copy Markdown
Member Author

Pushed follow-up commit 8c1249e (Address native C ABI review follow-ups (#9)).

Main changes:

  • Hyperheuristic runtime_seconds now measures graph metric computation, random-forest model selection, and the selected heuristic run, matching the upstream wrapper/CLI semantics more closely.
  • test/c_api_smoke.c now checks that hyperheuristic runtime is not materially shorter than elapsed work in the smoke process.
  • test/native_c_api_smoke.sh now defaults to vetted upstream MQLib commit 585496274af5abb0849d0d47e135496b4688680b, while MQLIB_UPSTREAM_REF=HEAD remains available for opt-in testing against current upstream.
  • Linux CI now passes the same pinned MQLIB_UPSTREAM_REF to the native smoke test.

Tests run:

  • bash test/native_c_api_smoke.sh - passed against pinned upstream 5854962; hyperheuristic runtime reported 0.522406 seconds.
  • MQLIB_UPSTREAM_REF=HEAD bash test/native_c_api_smoke.sh - passed; hyperheuristic runtime reported 0.526374 seconds.
  • julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 9/9.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 10/10.
  • git diff --check - passed.

Tests not passing locally:

  • julia +1.12 --project=. -e 'import Pkg; Pkg.test()' still fails before running tests because local QUBOTools is fixed at 0.12.0 while MQLib compat restricts it to 0.10.1 - 0.11. This is the existing resolver issue noted earlier, not introduced by this follow-up.

Comments intentionally not addressed:

  • None. The two older unresolved threads were already addressed by f175e52; this commit addresses the two newer threads.

Remaining risks / follow-up:

  • Native C ABI execution remains CI-covered on Linux only. Windows shared-library packaging remains follow-up work for MQLib_jll.

Comment thread c_api/src/mqlib_c_api.cpp Outdated
maxcut_heuristic.reset(factory.RunMaxCutHeuristic(
choice.code,
*mi,
input->runtime_limit_seconds,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: The selected heuristic still receives the full original runtime limit after hyperheuristic model selection has already consumed time. Upstream MaxCutHyperheuristic starts its timer before metrics/model selection and routes the selected heuristic through callbacks, so the runtime limit applies to the whole hyperheuristic run. Here result->runtime_seconds includes selection elapsed, but this branch and the QUBO branch below can run for an additional full budget. In a local targeted C ABI harness with runtime_limit_seconds = 0.75, the call returned runtime=1.273744. Please compute the remaining budget after select_hyperheuristic and pass that to the selected heuristic, and add a native smoke/contract test with a moderate hyperheuristic limit that would fail if total runtime can exceed the requested budget by the selection time. If history buffers are populated for the hyperheuristic path, also keep their times on the same total-run timeline.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in a446018: after model selection, the wrapper computes the remaining runtime budget and passes that to the selected heuristic; copied hyperheuristic history times are shifted onto the total-run timeline. test/c_api_smoke.c now includes a 0.75-second hyperheuristic budget regression case.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up in aeb2e5c: the budget regression still checks total-budget behavior, but now uses 1.50 seconds so slower CI runners have enough post-selection solve time to produce improvement history.

@bernalde bernalde left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review found one blocking correctness issue in the hyperheuristic runtime-budget semantics. I attempted to submit this as REQUEST_CHANGES, but GitHub rejects request-changes reviews from the PR author, so the inline comment should be treated as blocking before merge.

Blocking issues:

  • The C ABI hyperheuristic path reports total runtime but still gives the selected heuristic the full original runtime limit after selection has already consumed time.

Nonblocking issues:

  • None.

Questions:

  • None.

Tests run:

  • bash test/native_c_api_smoke.sh - passed against pinned upstream MQLib 5854962.
  • julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 9/9.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 10/10.
  • git diff --check origin/master...HEAD - passed.
  • Temporary targeted native harness with runtime_limit_seconds = 0.75 - reproduced the budget issue: returned runtime=1.273744.
  • gh pr checks 10 - Ubuntu and Windows checks passed.

Not passing locally:

  • julia +1.12 --project=. -e 'import Pkg; Pkg.test()' fails before tests because local QUBOTools is fixed at 0.12.0 while MQLib compat restricts it to 0.10.1 - 0.11; this is the existing resolver issue.

I would not merge this until the blocking issue above is addressed.

@bernalde

Copy link
Copy Markdown
Member Author

Pushed follow-up commit a446018 (Fix hyperheuristic runtime budget (#9)).

Main changes:

  • Hyperheuristic selection now computes the elapsed model-selection time and passes only the remaining runtime budget to the selected Max-Cut or QUBO heuristic.
  • Hyperheuristic runtime_seconds still reports the total elapsed hyperheuristic run, including model selection and selected heuristic execution.
  • Hyperheuristic history times are shifted onto the total-run timeline for entries produced by the selected heuristic, preserving the initial zero-time incumbent entry.
  • test/c_api_smoke.c now includes a moderate-budget hyperheuristic regression case that fails if total runtime can exceed the requested budget by the model-selection time, and checks that copied history times include selection elapsed time.
  • c_api/examples/solve_qubo.c now gives the hyperheuristic example a moderate runtime budget so it demonstrates an actual solve under the corrected total-budget semantics.

Tests run:

  • bash test/native_c_api_smoke.sh - passed against pinned upstream 5854962; hyperheuristic example reported runtime 0.750055 seconds.
  • MQLIB_UPSTREAM_REF=HEAD bash test/native_c_api_smoke.sh - passed; hyperheuristic example reported runtime 0.750046 seconds.
  • julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 9/9.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 10/10.
  • git diff --check - passed.

Tests not passing locally:

  • julia +1.12 --project=. -e 'import Pkg; Pkg.test()' still fails before running tests because local QUBOTools is fixed at 0.12.0 while MQLib compat restricts it to 0.10.1 - 0.11. This is the existing resolver issue noted earlier, not introduced by this follow-up.

Comments intentionally not addressed:

  • None. The older unresolved inline threads were already addressed by f175e52 and 8c1249e; this commit addresses the new runtime-budget thread.

Remaining risks / follow-up:

  • Native C ABI execution remains CI-covered on Linux only. Windows shared-library packaging remains follow-up work for MQLib_jll.

@bernalde

Copy link
Copy Markdown
Member Author

Pushed follow-up commit aeb2e5c (Harden hyperheuristic smoke budget (#9)).

Main changes:

  • Raised the hyperheuristic regression smoke budget from 0.75 to 1.50 seconds in test/c_api_smoke.c.
  • Updated the C example to use the same moderate hyperheuristic budget.
  • This keeps the regression meaningful while allowing slower CI runners enough post-selection solve time to produce improvement history.

Tests run:

  • bash test/native_c_api_smoke.sh - passed against pinned upstream 5854962; hyperheuristic example reported runtime 1.500054 seconds.
  • julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 9/9.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl - passed: QUBODrivers 124/124, C ABI contract 10/10.
  • git diff --check - passed.

Comments intentionally not addressed:

  • None. This is a robustness adjustment to the test added for the runtime-budget review comment.

Remaining risks / follow-up:

  • Native C ABI execution remains CI-covered on Linux only. Windows shared-library packaging remains follow-up work for MQLib_jll.

@bernalde bernalde left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review outcome: merge-ready. I attempted to submit APPROVE, but GitHub rejected approval from the PR author account.

Blocking issues: None.

Nonblocking issues: None.

Questions: None.

Tests run and outcomes:

  • git diff --check origin/master...HEAD: passed.
  • bash test/native_c_api_smoke.sh: passed against pinned upstream MQLib 5854962 after rerunning with network access; initial sandboxed attempt failed only because github.com was not resolvable.
  • julia +1.12 --project=. test/runtests.jl: passed; QUBODrivers 124/124, C ABI contract 9/9.
  • MQLIB_UPSTREAM_DIR=/tmp/MQLib-upstream julia +1.12 --project=. test/runtests.jl: passed; QUBODrivers 124/124, C ABI contract 10/10.
  • MQLIB_UPSTREAM_REF=HEAD bash test/native_c_api_smoke.sh: passed; current upstream HEAD resolved to 5854962.
  • make bin/MQLib in /tmp/MQLib-upstream: passed; upstream warnings only.
  • julia +1.12 --project=. -e 'import Pkg; Pkg.test()': did not run tests because the local environment has QUBOTools fixed at 0.12.0 while MQLib compat allows 0.10.1 - 0.11.
  • gh pr checks 10: Ubuntu and Windows CI passed.

Merge assessment: I found no blocking issues and no actionable inline comments. The PR is merge-ready as-is. The remaining Linux-only native ABI execution coverage is acceptable here because Windows shared-library packaging is the MQLib_jll follow-up.

@bernalde
bernalde merged commit 8b49187 into master May 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a stable C ABI for solving QUBOs with MQLib

1 participant