Add QUBODrivers 0.6 benchmark conformance

[bernalde]

Refs #26.

Summary

• Tighten compat to QUBODrivers = "0.6" and QUBOTools = "0.13".
• Add QUBODrivers benchmark metadata for MQLib solves: algorithm, backend, reads, seeds, status, and time.effective.
• Honor QUBODrivers.FinalNumberOfReads() for emitted samples and declare seed, final-read, and time-limit behavior when the QUBODrivers benchmark trait API is available.
• Document seed/read/time-limit semantics and run QUBODrivers.test(...; benchmark_conformance = true) when the upgraded test API is present.

Tests

• julia --project=. -e 'using Pkg; Pkg.test()'
  • Passed with released dependencies: QUBODrivers v0.6.0, QUBOTools v0.13.0, MQLib_jll v0.1.2+0.
  • Result summaries: Compatibility metadata 6/6, QUBODrivers 124/124, Julia C ABI bridge 21/21, C ABI contract 21/21.
• julia --project=/tmp/mqlib-issue26-conformance -e 'using Pkg; Pkg.activate("/tmp/mqlib-issue26-conformance"; shared=false); Pkg.develop(path="/tmp/QUBODrivers-issue-50"); Pkg.develop(path=pwd()); Pkg.add("Test"); Pkg.test("MQLib")'
  • Passed with QUBODrivers conformance checkout /tmp/QUBODrivers-issue-50.
  • Result summaries: Compatibility metadata 6/6, QUBODrivers 141/141, Julia C ABI bridge 30/30, C ABI contract 21/21.
• Conformance-only transcript posted on the issue: Release wave (reference driver): QUBODrivers 0.6 / QUBOTools 0.13 compat + benchmark conformance #26 (comment)
  • supports_seed=true, honors_final_reads=true, enforces_time_limit=true.
  • Benchmark Conformance 14/14; no trait-skipped sets.
• julia --project=/tmp/mqlib-issue26-qubo-resolve -e 'using Pkg; Pkg.activate("/tmp/mqlib-issue26-qubo-resolve"; shared=false); Pkg.add(Pkg.PackageSpec(name="QUBO", version="0.6")); Pkg.develop(path=pwd()); Pkg.status()'
  • Passed; the temporary environment resolves QUBO v0.6.0 plus this MQLib branch with QUBODrivers v0.6.0 and QUBOTools v0.13.0.

Notes

• The QUBODrivers conformance API from Add benchmark conformance checks to QUBODrivers.test QUBODrivers.jl#52 is merged but not tagged as a newer registry release yet; the current registered QUBODrivers v0.6.0 predates the trait functions. This branch keeps MQLib loadable with released v0.6.0 and activates the benchmark metadata/trait/conformance path when the merged API is available.
• The C ABI source syntax check remains skipped locally unless MQLIB_UPSTREAM_DIR is set.

[bernalde]

Blocking issues: 2

Nonblocking issues: none.

Questions: none.

Tests run and outcomes:

• julia --project=. -e 'using Pkg; Pkg.test()' passed with released dependencies: Compatibility metadata 6/6, QUBODrivers 124/124, Julia C ABI bridge 21/21, C ABI contract 21/21. The C ABI source syntax check was skipped because MQLIB_UPSTREAM_DIR is not set.
• julia --project=/tmp/mqlib-issue26-conformance -e 'using Test, MQLib; ... QUBODrivers.test(MQLib.Optimizer; examples=false, benchmark_conformance=true) ...' passed against the QUBODrivers #52 checkout: MQLib benchmark conformance 115/115, including Benchmark Conformance 14/14.
• julia --project=/tmp/mqlib-review-qubo-resolve -e 'using Pkg; Pkg.activate(...); Pkg.add(Pkg.PackageSpec(name="QUBO", version="0.6")); Pkg.develop(path=pwd()); Pkg.status()' passed; QUBO v0.6.0 resolves with this branch.
• gh pr checks 27 reports all three CI jobs passing.

Merge-readiness: I would not merge this until the blocking issues above are addressed. Since I am the PR author, I am submitting this as COMMENT; the formal approval or request-changes verdict needs to come from another maintainer.

[bernalde]

Blocking: this compat still allows the only registered QUBODrivers 0.6 tag, v0.6.0, but that tag predates the conformance API from JuliaQUBO/QUBODrivers.jl#52. Under the PR's normal dependency resolution and CI, validate_metadata, honors_final_reads, and enforces_time_limit are absent, so has_benchmark_metadata_api() is false and the test suite silently runs the legacy QUBODrivers.test path instead of benchmark_conformance = true. That misses issue #26's core acceptance criterion that CI runs the upgraded conformance suite. Please wait for/register a QUBODrivers release containing #52 and constrain compat to that release, or otherwise make CI intentionally test the merged conformance API and fail when it is unavailable.

[bernalde]

Not addressed. I verified there is still no QUBODrivers tag newer than v0.6.0, so there is no released build containing JuliaQUBO/QUBODrivers.jl#52 to constrain compat to. I did not pin CI to an unreleased source or modify workflow files. Leaving this Blocking thread unresolved pending maintainer review.

[bernalde]

Addressed in 47050a8. Compat now requires QUBODrivers = "0.6.1", the trait methods are required at load time, and the test suite always runs QUBODrivers.test(...; benchmark_conformance = true) with no legacy fallback. Verified against released deps: QUBODrivers v0.6.1, Benchmark Conformance 14/14.

[bernalde]

Blocking: with the currently allowed registered QUBODrivers v0.6.0, this helper exists but QUBODrivers does not post-stamp the configured seed into metadata. Because _mqlib_metadata does not pass a seeds dictionary, solving with MQLib.RandomSeed() set emits metadata["seeds"] == Dict() under the dependency version this PR allows. That violates issue #26's requirement to record seeds under metadata["seeds"]. Please pass the configured seed into this helper and initialize seeds = Dict("sampler" => seed) when it is non-nothing, or tighten compat to a QUBODrivers release that reliably stamps the seed and make CI exercise that release.

[bernalde]

Addressed in 1768fc2. _mqlib_metadata now receives the configured RandomSeed and sets metadata["seeds"]["sampler"] for both metadata paths. test_public_c_api_bool_max_objectives now asserts this under released dependencies. Tests: Pkg.test, QUBODrivers #52 conformance, QUBO v0.6 coexistence, native C ABI smoke.

[bernalde]

Declined Blocking comment needing maintainer decision: Project.toml QUBODrivers compat/conformance API remains unresolved. The latest QUBODrivers tag is still v0.6.0, which predates JuliaQUBO/QUBODrivers.jl#52, so there is no released compat bound I can set. I did not pin CI to unreleased sources or modify workflow files.

Commits pushed:

• 1768fc2f50386e7073d8019a8b88da02779e73d0 1768fc2

Main changes:

• Threaded configured MQLib.RandomSeed() into solution metadata.
• Populated metadata["seeds"]["sampler"] for both the _sampler_metadata and fallback metadata paths.
• Added a test assertion that seed metadata is present under the currently released dependency graph.

Tests run:

• julia --project=. -e using Pkg; Pkg.test(): passed.
• Focused seed metadata solve under released dependencies: passed, Dict("sampler" => 1234).
• QUBODrivers #52 conformance environment: passed, 115/115 including Benchmark Conformance 14/14.
• QUBO v0.6 coexistence environment: passed, using MQLib, QUBO loaded.
• bash test/native_c_api_smoke.sh: passed.
• git diff --check: passed.

Comments intentionally not addressed:

• Blocking Project.toml compat/conformance API comment remains unresolved for the reason above. It needs a QUBODrivers release containing #52 or an explicit maintainer decision to test an unreleased source.

Remaining risks or follow-up:

• This PR is still not merge-ready against registered dependencies until QUBODrivers releases the #52 API and compat/tests are updated accordingly.
• I found no separate lint or type-check command configured in this repo beyond the Julia tests and CI native smoke.

[bernalde]

Update for released QUBODrivers v0.6.1:

Commits pushed:

• 47050a8b5212142177780342125c1d85e1c43e78 47050a8

Main changes:

• Tightened QUBODrivers compat from 0.6 to 0.6.1.
• Made the benchmark conformance trait methods required instead of guarded behind isdefined checks.
• Removed the legacy QUBODrivers.test(MQLib.Optimizer) fallback and now always run QUBODrivers.test(...; benchmark_conformance = true).
• Made metadata validation unconditional in the public C API smoke test.

Tests run:

• julia --project=. -e "using Pkg; Pkg.test()": passed with QUBODrivers v0.6.1; QUBODrivers testset passed 141/141.
• Focused released-dependency conformance run: passed 115/115, including Benchmark Conformance 14/14.
• bash test/native_c_api_smoke.sh: passed.
• git diff --check: passed.

Comments intentionally not addressed:

• None in this update.

Remaining risks or follow-up:

• CI still needs to complete on GitHub for this new head commit.

[bernalde]

Blocking issues: none.

Nonblocking issues: 1

• CHANGELOG compatibility wording is stale. It says QUBODrivers 0.6, but the PR now requires QUBODrivers = "0.6.1"; update the entry so release notes do not imply v0.6.0 remains acceptable.

Questions: none.

Tests run and outcomes:

• julia --project=. -e 'using Pkg; Pkg.test()': passed with released QUBODrivers v0.6.1; Compatibility metadata 6/6, QUBODrivers 141/141, Julia C ABI bridge 30/30, C ABI contract 21/21. The C ABI source syntax check was skipped because MQLIB_UPSTREAM_DIR is not set.
• Focused released-dependency conformance command: passed 115/115, including Benchmark Conformance 14/14; supports_seed=true, honors_final_reads=true, enforces_time_limit=true.
• QUBO v0.6 coexistence check in a temporary environment: passed; resolved QUBO v0.6.0, local MQLib v0.6.0, QUBODrivers v0.6.1, and QUBOTools v0.13.0.
• bash test/native_c_api_smoke.sh: passed against upstream MQLib 5854962.
• git diff --check: passed.
• gh pr checks 27: all three CI jobs passed.

Merge-readiness: merge-ready from a correctness standpoint; only the nonblocking changelog wording remains. Since I am the PR author, this review is submitted as COMMENT; formal approval must come from another maintainer.

[bernalde]

Nonblocking: This says QUBODrivers 0.6, but the PR now requires QUBODrivers = "0.6.1". Please update the entry to name 0.6.1 so the release notes do not imply v0.6.0 remains acceptable.

[bernalde]

Addressed in c419e68. The changelog now names QUBODrivers 0.6.1 to match the compat bound. Verified with a compat/changelog assertion, Pkg.test, native C ABI smoke, and git diff --check.

[bernalde]

Commits pushed:

• c419e682370736c9154a01b2b379a068904c195b c419e68

Main changes:

• Updated the unreleased changelog entry from QUBODrivers 0.6 to QUBODrivers 0.6.1 so it matches Project.toml compat.

Tests run and results:

• julia --project=. -e using TOML; project = TOML.parsefile("Project.toml"); changelog = read("CHANGELOG.md", String); @assert project["compat"]["QUBODrivers"] == "0.6.1"; @assert occursin("QUBODrivers 0.6.1", changelog); println("compat/changelog check passed"): passed.
• julia --project=. -e using Pkg; Pkg.test(): passed with QUBODrivers v0.6.1; QUBODrivers testset 141/141.
• bash test/native_c_api_smoke.sh: passed.
• git diff --check: passed.

Comments intentionally not addressed:

• None.

Remaining risks or follow-up items:

• GitHub Actions for this new head commit may still be running.