Merge pull request #82 from bernalde/fix/issue-71-tutorial-scope

Clarify repeat reliability tutorial scope

Author: bernalde

examples/Simulated_Annealing/reliability_analysis.ipynb

@@ -3012,6 +3012,19 @@
     "display(after_changed_rows.reset_index(drop=True))\n",
     "reference_alignment\n"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Relationship to the Other Tutorials\n",
+    "\n",
+    "This repeat-reliability tutorial does not invalidate the other Window Sticker tutorials. Those tutorials still demonstrate cross-instance benchmarking: bootstrapped Response and PerfRatio curves, interpolation across resource levels, train/test splits, virtual-best references, and parameter-recommendation strategies for unseen instances from the same problem family.\n",
+    "\n",
+    "The reliability analysis here answers a different per-instance question: whether a solver, parameter setting, and resource level has enough repeated stochastic runs to support a Bernoulli success probability and derived `R_c`, RTT/TTS, and CETS estimates. Continuous Response and PerfRatio curves remain bootstrap-based unless the workflow converts them into thresholded success events.\n",
+    "\n",
+    "Use this tutorial as an additional diagnostic layer when benchmark conclusions depend on stochastic repeat counts, especially for low-success-probability or close-comparison regions. Use the other tutorials for the core Window Sticker workflow and add repeat reliability only when the question is about per-instance repeat sufficiency.\n"
+   ]
   }
  ],
  "metadata": {

tests/test_verify_tutorials.py

@@ -141,6 +141,10 @@ def test_simulated_annealing_companion_reliability_notebook_is_self_contained():
     assert "load_targeted_rerun_runs" in source
     assert "targeted_sa_reruns.npz" in source
     assert "after_rerun_reliable_sweeps" in source
+    assert "Relationship to the Other Tutorials" in source
+    assert "does not invalidate the other Window Sticker tutorials" in source
+    assert "cross-instance benchmarking" in source
+    assert "per-instance question" in source
 
 
 def test_load_manifest_requires_skip_reason(tmp_path):