Simple wrapper scripts for running Quibble locally via Docker.
Inspired by mediawiki-quickstart, using the same one-file-per-command convention.
- Bash
- Git
- Docker
- ShellCheck (optional, for linting)
- Bats (optional, for unit tests)
Alternatively, open the repo in the provided dev container (.devcontainer/devcontainer.json) to get Bash, Git, Docker (via the host socket), ShellCheck, Bats, and the Claude Code CLI pre-installed.
All commands run silently: they print debug info and a logs: line pointing at the log location, then a dot per line of output for progress feedback. The full output — including a set -x trace of every command — is saved to the log named on that first line. Each run gets its own timestamped directory under log/ (see lib/log_dir): single commands write e.g. log/2026-07-06-14-03-59/fresh_install.log, and batch scripts write per-step logs like log/2026-07-06-14-03-59/extensions-Echo--install.log into the same run directory. The container mounts that same directory at /workspace/log, so Quibble's own logs, MediaWiki debug logs, WDIO result files, and failure screenshots land in the run's directory too — runs never overwrite each other's artifacts. To watch a run live, tail -f the log file from another terminal. Run directories accumulate until removed (./remove_all wipes log/ entirely, or rm -rf log by hand). ./shellto is the exception: it is interactive and always shows its output. ./remove_all also streams directly (and prints no logs: pointer): it deletes log/ itself, so a capture log could not survive its own run. ./help and the data-query scripts also print directly — their output is the product.
UTC timestamps in batch script output (per-step result lines, wave/combination headers, and the EXIT-trap total UTC line) are off by default. Set TIME_UTC=1 to append a YYYY-MM-DD HH:MM:SS UTC timestamp to those lines.
./test_integration # no timestamps (default)
TIME_UTC=1 ./test_integration # append UTC timestamps
Useful for long-running batch scripts (find_dependencies_minimal_*, run_selenium_tests_*_gated, install_all_gated, install_each_gated, test_integration*) where knowing when each step finished is helpful.
Elapsed-time durations (per-step / per-test (Xs) strings, (Xs) (success) / FAIL (Xs) lines, the EXIT-trap total (Xs) line, and generate_examples/generate_examples_slow's took (Xs) summary plus, at PARALLEL=N, their per-item pool completion times and a sorted Slowest items: list) are off by default. Set TIME_ELAPSED=1 to enable them.
./test_integration # no durations (default)
TIME_ELAPSED=1 ./test_integration # show durations
Combinable with TIME_UTC=1 for full timing output:
TIME_ELAPSED=1 TIME_UTC=1 ./test_integration
Override the Docker image used by all commands. Useful when developing or testing changes to Quibble itself.
QUIBBLE_IMAGE=my-quibble:dev ./fresh_install
QUIBBLE_IMAGE=docker-registry.wikimedia.org/releng/quibble-bookworm-php83:1.2.3 ./install extensions/Echo
Default: docker-registry.wikimedia.org/releng/quibble-bookworm-php83:latest
Run multiple independent sessions on the same machine simultaneously. Each environment uses isolated directories (src_N/, src_save_N/) so they don't conflict.
# Terminal 1: install and test Echo
ENVIRONMENT=0 ./fresh_install
ENVIRONMENT=0 ./install extensions/Echo
ENVIRONMENT=0 ./run_selenium_tests extensions/Echo
ENVIRONMENT=0 ./remove
# Terminal 2 (at the same time): run find_dependencies_minimal_greedy for MinervaNeue
ENVIRONMENT=1 ./find_dependencies_minimal_greedy skins/MinervaNeue
ENVIRONMENT=1 ./remove
Sets QUIBBLE_SRC=src_N and QUIBBLE_SAVE=src_save_N. Cache and ref directories are shared (safe for concurrent use). Use ./remove_srcs to remove all environments at once.
FAST=1 runs ./fresh_install once, saves the state with ./save, then uses ./restore instead of re-running ./fresh_install for each subsequent component (or, for find_dependencies_minimal_*, each combination).
Used by ./install_each_gated, ./run_selenium_tests_all_gated, ./run_selenium_tests_required_gated, and ./find_dependencies_minimal_* (find_dependencies_minimal_gated propagates FAST to its find_dependencies_minimal_greedy children).
FAST=1 ./install_each_gated
FAST=1 ./run_selenium_tests_all_gated
FAST=1 ./find_dependencies_minimal_greedy extensions/Echo
DRY_RUN=1 passes --dry-run to Quibble so it prints what it would do without actually running tests or installing anything. Useful for testing wrapper-script output (especially long-running commands) without paying the cost of a real run. A dry run must not destroy state either: ./fresh_install skips its usual ./remove of src/ under DRY_RUN=1 (Quibble installs nothing, so the removal would be the dry run's only real effect — it used to leave src/ empty, breaking the next ./save or test run).
Applied directly by ./fresh_install, ./install, ./run_selenium_tests, and ./run_php_unit_tests. Inherited via the env by wrapper scripts that call them: ./generate_examples, ./generate_examples_slow, ./find_dependencies_minimal_*, ./install_each_gated, ./install_all_gated, ./run_selenium_tests_all_gated, and ./run_selenium_tests_required_gated. Their inner Quibble-running calls short-circuit too.
DRY_RUN=1 ./fresh_install
DRY_RUN=1 ./install extensions/Echo
DRY_RUN=1 ./run_selenium_tests extensions/Echo
DRY_RUN=1 ./run_php_unit_tests extensions/Echo
DRY_RUN=1 ./generate_examples
DRY_RUN=1 ./find_dependencies_minimal_greedy extensions/Echo
RESOLVE_REQUIRES controls whether --resolve-requires is passed to Quibble. Default 1 (on): Quibble reads each repo's extension.json/skin.json requires field and installs transitive dependencies automatically. Set to 0 to install exactly the deps listed in QUIBBLE_DEPS (or zuul/dependencies.yaml) with no auto-resolution.
RESOLVE_REQUIRES=0 ./install extensions/Echo
./find_dependencies_minimal_* sets RESOLVE_REQUIRES=0 automatically (via the shared lib/minimal_setup) — otherwise Quibble silently re-installs an optional dependency the algorithm just removed (via some kept dep's transitive requires), and the reported minimum is artificially small.
BRANCH selects which git branch to install and check out, matching mediawiki-quickstart's BRANCH. When set, it passes --branch to Quibble (which branch to clone for core and any components) and then re-attaches HEAD to that branch, so src/ ends up on a named branch. Unset (the default), src/ is left in Quibble's detached HEAD state.
BRANCH="wmf/1.44.0-wmf.20" ./fresh_install
BRANCH="REL1_44" ./install extensions/Echo
Applied by ./fresh_install and ./install. Repos that lack the requested branch fall back to their own default branch (the same fallback Quibble uses), so no repo is left detached. If a repo can't be put on the branch or its fallback, the command fails — since BRANCH is explicit, an unmet request is surfaced rather than silently ignored.
GERRIT_PATCHES fetches a Gerrit patch and checks it out after install, matching mediawiki-quickstart's GERRIT_PATCHES. Set it to a single change ref; the patch is applied to core for ./fresh_install and to the named component for ./install, on a branch named after the ref (e.g. refs-changes-94-1146994-6).
GERRIT_PATCHES="refs/changes/94/1146994/6" ./fresh_install # a mediawiki/core change
GERRIT_PATCHES="refs/changes/20/1306020/1" ./install extensions/Echo # an extensions/Echo change
Applied by ./fresh_install and ./install. The ref must belong to the repo being patched (core, or the named component) — Gerrit change refs are repo-specific, so a ref that isn't in that repo fails the run loudly rather than being silently ignored. Because the patch lands after install, it takes effect immediately for code, tests, and in-tree config (MediaWiki commits autoload metadata in-tree); a patch that adds or changes third-party composer/npm dependencies needs a re-install. Only a single patch is supported (unlike upstream's space-separated list): to patch core or a dependency while installing a component, patch that repo directly (e.g. ./fresh_install for core).
RUNS controls how many times ./find_flaky_selenium_tests runs the Selenium tests. Default 1 — the minimal behavior (like PARALLEL), so a bare invocation doesn't multiply the runtime. Set it to 2 or higher to detect flakiness. Must be a positive integer; anything else fails loudly.
RUNS=5 ./find_flaky_selenium_tests extensions/Echo
Applied only by ./find_flaky_selenium_tests.
Set up MediaWiki (without running tests). Runs ./prepare first if needed, then ./remove to ensure a fresh src/ (the removal is skipped under DRY_RUN=1 — a dry run must not destroy state). Run ./shellto afterwards to open a shell with MediaWiki running.
./fresh_install
DRY_RUN=1 ./fresh_install # pass --dry-run to Quibble (no real install; src/ is left untouched)
BRANCH=REL1_44 ./fresh_install # install and check out a branch (default: leave detached HEAD)
GERRIT_PATCHES="refs/changes/94/1146994/6" ./fresh_install # fetch and check out a Gerrit patch for core
Install an extension or skin. Assumes ./fresh_install has been run first. Run ./shellto afterwards to open a shell with MediaWiki running.
./install extensions/Echo
./install skins/MinervaNeue
QUIBBLE_DEPS="" ./install extensions/Echo # no dependencies
QUIBBLE_DEPS="EventLogging" ./install extensions/Echo # only specific dependencies
DRY_RUN=1 ./install extensions/Echo # pass --dry-run to Quibble (no real install)
RESOLVE_REQUIRES=0 ./install extensions/Echo # install only QUIBBLE_DEPS; do not auto-resolve transitive requires
BRANCH=REL1_44 ./install extensions/Echo # install and check out a branch (default: leave detached HEAD)
GERRIT_PATCHES="refs/changes/20/1306020/1" ./install extensions/Echo # fetch and check out a Gerrit patch for the component
Environment variables:
QUIBBLE_DEPS: Override which dependencies to install (space-separated). When set, replaces the dependencies fromzuul/dependencies.yaml. Set to empty string for no dependencies.RESOLVE_REQUIRES: Default1. Set to0to install exactly the deps listed inQUIBBLE_DEPSwithout Quibble auto-resolving transitiverequires. SeeRESOLVE_REQUIRES.BRANCH: When set, install and check out this branch (passed to Quibble as--branch, thenHEADre-attached); unset leaves Quibble's detachedHEAD. SeeBRANCH.GERRIT_PATCHES: When set, fetch and check out a single Gerrit patch (a change ref belonging to the component) in the component after install. SeeGERRIT_PATCHES.
See: Install MediaWiki Core and an Extension
Run Selenium tests. Assumes ./fresh_install (or ./install) has been run first.
./run_selenium_tests
./run_selenium_tests extensions/Echo
./run_selenium_tests --spec tests/selenium/wdio-mediawiki/specs/BlankPage.js
./run_selenium_tests --spec tests/selenium/specs/page.js
./run_selenium_tests extensions/Echo --spec tests/selenium/specs/notifications.js
./run_selenium_tests --spec tests/selenium/specs/user.js --mochaOpts.grep "should be able to create account"
./run_selenium_tests extensions/Echo --spec tests/selenium/specs/notifications.js --mochaOpts.grep "alerts and notices are visible"
DRY_RUN=1 ./run_selenium_tests extensions/Echo # pass --dry-run to Quibble (no real run)
See: Run tests targeting Quibble
Run PHPUnit tests. Assumes ./fresh_install (or ./install) has been run first.
./run_php_unit_tests
./run_php_unit_tests --filter testValidSpecialPageAliases
./run_php_unit_tests extensions/Echo
./run_php_unit_tests extensions/Echo --filter testNotificationCount
DRY_RUN=1 ./run_php_unit_tests extensions/Echo # pass --dry-run to Quibble (no real run)
See: Run tests targeting Quibble
Open a shell in the container with MediaWiki running at http://127.0.0.1:9413. Assumes ./fresh_install (or ./install) has been run first, and fails fast with a clear error when src/ has no MediaWiki checkout (instead of letting Quibble die a minute later with a confusing traceback). Container logs (Apache, MySQL, MediaWiki debug logs) land in this run's log directory — the logs: line printed at startup names it.
./shellto
Run fast integration tests (~20 minutes total) and report which ones passed or failed. Useful for detecting regressions after changes. Slow entries live in ./test_integration_slow.
./test_integration
To start from a completely clean state, run ./remove_all first.
Warning: This script inhibits sleep to prevent the machine from suspending.
Run integration tests that are too slow for the fast suite (./test_integration). Includes network-heavy setup, full test suites, exhaustive algorithms, and gated repository tests. PARALLEL=N runs the independent entries across N parallel slots (each in its own ENVIRONMENT=N/src_N), after a serial prelude (core install + PHPUnit, then prepare_gated). Each pooled entry captures to log/<timestamp>/parallel/<index>.log, with step logs in a per-entry <index>/ directory, so entries that reuse a slot can't overwrite each other's logs. The default is fully serial. Use ./suggest_parallel to pick N.
./test_integration_slow
Warning: This script inhibits sleep to prevent the machine from suspending. This will take a long time to run.
Run Bats unit tests. Fast, no Docker needed. Requires Bats in addition to the base prerequisites. Unit tests are enforced offline: HTTP(S) is routed through a dead local proxy, so an accidental network call fails in under a second instead of hanging the run (a test that genuinely needs the network belongs in an integration suite instead).
./test_unit
./test_unit test/lib_awk.bats # run a single test file
Prepare the local environment for running Quibble. Pulls the Docker image, clones bare git repos as references, and creates working directories. PARALLEL=N caps concurrency to N jobs at a time. The default is all at once (the job set is small and fixed) — unlike fetch/prepare_gated, which default to sequential; PARALLEL=N only throttles prepare for constrained machines.
./prepare
PARALLEL=1 ./prepare
Save the current state of src/ (MediaWiki installation) for fast restore later. Uses Docker-as-root to copy files (works on both macOS and Linux).
./save
Restore src/ from a previously saved state (created by ./save). Much faster than running ./fresh_install again.
./restore
Fetch the latest changes for bare git repos in ref/ from Gerrit. Updates each repo's branch refs and prunes branches deleted upstream (see lib/fetch_bare; a plain git fetch would update nothing in these refspec-less bare clones). With no arguments, fetches all repos. With arguments, fetches only the specified repos.
./fetch
./fetch ref/mediawiki/core.git
./fetch ref/mediawiki/extensions/Echo.git ref/mediawiki/skins/Vector.git
PARALLEL=1 ./fetch
Remove src/ (MediaWiki source code). Cache, logs, bare git repos, and the Docker image are kept.
./remove
Remove all src/ directories across all environments (src/, src_save/, src_N/, src_save_N/, src_worker_N/). Keeps ref/, cache/, log/, and the Docker image.
./remove_srcs
Remove everything created by quibble-local, including bare git repos in ref/ and the Docker image. Streams its output directly instead of capturing it (like ./shellto): it deletes log/ itself, so a capture log could not survive its own run — and a failure reason must stay visible. For the same reason it prints no logs: pointer.
./remove_all
These scripts output information and don't run Docker containers.
Output dependencies for an extension or skin from zuul/dependencies.yaml.
./list_dependencies extensions/Echo
./list_dependencies skins/MinervaNeue
Output required dependencies for an extension or skin from its extension.json or skin.json. These are the extensions/skins listed in the requires field that must always be present.
./list_dependencies_required extensions/Echo
./list_dependencies_required extensions/GrowthExperiments
./list_dependencies_required skins/MinervaNeue
Output optional dependencies for an extension or skin. These are dependencies in zuul/dependencies.yaml that are NOT in extension.json/skin.json requires field. Complement of ./list_dependencies_required.
./list_dependencies_optional extensions/Echo
./list_dependencies_optional skins/MinervaNeue
Output all possible combinations of dependencies for an extension or skin. One combination per line (space-separated), starting with one dependency, ending with all dependencies.
./list_dependencies_combinations extensions/Echo
./list_dependencies_combinations skins/MinervaNeue
Output the list of gated repositories (extensions and skins) from parameter_functions.py. Clones integration/config into src/config if needed. Assumes ./prepare has been run first.
./list_gated
Check if a component has Selenium tests. Exits 0 if selenium-test script exists in package.json, 1 otherwise. Reads package.json from src/<component> if a materialized working copy is present (it contains .git — the empty src/ stub ./prepare creates doesn't count), otherwise from the bare repo in ref/ via git show. Clones the bare repo from Gerrit if not already cloned. No Docker or fresh_install required, and no host write to src/.
./selenium_tests_exist
./selenium_tests_exist extensions/Echo
./selenium_tests_exist skins/MinervaNeue
./selenium_tests_exist skins/Vector
Suggest the number of parallel workers based on available CPU and memory. Each worker needs ~2 CPU cores and ~2 GB of Docker memory. Outputs a single number. Used by find_dependencies_minimal_bottom_up, find_dependencies_minimal_gated, find_dependencies_minimal_thorough, install_each_gated, run_selenium_tests_all_gated, and run_selenium_tests_required_gated.
On macOS, the result may differ depending on whether Docker is running. When Docker is running, CPU and memory are read from docker info, which reports the Docker Desktop VM allocation (often lower than host resources). When Docker is not running, the script falls back to sysctl, which reports full system CPU and memory — potentially suggesting more workers than Docker can actually support.
To maximize parallel workers on macOS, increase Docker Desktop memory: Docker Desktop → Settings → Resources → Memory. The formula is: workers = min(CPUs / 2, memory / 2 GB). On a 10-core / 64 GB machine with Docker Desktop defaults (~6 GB), memory is the bottleneck (3 workers instead of 5). Increasing Docker memory to 10+ GB removes the bottleneck.
./suggest_parallel
PARALLEL=1 ./run_selenium_tests_all_gated extensions/Echo
These scripts find which optional dependencies are actually needed for Selenium tests to pass. They are long-running.
Find the minimum dependencies using a greedy algorithm: starts with all optional deps, removes one at a time. O(N). Repeats until stable to catch order-dependent removals. Good general-purpose choice.
Pick this when: speed matters more than guaranteed correctness (greedy can miss the true minimum). For a guaranteed minimum, use ./find_dependencies_minimal_bottom_up (fast when few deps are needed) or ./find_dependencies_minimal_thorough (fast when many).
./find_dependencies_minimal_greedy extensions/Echo
FAST=1 ./find_dependencies_minimal_greedy extensions/Echo
- Fast for extensions/GrowthExperiments (17 deps, ~8 needed): ~17 tests regardless of how many are needed.
- Slower for extensions/Echo (4 deps, 0 needed): tests all 4 before concluding none are needed, while
find_dependencies_minimal_bottom_upfinds the answer in 1 test.
Environment variables:
FAST=1: Runs./fresh_installonce, saves state, then restores instead of re-running./fresh_installfor each combination.
Warning: This script inhibits sleep to prevent the machine from suspending.
Find the minimum dependencies by testing combinations from smallest (0 deps) to largest. Stops at the first passing combination — guaranteed smallest.
Pick this when: you need a guaranteed minimum and expect the answer to be small (few deps actually needed). When many deps are needed, use ./find_dependencies_minimal_thorough instead — its greedy upper bound prunes the search space.
./find_dependencies_minimal_bottom_up extensions/Echo
FAST=1 ./find_dependencies_minimal_bottom_up extensions/Echo
PARALLEL=1 ./find_dependencies_minimal_bottom_up extensions/Echo
PARALLEL=1 FAST=1 ./find_dependencies_minimal_bottom_up extensions/Echo
- Fast for extensions/Echo (4 deps, 0 needed): tests empty set first, passes in 1 test.
- Extremely slow for extensions/GrowthExperiments (17 deps, ~8 needed): tests up to 2^17 = 131,072 combinations (~10 min each).
Environment variables:
FAST=1: Runs./fresh_installonce, saves state, then restores instead of re-running./fresh_installfor each combination.PARALLEL=N: Run N combinations simultaneously, each in an isolatedENVIRONMENT=N(src_N/). Use./suggest_parallelto determine N for your machine. Each worker needs ~2 CPU cores and ~2 GB of Docker memory.
Warning: This script inhibits sleep to prevent the machine from suspending.
Find and verify the minimum dependencies. Phase 1: greedy for a fast estimate. Phase 2: exhaustive verification of all smaller combinations. Confirms the result is truly minimal.
Pick this when: you need a guaranteed minimum and expect the answer to be large (many deps actually needed). Always slower than ./find_dependencies_minimal_greedy (it runs greedy plus verification); when few deps are needed, ./find_dependencies_minimal_bottom_up is faster.
./find_dependencies_minimal_thorough extensions/Echo
FAST=1 ./find_dependencies_minimal_thorough extensions/Echo
PARALLEL=1 ./find_dependencies_minimal_thorough extensions/Echo
PARALLEL=1 FAST=1 ./find_dependencies_minimal_thorough extensions/Echo
- Fast for extensions/Echo (4 deps, 0 needed): greedy finds 0 in ~4 tests, verification confirms immediately.
- Moderate for extensions/GrowthExperiments (17 deps, ~8 needed): greedy finds ~8 in ~17 tests, then verifies by testing combinations of size 0–7 only (not all 131,072).
Environment variables:
FAST=1: Runs./fresh_installonce, saves state, then restores instead of re-running./fresh_installfor each combination.PARALLEL=N: Run N combinations simultaneously, each in an isolatedENVIRONMENT=N(src_N/). Use./suggest_parallelto determine N for your machine. Each worker needs ~2 CPU cores and ~2 GB of Docker memory.
Warning: This script inhibits sleep to prevent the machine from suspending.
Run ./run_selenium_tests RUNS times (default 1) against the current install and report flaky tests. A test that passes in some runs and fails in others is flaky; a test that fails in every run is consistently failing (likely a real failure, not flakiness). Assumes ./fresh_install (or ./install) has been run first. Exits 0 only when every run passed.
The report is built from the JUnit XML result files WDIO writes into this run's log directory (log/<timestamp>/WDIO.xunit-*.xml — the directory the container mounts at /workspace/log), not from console output. After each run, the result files that run produced are moved to log/<timestamp>/find_flaky_selenium_tests/run-N/ and parsed; result files from other invocations are never touched (each has its own run directory, so concurrent Selenium runs in other terminals can't pollute this report). Failure screenshots stay in the run directory.
Components whose selenium-test script runs Cypress instead of WDIO (e.g. extensions/Wikibase) write no XML result files into the run directory (only artifacts like videos in its <Component>/ subdirectory) — for those, every run shows (no result files), the per-test sections stay empty, and flakiness is still detected per run from exit codes.
All arguments pass through unchanged to ./run_selenium_tests (component, --spec, --mochaOpts.grep, ...). To check whether ONE suspected test is flaky, combine --spec and --mochaOpts.grep and raise RUNS — far faster than repeating a whole suite (though each run still pays the fixed container and wdio startup overhead, so 100 targeted runs take on the order of an hour):
RUNS=100 ./find_flaky_selenium_tests extensions/Echo --spec tests/selenium/specs/notifications.js --mochaOpts.grep "alerts and notices are visible"
Usage:
./find_flaky_selenium_tests --spec tests/selenium/wdio-mediawiki/specs/BlankPage.js
./find_flaky_selenium_tests extensions/Echo
RUNS=2 ./find_flaky_selenium_tests extensions/Echo --spec tests/selenium/specs/notifications.js
RUNS=5 ./find_flaky_selenium_tests extensions/Echo
Environment variables:
RUNS=N: How many times to run the tests (positive integer, default1). The default is the minimal behavior (likePARALLEL); setRUNS=2or higher to detect flakiness. SeeRUNS.
Warning: This script inhibits sleep to prevent the machine from suspending.
See: Run tests targeting Quibble
These scripts operate on all gated extensions and skins (from ./list_gated). They take a long time to run.
Clone or fetch bare repos for all gated repositories. Extends ./prepare by cloning bare repos for all gated extensions and skins. Assumes ./prepare has been run first (needs ref/integration/config.git). PARALLEL=N clones/fetches N repos at a time (default: sequential).
./prepare_gated
PARALLEL=1 ./prepare_gated
Install all gated extensions and skins into a single MediaWiki. Runs ./fresh_install for core, then installs each gated component on top. Reports per-step and total duration. Unlike ./install_each_gated (which gives each component its own MediaWiki), this stacks everything into one MediaWiki. Run ./shellto afterwards to open a shell with MediaWiki running.
./install_all_gated
Warning: This script inhibits sleep to prevent the machine from suspending. This will take a very long time to run (50+ components).
Install each gated extension or skin into its own fresh MediaWiki, one at a time. For each component: ./fresh_install, then ./install. Reports per-step and total duration. Unlike ./install_all_gated (which stacks everything into one MediaWiki), this gives each component a clean MediaWiki so per-component install times are comparable.
./install_each_gated
./install_each_gated extensions/Echo
FAST=1 ./install_each_gated extensions/Echo
PARALLEL=1 ./install_each_gated extensions/Echo
PARALLEL=1 FAST=1 ./install_each_gated extensions/Echo
Warning: This script inhibits sleep to prevent the machine from suspending. This will take a very long time to run (50+ components).
Run Selenium tests for core and all gated repositories. For each component: ./fresh_install, ./install (if not core), check if Selenium tests exist, and run them.
./run_selenium_tests_all_gated
./run_selenium_tests_all_gated extensions/Echo
FAST=1 ./run_selenium_tests_all_gated extensions/Echo
PARALLEL=1 ./run_selenium_tests_all_gated extensions/Echo
PARALLEL=1 FAST=1 ./run_selenium_tests_all_gated extensions/Echo
Warning: This script inhibits sleep to prevent the machine from suspending. This will take a very long time to run (50+ components).
Install all gated extensions and skins into a single MediaWiki, then run all Selenium tests. Unlike ./run_selenium_tests_all_gated (which does ./fresh_install per component), this installs everything together into one src/.
./run_selenium_tests_gated
Warning: This script inhibits sleep to prevent the machine from suspending. This will take a long time to run (50+ components).
Run Selenium tests for all gated repositories using only required dependencies (from extension.json/skin.json). For each component: ./fresh_install, ./install with required deps only, check if Selenium tests exist, and run them. Fails loudly when listing a component's required dependencies fails, instead of silently running it with no dependencies at all.
./run_selenium_tests_required_gated
./run_selenium_tests_required_gated extensions/Echo
FAST=1 ./run_selenium_tests_required_gated extensions/Echo
PARALLEL=1 ./run_selenium_tests_required_gated extensions/Echo
PARALLEL=1 FAST=1 ./run_selenium_tests_required_gated extensions/Echo
Warning: This script inhibits sleep to prevent the machine from suspending. This will take a very long time to run (50+ components).
Find minimum dependencies for all gated repositories (or a single component). For each component: check if Selenium tests exist, check if it has optional dependencies, and run ./find_dependencies_minimal_greedy to find the minimum set.
./find_dependencies_minimal_gated
./find_dependencies_minimal_gated extensions/Echo
FAST=1 ./find_dependencies_minimal_gated extensions/Echo
PARALLEL=1 ./find_dependencies_minimal_gated extensions/Echo
PARALLEL=1 FAST=1 ./find_dependencies_minimal_gated extensions/Echo
Environment variables:
FAST=1: Propagated to each./find_dependencies_minimal_greedychild. The child runs./fresh_installonce, saves state, then restores instead of re-running./fresh_installfor each combination.PARALLEL=N: Run N components simultaneously, each in an isolatedENVIRONMENT=N. Use./suggest_parallelto determine N for your machine. Each worker needs ~2 CPU cores and ~2 GB of Docker memory.
See also: ./find_dependencies_minimal_greedy for single-component usage.
Warning: Without arguments, this will take a very long time to run (50+ components). This script inhibits sleep to prevent the machine from suspending.
List all scripts with their description and usage.
./help
./help install
./help ./install
Run the same checks that GitLab CI runs (ShellCheck lint and Bats unit tests), using Docker in the same images as CI. Does not require ShellCheck or Bats to be installed locally. The Bats job runs with the same dead-proxy offline enforcement as ./test_unit, so an accidental network call in a unit test fails in under a second instead of hanging the job. Silent like every command: the first line points at this run's log file and the terminal shows a dot per line. Reports each job loudly in the log (see lib/ci_report): on failure it prints a banner and re-surfaces the failing tests at the end of the log, the terminal shows FAIL (see <log>), and ./ci exits non-zero.
./ci
Run ShellCheck on all shell scripts in the repo (root, lib/, and .devcontainer/). Requires ShellCheck in addition to the base prerequisites.
./lint
Generate an example output file by running a command and capturing its output. Used to refresh the example .txt files in examples/. Writes a $ <command> header followed by the captured stdout+stderr. The captured command is allowed to fail, so usage-on-failure outputs can also be captured.
./generate_example examples/help.txt './help'
./generate_example examples/install.txt './install'
Regenerate the quick example output files in examples/ in bulk by iterating each project-root script's # Usage: block, skipping every Usage line that invokes a slow script (the curated list in lib/slow_scripts — the gated scripts, find_dependencies_minimal_*, the test suites, run_php_unit_tests, find_flaky_selenium_tests). ./generate_examples_slow (below) regenerates every example, slow ones included. Filenames are derived from the command via lib/cmd_to_filename. Takes roughly 30–60 minutes serially (less with PARALLEL=N); useful for refreshing examples/ after script renames or behavior changes without paying for the multi-hour slow set.
The skip is per Usage line, keyed on the script the line actually invokes (cmd_to_script in lib/cmd_to_filename), not on the script whose header contains it: suggest_parallel's Usage block cross-references ./run_selenium_tests_all_gated, and a per-script filter would let that multi-hour line into the quick run the moment its home script's identical line — which normally claims the shared filename first — was filtered out. The summary reports the skipped count (slow N) plus a pointer at ./generate_examples_slow; a full run (via the wrapper) skips nothing, prints no pointer, and keeps the pre-split summary field layout — only the leading label names the invoked variant.
./generate_examples
PREVIEW=1 ./generate_examples
DRY_RUN=1 ./generate_examples
PARALLEL=N ./generate_examples
PARALLEL=N DRY_RUN=1 ./generate_examples
PREVIEW, DRY_RUN, and PARALLEL do different things:
PREVIEW=1 ./generate_examples— outer-level preview. PrintsWould generate ...for each Usage line. No files are written, no inner scripts run. NamedPREVIEW, notDRY_RUN, becausePREVIEWskips execution entirely;DRY_RUNstill runs the wrapper scripts (see below).DRY_RUN=1 ./generate_examples— actually generate every file, butDRY_RUN=1is inherited by every inner Usage command via the env so Quibble short-circuits. Inner scripts that honorDRY_RUN(install,fresh_install,run_php_unit_tests,run_selenium_tests, and batch scripts that propagate the env var to them vialib/setup) finish in seconds instead of minutes. Other scripts ignore the unused env var. This matches howDRY_RUNalready propagates infind_dependencies_minimal_*and other batch scripts. Captures are written toexamples/dry_run/— committed like the real captures, but kept in their own directory so a dry run never overwrites the realexamples/*.txt.PARALLEL=N ./generate_examples— run middle-phase Usage lines (not whole scripts) concurrently across N reusable worker slots (lib/run_pool): as each line finishes, its slot is refilled, so a script's slow variants spread across slots and no slot idles while work remains. Per-line (not per-script) granularity means the slowest single command — not the slowest script's total across all its variants — bounds the run. Each slot runs in an isolatedENVIRONMENT=N(src_N/) seeded from the sharedsrc_save(so every line runs against a clean baseline), withQUIBBLE_SRC/QUIBBLE_SAVEpinned to the slot (so an innerPARALLEL=1 ...line's ownENVIRONMENT=1export cannot escape intosrc_1and collide with another slot), and with its ownQUIBBLE_LOG_DIR(a per-itemlog/<timestamp>/<example>subdirectory keyed by the line's examples file, not the slot) so concurrent batch scripts don't clobber each other's logs and a later line reusing the slot can't wipe a failed line's logs. Each worker also unsetsPARALLELso a bare inner command captures serial output (an inlinePARALLEL=1 ...in a Usage line still wins). Early scripts (prepare,prepare_gated,fresh_install,save) and late scripts (remove_srcs,remove,remove_all) stay serial — they prepare/destroy state every middle worker depends on. Per-item output goes tolog/<timestamp>/<example>/, where each worker also writes apassmarker as its last action. A missing marker means the worker died mid-item (a captured command's own non-zero exit is NOT a failure — seegenerate_example): the pool prints a liveSlot N FAILEDline pointing at the item's log, keeps going, recaps every failed item at the end, reportsfailed Nin the summary, and exits non-zero. A failed item'sexamples/*.txtis missing or stale — regenerate it before committing. (On a completed full run the late-phaseremove_alldeleteslog/, so the recap's command list is the durable record.)PREVIEW=1forces serial regardless ofPARALLELbecause parallel worker output would interleave unhelpfully when previewing the work plan.
Combinations: PREVIEW=1 DRY_RUN=1 ./generate_examples previews the DRY_RUN command list (PREVIEW wins — nothing runs). PARALLEL=N DRY_RUN=1 ./generate_examples runs the parallel path while short-circuiting Quibble. Use DRY_RUN=1 to iterate on generate_examples itself or to validate the pipeline end-to-end. DRY_RUN captures do not reflect real script behavior — which is why they live in their own committed examples/dry_run/ directory and never overwrite the real examples/.
Regenerate ALL example output files in examples/ — the quick set ./generate_examples covers plus every Usage line of the slow scripts listed in lib/slow_scripts (the gated scripts, find_dependencies_minimal_*, the test suites, run_php_unit_tests, find_flaky_selenium_tests). Unlike ./test_integration_slow (which runs only what ./test_integration doesn't), this is a superset of the quick run: examples/ is meant to be regenerated as one consistent set with one command.
./generate_examples_slow
PREVIEW=1 ./generate_examples_slow
DRY_RUN=1 ./generate_examples_slow
PARALLEL=N ./generate_examples_slow
PARALLEL=N DRY_RUN=1 ./generate_examples_slow
PREVIEW, DRY_RUN, and PARALLEL behave exactly as in ./generate_examples above: the wrapper is a single exec of ./generate_examples with the internal _QUIBBLE_EXAMPLES_ALL=1 flag exported, so they — like every other inherited variable (TIME_ELAPSED, TIME_UTC, ...) — reach the exec'd engine via the environment.
Warning: This will take a long time to run (many hours serially; about a day with PARALLEL=N). This script inhibits sleep (via the exec'd ./generate_examples) to prevent the machine from suspending.
These are sourced by other scripts and are not intended to be run directly.
Shared setup for batch scripts (test_integration, test_integration_slow, install_all_gated, install_each_gated, run_selenium_tests_all_gated, run_selenium_tests_gated, run_selenium_tests_required_gated, find_dependencies_minimal_greedy, find_dependencies_minimal_bottom_up, find_dependencies_minimal_gated, find_dependencies_minimal_thorough, find_flaky_selenium_tests). Sets up silent output, sources helper libraries (inhibit_sleep, print_results, heartbeat), creates the per-run log directory via lib/log_dir (which exports QUIBBLE_LOG_DIR), and initializes result tracking variables. run_step, lib/run_test, and lib/minimal_setup all write into QUIBBLE_LOG_DIR. A parent that runs batch scripts concurrently (e.g. generate_examples' pool workers) pins it to a per-item subdirectory (e.g. log/<timestamp>/<example>) so concurrent workers don't clobber each other's logs or delete a log a sibling is reading back. The user-facing "logs:" hint names the run-dir root (_QUIBBLE_LOG_ROOT, not the per-worker path) so a worker path can't leak into captured examples/.
Provides the ci_report function used by ./ci to report each job's result loudly. Given a job's name, exit status, and a file holding its captured output, prints either a one-line " passed" or a loud FAILED banner that re-surfaces the failing tests (bats "not ok" lines) at the end of the run, then returns the job's status so ./ci aborts on failure. Sourced by ci.
Run a command, save output to a log file, and print a dot for each line of output. Exports _QUIBBLE_TRACE=1 into the command's environment so a child sourcing lib/capture enables set -x — the trace lands in the per-step log being written here. Sourced by lib/batch_setup, which makes run_with_dots available to all batch scripts for progress feedback (called by run_step in lib/batch_setup, lib/run_test, and lib/minimal_setup). Provides run_with_dots function.
Outputs debug information (OS, CPU, RAM, bash, git, docker version, docker CPUs and RAM), checks basic prerequisites (git), shows a logs: pointer to where output is saved (only for scripts whose output is captured — set via _QUIBBLE_LOG), and sets up duration tracking via lib/duration_trap. Sourced by most scripts, except the data scripts (list_dependencies, list_dependencies_combinations, list_gated, list_dependencies_optional, list_dependencies_required, selenium_tests_exist) whose stdout is consumed by other scripts and would be corrupted by debug output.
Provides _quibble_format_duration function that formats elapsed seconds as a human-readable duration string (e.g. "1h 5m 30s"). Omits zero-value days, hours, and minutes; always shows seconds. Gated on the TIME_ELAPSED environment variable: off by default (returns empty), set TIME_ELAPSED=1 to enable. Sourced by lib/duration_trap and lib/batch_setup.
Provides the pluralize function that returns the singular or plural form of a word for a given count, so counts read grammatically (e.g. "1 worker" vs "2 workers"). Usage: pluralize COUNT SINGULAR [PLURAL]; PLURAL defaults to SINGULAR + "s", or pass it explicitly for irregular words. Sourced by lib/run_pool, lib/greedy, find_dependencies_minimal_gated, and find_dependencies_minimal_thorough.
Composable EXIT-trap registry. Bash allows only one handler per signal, so a second trap ... EXIT replaces the first; this installs a single dispatcher and runs every registered handler, in registration order, preserving the script's exit code. Handlers are keyed, so registering an existing key replaces it (the mutually-exclusive display handlers lib/duration_trap and lib/silent_output share the display key — last wins) while a cleanup handler under its own key always runs too. This is what stops lib/duration_trap from clobbering lib/inhibit_sleep's cleanup. Provides quibble_register_exit_trap. Sourced by lib/inhibit_sleep, lib/duration_trap, and lib/silent_output.
Registers (via lib/exit_trap, under the shared display key) an EXIT handler that prints total script duration when the script exits. Only activates when stdout is a terminal. Output is empty unless TIME_ELAPSED=1. Sources lib/format_duration and lib/exit_trap. Sourced by lib/debug_info and lib/batch_setup. lib/silent_output registers the same display key, so when both are sourced its handler replaces this one.
Shared setup sourced by scripts that run Docker commands. Sources lib/capture (per-run log directory, debug banner + logs: pointer, output capture with dots, set -x trace into the log), checks Docker prerequisites (docker installed, Docker daemon running) — whose failures land in the log like any other failure — and sources lib/default_image and exports QUIBBLE_IMAGE (defaulting to QUIBBLE_DEFAULT_IMAGE) and QUIBBLE_VOLUMES, which mount this run's log/<timestamp> directory at /workspace/log so Quibble's own logs land per-run (the directory is stamped unconditionally — even piped invocations need the mount). Self-managed interactive scripts (shellto) skip capture but still get the run directory and a directory-shaped logs: pointer — their container-side logs land there (remove_all is self-managed too and additionally sets _QUIBBLE_NO_LOG_POINTER=1: it deletes log/, so a pointer would name a directory that no longer exists). Utility scripts that must work without Docker (ci, test_unit, lint) source lib/capture directly instead of this file.
Single source of truth for the default Quibble Docker image, as QUIBBLE_DEFAULT_IMAGE. Sourced by lib/setup to seed QUIBBLE_IMAGE for the user-facing scripts, and by generate_examples (which doesn't source lib/setup) to seed the image for its pool workers' remove_dirs fallback. To move to a newer image (e.g. a new Debian base), edit only this file. Override at runtime with QUIBBLE_IMAGE.
Output redirection. Saves all output to a log file inside this run's timestamped directory from lib/log_dir (e.g. log/2026-07-06-14-03-59/fresh_install.log) and prints a dot per line to the terminal. On success the progress dots are the only feedback (optionally followed by the elapsed time when TIME_ELAPSED=1); on failure it prints "FAIL" with the log file path. Registers its exit handler via lib/exit_trap under the shared display key, replacing lib/duration_trap's handler. Sourced by lib/capture.
Creates (or inherits) the per-run timestamped log directory and exports it as QUIBBLE_LOG_DIR. The outermost script stamps log/YYYY-MM-DD-HH-MM-SS once (a same-second collision between two runs gets a -2, -3, ... suffix — names are claimed atomically with mkdir); child scripts, batch steps, pool workers, and nested batches inherit the exported directory and only ensure it exists. Also exports _QUIBBLE_LOG_ROOT (the run-directory root) for user-facing pointers, so a per-worker subdirectory path never leaks into captured examples/. Fails loudly (with ./remove_all remediation) when the directory cannot be created. Sourced by lib/capture, lib/silent_output, lib/batch_setup, and generate_examples.
Makes the sourcing script a captured command: stamps (or inherits) the per-run log directory via lib/log_dir, prints the debug banner with a logs: pointer as the first line (lib/debug_info), redirects all output to the log file with a dot per line on the terminal (lib/silent_output), and enables set -x trace with a custom PS4 so the log contains every command. Skips everything for self-managed interactive scripts (_QUIBBLE_OUTPUT_MANAGED=1, e.g. shellto) and stays passive in piped invocations (output flows through untraced, so a human pipe or GitLab CI running ./lint stays readable) — unless a capturing parent exported _QUIBBLE_TRACE=1 (lib/heartbeat's run_with_dots and the pool workers do), in which case the trace joins the log that parent is writing. Sourced by lib/setup and directly by the utility scripts that must work without Docker (ci, test_unit, lint).
Provides docker_chmod function that sets directories to world-writable (chmod 777), falling back to Docker-as-root when directories are owned by root from previous container runs. Sourced by fresh_install, save, and restore.
Provides the remove_dirs function that removes a list of host directories which may contain files a container created as root. Tries a plain host rm -rf first (the fast path, and all that's needed on macOS, where bind-mounted files are host-user-owned); for root-owned leftovers it falls back to deleting them inside a container running as root, reusing QUIBBLE_IMAGE only if it is already present locally (so it never auto-pulls a multi-GB image just to delete files). If files still remain, it fails loudly with remediation and a non-zero exit rather than leaving a half-cleaned tree that would break the next run. Mirrors lib/docker_chmod's local-first, Docker-as-root-fallback strategy. Sourced by remove, remove_srcs, remove_all, and generate_examples (whose parallel pool workers reset their per-item log directory before dispatching — Quibble containers write to the mounted log dir as a different user, so any leftovers there, e.g. browser-video subdirectories from a rerun that inherited QUIBBLE_LOG_DIR, can be root-owned).
Provides the checkout_branch function: after a Quibble run, checks out the branch named by BRANCH in the MediaWiki repos under src/ (core, vendor, skins, extensions), so they end up on a named branch instead of Quibble's detached HEAD. No-op unless BRANCH is set (and skipped when DRY_RUN=1). Runs the re-attach inside the Quibble image as the file-owning user — so host UID and file ownership don't matter, the same reason ./remove and lib/docker_chmod use Docker; the git logic lives in lib/checkout_branch_worker. Sourced by fresh_install and install.
The git logic run by lib/checkout_branch, executed inside the Quibble Docker container (mounted read-only), not sourced. Walks the MediaWiki layout (core, vendor, skins/*, extensions/*) and, for each repo still in detached HEAD, checks out the requested branch — falling back to the repo's own default branch (origin/HEAD) when it lacks the requested one, mirroring Quibble's fallback. Exits non-zero if any detached repo can't be put on a branch, so an unmet BRANCH request fails the run rather than being silently ignored. Kept as a separate file so it can be unit-tested with Bats on the host without Docker.
Provides the apply_gerrit_patch function: after a Quibble run, fetches the single Gerrit patch named by GERRIT_PATCHES and checks it out in one MediaWiki repo under src/. No-op unless GERRIT_PATCHES is set (and skipped when DRY_RUN=1). The caller passes which repo to patch (core for fresh_install, the named component for install), so there is no repo discovery. Runs the fetch/checkout inside the Quibble image as the file-owning user — so host UID and file ownership don't matter, the same reason ./remove, lib/docker_chmod, and lib/checkout_branch use Docker; the git logic lives in lib/apply_gerrit_patch_worker. Sourced by fresh_install and install.
The git logic run by lib/apply_gerrit_patch, executed inside the Quibble Docker container (mounted read-only), not sourced. Fetches a change ref straight from its project's Gerrit URL and checks it out on a branch named after the ref. Exits non-zero if the ref can't be fetched (wrong ref or wrong target repo), so an unmet GERRIT_PATCHES request fails the run rather than being silently ignored. Kept as a separate file so it can be unit-tested with Bats on the host without Docker (the test fetches over a file:// URL via the GERRIT_BASE_URL override).
Provides print_quibble_command function that pretty-prints a docker command with one logical option per line, framed by a # banner so it stands out in busy logs. Flag-and-value pairs stay on the same line and continuation backslashes make the output a valid copy-pasteable shell command. Sourced by fresh_install, install, shellto, and lib/run_quibble_test.
Parses optional component path (extensions/X or skins/X) and extra arguments from the command line. Sets component, zuul_project, and extra_args variables. Sourced by run_selenium_tests and run_php_unit_tests.
Provides quote_args function that shell-quotes each argument (printf %q) and joins them with spaces, so an argument list can be embedded in a shell command string (Quibble's --command) without the container's shell re-splitting multi-word values — plain "${arr[*]}" would turn --mochaOpts.grep "a b c" into --mochaOpts.grep a b c and let characters like | act as shell operators inside the container. Sourced by run_selenium_tests and run_php_unit_tests.
Provides run_quibble_test function that runs a Quibble test command in Docker with --skip-zuul and --skip-deps. Takes Quibble arguments (e.g. --run selenium). Expects lib/setup and lib/parse_component_args to be sourced first. Sourced by run_selenium_tests and run_php_unit_tests.
Builds the components array from either $1 (single component) or ./list_gated (all gated extensions/skins). Captures ./list_gated's output and checks its exit code, so a broken gated list fails loudly instead of being swallowed by a process-substitution read and silently emptying the run. Sourced by the *_gated batch scripts (every script that processes the gated component list).
Provides clone_or_fetch function that clones or fetches a bare repo from Gerrit into ref/. If the repo already exists, updates its branch refs via lib/fetch_bare; otherwise clones it. Accepts optional --quiet flag. Sourced by install, prepare_gated, selenium_tests_exist, and lib/resolve_deps.
Provides fetch_bare function that updates a bare repo's branch refs from its origin remote. A git clone --bare clone has no remote.origin.fetch refspec, so a plain git fetch only writes FETCH_HEAD and never moves branch refs — everything reading refs from ref/ (lib/ensure_config, git show HEAD:... in selenium_tests_exist and list_dependencies_required) would silently keep serving first-clone state. Fetches with an explicit +refs/heads/*:refs/heads/* refspec and --prune. Accepts optional --quiet flag. Sourced by fetch, prepare, lib/clone_or_fetch, and lib/minimal_setup.
Provides dep_repo_path function that converts a dependency name to a Gerrit repo path. Extensions (e.g. Echo) map to mediawiki/extensions/Echo. Skins (e.g. skins/MinervaNeue) map to mediawiki/skins/MinervaNeue. Sourced by lib/resolve_deps and lib/minimal_setup.
Resolves dependency repos for a component, cloning bare repos as needed. Sets the deps array with repo paths. Reads from QUIBBLE_DEPS env var if set, otherwise from ./list_dependencies — and fails loudly (exit 1) when ./list_dependencies fails, instead of silently resolving zero dependencies. Sourced by install.
Sourced by scripts that need zuul config (list_dependencies, list_gated, install). Ensures the integration/config working copy exists in src/config by cloning from the bare repo.
Sourced by long-running scripts (find_dependencies_minimal_greedy, find_dependencies_minimal_bottom_up, find_dependencies_minimal_gated, find_dependencies_minimal_thorough, find_flaky_selenium_tests, install_all_gated, install_each_gated, run_selenium_tests_all_gated, run_selenium_tests_gated, run_selenium_tests_required_gated, test_integration, test_integration_slow, generate_examples — and thereby generate_examples_slow, which execs generate_examples) to prevent the machine from suspending. Uses caffeinate on macOS and systemd-inhibit on Linux. On Linux it registers its cleanup via lib/exit_trap (under its own key) so a later-sourced EXIT handler such as lib/duration_trap cannot clobber it.
Sourced by lib/batch_setup; batch scripts that track per-item results call it to print the final summary (the find_dependencies_minimal_* algorithm scripts print FOUND blocks instead, except find_dependencies_minimal_gated which tracks per-component results). Provides print_results function that prints pass/fail summary and exits with error if any failures.
Provides utc_timestamp function that prints the current UTC time in YYYY-MM-DD HH:MM:SS UTC format. Gated on the TIME_UTC environment variable: off by default, set TIME_UTC=1 to enable. Sourced by lib/batch_setup.
Provides print_header function that prints a section header for a component in batch scripts: the label followed by a space (dots follow on the same line). Sourced by lib/batch_setup.
Provides record_passed function that records a component as passed (if not already in the failed list). Sourced by lib/batch_setup.
Provides run_test function and test_counter for test_integration-style scripts. Runs a command, prints what it does, and records pass/fail in $passed/$failed. Saves output to a numbered log file in the per-run log directory (e.g. log/<timestamp>/01-help.log) and prints a dot per line. Must be sourced after lib/batch_setup. Sourced by test_integration and test_integration_slow.
Generic dynamic worker pool: keeps $parallel reusable slots busy and refills each slot the instant its item finishes, so one slow item never idles the rest. The caller sets items[] and parallel and defines _run_pool_worker SLOT ITEM INDEX (runs in a background subshell; SLOT is a stable 1..parallel id reused as items complete, for per-slot isolation; INDEX is the item's 0-based position in items[]), plus optional _pool_worker_label (custom per-dispatch progress line) and _pool_reap SLOT ITEM INDEX (called the instant a slot's item finishes, so a caller can surface results live rather than after the pool drains) hooks. A caller may also set _quibble_run_pool_stop (e.g. inside _pool_reap) to stop dispatching new items and drain the in-flight slots — the ordered-search early exit used by lib/parallel. Because bash 3.2 has no wait -n, completion is detected via a per-slot sentinel file written by an EXIT trap inside each worker and polled (interval overridable with _QUIBBLE_POOL_POLL_SECONDS, default 1s); the sentinel temp dir is removed on normal completion and via an INT/TERM trap. Sourced by generate_examples, find_dependencies_minimal_gated, install_each_gated, run_selenium_tests_all_gated, run_selenium_tests_required_gated, and lib/parallel in parallel mode.
Shared pieces for lib/run_pool callers. Sourcing it (inside the parallel-mode branch, before defining the other run_pool hooks) resets _pool_log_dir ($QUIBBLE_LOG_DIR/parallel) — the per-run directory holding one log + result file per item, removed and recreated empty so a prior run's stale files can't pollute this run's results; files are keyed by item (component name or queue index), never by slot, because run_pool reuses slot ids the moment an item finishes — and defines the default _pool_worker_label hook ( Slot N: ITEM (ENVIRONMENT=N)). A caller wanting a different label defines its own after sourcing (later definition wins, as lib/parallel does). Also provides pool_worker_init SLOT LOG_FILE, the shared prologue every _run_pool_worker calls first: exports ENVIRONMENT=$SLOT (per-slot src_N isolation via lib/setup) and QUIBBLE_BACKGROUND=1 (no Docker -i in a background worker), redirects the worker's output to its per-item log, and exports _QUIBBLE_TRACE=1 so children's set -x trace lands there too. It deliberately does not pin QUIBBLE_SAVE — FAST-mode workers that restore the shared save pin it themselves, while find_dependencies_minimal_gated's greedy children save/restore per-slot. Also defines the default _pool_reap hook for component-keyed pools: reads the worker's pass/fail result file (a missing file counts as a failure) and appends to lib/batch_setup's $passed/$failed live, printing ok or FAIL (see <log>) per component; callers with richer per-completion needs define their own after sourcing (run_selenium_tests_required_gated adds a skip state, find_dependencies_minimal_gated surfaces FOUND blocks, lib/parallel tracks the lowest passing combination index). Also provides pool_fast_setup, the FAST-mode prelude for pools whose workers restore the shared save: when FAST is on and the shared save is missing, it runs ./fresh_install + ./save once, serially, before the pool starts (exiting if fresh_install fails); no-op otherwise. Called by install_each_gated and run_selenium_tests_required_gated (run_selenium_tests_all_gated instead errors out, since its core step already creates the save). Sourced by find_dependencies_minimal_gated, install_each_gated, run_selenium_tests_all_gated, run_selenium_tests_required_gated, and lib/parallel.
Bounded-concurrency wave runner: runs independent background jobs $parallel at a time, waiting for each full wave before launching the next and draining the final partial wave. A job that exits non-zero only warns (it never aborts the run). The caller sets items[] and parallel and defines _run_waves_job ITEM (run in the background by the helper). Simpler than lib/run_pool (no dynamic refill, sentinel polling, or per-slot src_N isolation) — a good fit for short, uniform jobs like git clone/fetch. Sourced by prepare (which runs its fixed job set through it at every PARALLEL setting), and by fetch and prepare_gated in parallel mode.
Curated list of the heaviest middle scripts (defines _quibble_heavy_scripts, most-expensive first), used only as a scheduling hint by generate_examples' parallel pool: their Usage lines are dispatched before everything else so the long-running jobs start immediately and the many short jobs backfill the tail, keeping every worker slot busy to the end. Best-effort hint only — lib/run_pool is correct in any order, so a stale or missing entry only costs a little scheduling efficiency. Contrast with lib/slow_scripts below, which is load-bearing. Sourced by generate_examples in parallel mode.
Curated list of the slow scripts (defines _quibble_slow_scripts, most-expensive first) whose Usage lines ./generate_examples (the quick variant) skips — ./generate_examples_slow regenerates them too. Unlike lib/heavy_scripts above (a best-effort scheduling hint), this list is load-bearing: membership decides which examples the quick run regenerates, so add any new slow script here to keep the quick run quick. Membership is per script — every Usage line of a listed script is skipped, so run_php_unit_tests' cheap variants are deliberate collateral of its 13-minute bare line — and the early/late pinned scripts (prepare, prepare_gated, fresh_install, save, remove_srcs, remove, remove_all) must never be listed: they build/destroy the state every run depends on. test/generate_examples.bats enforces that every entry is a real script, that every lib/heavy_scripts entry is listed here too, and that no pinned script is. Sourced by generate_examples.
Awk script that parses requires.extensions and requires.skins from extension.json or skin.json. Used by list_dependencies_required.
Awk script that generates all bitmask combinations of items, ordered by size (starting from 1). With awk -v with_empty=1 the empty set is included first, printed as a blank line (ordered by size starting from 0). Used by list_dependencies_combinations (without the flag), and by find_dependencies_minimal_bottom_up and find_dependencies_minimal_thorough (with it).
Awk script that parses a YAML list under a given key from zuul/dependencies.yaml. Inline YAML comments and trailing whitespace are stripped (- Echo # T429501 prints Echo), so callers always get a bare value. Used by list_dependencies.
Awk script that extracts entries from a Python list assignment in parameter_functions.py. Used by list_gated.
Awk script that extracts the # Usage: block from a script header. Used by generate_examples.
Provides two Bash functions for interpreting a Usage command string: cmd_to_filename converts it to an examples/*.txt filename (e.g. ./install extensions/Echo → examples/install-extensions_echo.txt), and cmd_to_script extracts the name of the script it invokes (the first ./-prefixed token, e.g. PARALLEL=1 ./run_selenium_tests_all_gated extensions/Echo → run_selenium_tests_all_gated) using the same quote-aware tokenization, so the two can never disagree about which script a line belongs to. Sourced by generate_examples.
Awk script that replaces literal occurrences of the project's absolute path ($PWD) with the placeholder $PWD, keeping captured output in examples/*.txt machine-independent so the files don't churn when regenerated on different machines. Used by generate_example.
Awk script that strips ANSI CSI escape sequences (color codes like [33m, cursor moves like [2K) from captured output. Tools running inside the Quibble image (colorlog, composer, npm) emit these when they think their stdout is a terminal; without scrubbing they land in examples/*.txt as literal escape sequences and make the files noisy to read and to diff. Used by generate_example.
Awk script that replaces per-run log/YYYY-MM-DD-HH-MM-SS directory paths (including lib/log_dir's -N same-second collision suffix) with the fixed placeholder log/YYYY-MM-DD-HH-MM-SS. Without it, every regeneration would churn all the logs: pointer and FAIL (see ...) lines in examples/*.txt with that run's timestamp. Fixed paths under log/ (log/find_flaky_selenium_tests/, WDIO result files) are left alone. Used by generate_example.
Awk script that parses WDIO JUnit XML result files into one run<TAB>status<TAB>test line per test case (status: pass, fail, or skip; requires -v run=N to tag the run number). Assumes the constrained JUnit shape @wdio/junit-reporter emits — arbitrary hand-written JUnit XML is out of scope. Used by find_flaky_selenium_tests.
Awk script that aggregates run<TAB>status<TAB>test lines from all runs into the four flaky-report sections (flaky, consistently failing, always passing, always skipped), listing tests in first-seen order with (none) placeholders for empty sections. Used by find_flaky_selenium_tests.
Shared setup for find_dependencies_minimal_greedy, find_dependencies_minimal_bottom_up, and find_dependencies_minimal_thorough. Reads dependencies, classifies them into required/optional, pre-clones bare repos. Sets up fresh_or_restore function and all shared variables.
Prints a summary of all, required, and optional dependencies plus total combinations to test. Sourced by lib/minimal_setup.
Builds a full dependency combination from required + optional deps. Sets the full_combo variable. Sourced by find_dependencies_minimal_bottom_up, find_dependencies_minimal_thorough, lib/greedy, lib/parallel, and lib/print_found.
Defines the fresh_or_restore function: runs ./fresh_install or ./restore depending on FAST mode. In fast mode, saves state after first fresh_install + install so subsequent calls restore instead. Sourced by lib/minimal_setup.
Greedy algorithm for find_dependencies_minimal_greedy: starts with all optional deps, removes one at a time. O(N) instead of O(2^N). Repeats until stable to catch order-dependent removals. Sourced by find_dependencies_minimal_greedy.
Parallel exhaustive search for the minimum dependencies, built on lib/run_pool: dispatches size-ordered combinations across N reusable slots, each in an isolated src_N/ directory (ENVIRONMENT=N). The lowest-index passing combination is the minimum, so it sets lib/run_pool's _quibble_run_pool_stop on the first pass and takes the lowest passing index across the drained in-flight slots as the winner. Defines run_pool's worker/label/reap hooks; cleanup and the INT/TERM trap come from run_pool. Sourced by find_dependencies_minimal_bottom_up and find_dependencies_minimal_thorough when PARALLEL ≥ 1.
Prints the "minimum dependencies found" results (header, required deps, optional deps). Sources lib/build_full_combo to decide whether any dependency is needed at all. Sourced by find_dependencies_minimal_bottom_up, find_dependencies_minimal_thorough, lib/greedy, and lib/parallel.
Awk script that extracts the "FOUND: minimum dependencies" block (produced by lib/print_found) from a find_dependencies_minimal_* log file, stopping before the trailing duration line added by lib/duration_trap. Used by find_dependencies_minimal_gated.