Skip to content

Commit 5b51639

Browse files
add a Development section in the README file
1 parent a335942 commit 5b51639

1 file changed

Lines changed: 48 additions & 0 deletions

File tree

README.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,3 +94,51 @@ __Compile:__
9494

9595

9696
## [Release protocol](https://github.com/typemytype/drawbot/wiki/DrawBot-release-protocol)
97+
98+
---
99+
100+
## Development
101+
102+
### Setup
103+
104+
From the repository root, create a virtual environment, activate it, and install both the package and test dependencies:
105+
106+
python3 -m venv .venv
107+
source .venv/bin/activate
108+
pip install -r requirements.txt -r test-requirements.txt
109+
110+
### Running the tests
111+
112+
From the repository root (with the environment activated):
113+
114+
python tests/runAllTests.py
115+
116+
`runAllTests.py` discovers every `test*.py` module under `tests/`. Individual test modules can also be executed directly (e.g. `python tests/testExport.py`).
117+
118+
The runtime dependencies for the test suite (`Pillow`, `attrs`, `packaging`, `mypy`, `ruff`, `ruff-api`) live in `test-requirements.txt`.
119+
120+
### Benchmarks and tempTestData
121+
122+
`tests/data/` is the benchmark folder. Filenames follow a prefix convention:
123+
124+
- `expected_<name>.<ext>` — reference output for `testScripts.py` and parts of `testExport.py`.
125+
- `example_<name>.png` — reference renders for `testExamples.py` (also used by the docs site).
126+
- no prefix — assets consumed by the test scripts themselves (sample images, fonts, etc.).
127+
128+
Freshly rendered output goes into `tests/tempTestData/` (gitignored). When a comparison fails, that's where you'll find the actual rendered file alongside the expected one.
129+
130+
### Rendering drift across macOS versions and CI
131+
132+
Quartz / Core Image rendering is not bit-identical across macOS versions, so the comparison helpers in `testSupport.py` fall back to a fuzzy image diff with a small tolerance whenever raw byte equality fails. Even so, CI on GitHub Actions might run on a different macOS image than your machine, and some tests may pass locally while failing in CI (or vice versa). To find the exact macOS version GitHub Actions is using, open the workflow webpage, expand the **Set up job** step, and read the `Image OS` / `Runner Image` lines in its log.
133+
134+
To investigate further, use `tests/differenceBuilder.py`. It builds a PDF report visualizing the diff between a folder of freshly rendered images and the benchmarks in `tests/data/`. For each pair it lays out the new render, the expected image tracked in the benchmark folder, a pixel-difference image, and a histogram side by side. Pages where the fuzzy similarity exceeds the tolerance are drawn with a red border so failures are easier to spot. Run it as:
135+
136+
python tests/differenceBuilder.py <folder_with_renders> <output.pdf>
137+
138+
or invoke it with no arguments to pick the folders via a dialog. The resulting PDF is generated on the fly and not committed.
139+
140+
When a test passes locally but fails in CI (or vice versa):
141+
142+
1. Pull the failing artifacts from the Github Actions failed workflow and check the "Drawbot Test Differences" PDF. Keep the "DrawBot Temp Data Results" at hand as well.
143+
2. If the difference between the images is small and clearly attributable to an OS version change (check the overlay and the histogram), the benchmark in `tests/data/` can be updated — but don't blindly copy the CI render over the local one, remember to add the correct prefix as needed.
144+
3. If the difference is large, treat it as a real problem.

0 commit comments

Comments
 (0)