diff --git a/README.rst b/README.rst index c42b5c1..8301b6a 100644 --- a/README.rst +++ b/README.rst @@ -1,402 +1,175 @@ ############################################################################## -Text progress bar library for Python. +progressbar2 ############################################################################## -Build status: +A mature, typed terminal progress bar library for Python scripts that need +custom widgets, clean output around prints and logs, multiple concurrent bars, +unknown-length progress, and pipe-friendly CLI usage. .. image:: https://github.com/WoLpH/python-progressbar/actions/workflows/main.yml/badge.svg :alt: python-progressbar test status :target: https://github.com/WoLpH/python-progressbar/actions -Coverage: - .. image:: https://coveralls.io/repos/WoLpH/python-progressbar/badge.svg?branch=master - :target: https://coveralls.io/r/WoLpH/python-progressbar?branch=master + :alt: coverage status + :target: https://coveralls.io/r/WoLpH/python-progressbar?branch=master -****************************************************************************** Install -****************************************************************************** +============================================================================== -The package can be installed through `pip` (this is the recommended method): +.. code:: sh pip install progressbar2 -Or if `pip` is not available, `easy_install` should work as well: - - easy_install progressbar2 - -Or download the latest release from Pypi (https://pypi.python.org/pypi/progressbar2) or Github. - -Note that the releases on Pypi are signed with my GPG key (https://pgp.mit.edu/pks/lookup?op=vindex&search=0xE81444E9CE1F695D) and can be checked using GPG: - - gpg --verify progressbar2-.tar.gz.asc progressbar2-.tar.gz - -****************************************************************************** -Introduction -****************************************************************************** - -A text progress bar is typically used to display the progress of a long -running operation, providing a visual cue that processing is underway. - -The progressbar is based on the old Python progressbar package that was published on the now defunct Google Code. Since that project was completely abandoned by its developer and the developer did not respond to email, I decided to fork the package. This package is still backwards compatible with the original progressbar package so you can safely use it as a drop-in replacement for existing project. - -The ProgressBar class manages the current progress, and the format of the line -is given by a number of widgets. A widget is an object that may display -differently depending on the state of the progress bar. There are many types -of widgets: - - - `AbsoluteETA `_ - - `AdaptiveETA `_ - - `AdaptiveTransferSpeed `_ - - `AnimatedMarker `_ - - `Bar `_ - - `BouncingBar `_ - - `Counter `_ - - `CurrentTime `_ - - `DataSize `_ - - `DynamicMessage `_ - - `ETA `_ - - `FileTransferSpeed `_ - - `FormatCustomText `_ - - `FormatLabel `_ - - `FormatLabelBar `_ - - `GranularBar `_ - - `Percentage `_ - - `PercentageLabelBar `_ - - `ReverseBar `_ - - `RotatingMarker `_ - - `SimpleProgress `_ - - `SmoothingETA `_ - - `Timer `_ - -The progressbar module is very easy to use, yet very powerful. It will also -automatically enable features like auto-resizing when the system supports it. - -****************************************************************************** -Performance -****************************************************************************** - -The default ``progressbar.progressbar(...)`` is the **fastest** progress bar -available -- on iteration overhead, per-update render cost, *and* import time. -On the benchmark machine (CPython 3.13, macOS arm64): - -============ ==================== ============== =========== -Metric progressbar2 tqdm rich -============ ==================== ============== =========== -Per iter **5 ns** *(fast)* 54 ns 19 ns -Per render **~5 us** *(fast)* 11 us 172 us -Cold import **~1.5 ms** ~22 ms ~47 ms -============ ==================== ============== =========== - -How the default stays fast: - -- **Iteration** -- an integer "next update" gate keeps the common iteration to - an increment and a compare, entering the (rate-limited) redraw machinery only - a few times per second. ~30 ns/iter in pure Python (already faster than - tqdm); ~5 ns with the optional native iterator - (``pip install progressbar2[fast]``), which counts in a C field. -- **Render** -- the default bar uses a fixed formatter (percentage, count, bar, - elapsed/ETA) built directly each redraw, so it renders in ~5 us/update, - roughly **2x faster than tqdm**, without the per-widget overhead. -- **Import** -- ``import progressbar`` is lazy (PEP 562): widgets, the - terminal/colour tables and multi-bar support load only when used, so a bare - import is ~1.5 ms and pulls in nothing heavy (no ``asyncio``). - -The fast path stays close to the classic look but drops the gradient and -per-iteration ``value`` liveness (``value`` is synced at redraw crossings, like -``tqdm.n``). For the full widget set -- gradient ``Bar``, custom widgets, -dynamic variables -- pass ``widgets=[...]`` or ``fast=False`` to -``progressbar()``, or construct ``progressbar.ProgressBar(...)`` directly; that -path is unchanged (and intentionally richer, so a touch slower). Set -``PROGRESSBAR_DISABLE_FASTPATH=1`` to force the classic path everywhere. - -The benchmark is fully reproducible and pits ``progressbar2`` against ``tqdm``, -``rich``, ``alive-progress`` and ``click`` across iteration overhead, forced -redraw cost, and import time -- all rendered to a real pseudo-terminal so the -comparison is apples-to-apples:: - - python benchmarks/bench.py && python benchmarks/report.py - -.. image:: https://raw.githubusercontent.com/WoLpH/python-progressbar/master/benchmarks/chart.png - :alt: progressbar2 vs common Python progress-bar libraries - -****************************************************************************** -Known issues -****************************************************************************** - -- The Jetbrains (PyCharm, etc) editors work out of the box, but for more advanced features such as the `MultiBar` support you will need to enable the "Enable terminal in output console" checkbox in the Run dialog. -- The IDLE editor doesn't support these types of progress bars at all: https://bugs.python.org/issue23220 -- Jupyter notebooks buffer `sys.stdout` which can cause mixed output. This issue can be resolved easily using: `import sys; sys.stdout.flush()`. Linked issue: https://github.com/WoLpH/python-progressbar/issues/173 - -****************************************************************************** -Links -****************************************************************************** - -* Documentation - - https://progressbar-2.readthedocs.org/en/latest/ -* Source - - https://github.com/WoLpH/python-progressbar -* Bug reports - - https://github.com/WoLpH/python-progressbar/issues -* Package homepage - - https://pypi.python.org/pypi/progressbar2 -* My blog - - https://w.wol.ph/ - -****************************************************************************** -Usage -****************************************************************************** - -There are many ways to use Python Progressbar, you can see a few basic examples -here but there are many more in the examples file. - -Wrapping an iterable +Quick start ============================================================================== + .. code:: python import time import progressbar - for i in progressbar.progressbar(range(100)): + for item in progressbar.progressbar(range(100), desc='Loading'): time.sleep(0.02) -Progressbars with logging +Progress with clean logs ============================================================================== -Progressbars with logging require `stderr` redirection _before_ the -`StreamHandler` is initialized. To make sure the `stderr` stream has been -redirected on time make sure to call `progressbar.streams.wrap_stderr()` before -you initialize the `logger`. - -One option to force early initialization is by using the `WRAP_STDERR` -environment variable, on Linux/Unix systems this can be done through: - -.. code:: sh - - # WRAP_STDERR=true python your_script.py - -If you need to flush manually while wrapping, you can do so using: - -.. code:: python - - import progressbar - - progressbar.streams.flush() - -In most cases the following will work as well, as long as you initialize the -`StreamHandler` after the wrapping has taken place. +.. image:: docs/_static/progressbar-hero.svg + :alt: progressbar2 showing clean progress output with logs .. code:: python + import sys import time - import logging import progressbar - progressbar.streams.wrap_stderr() - logging.basicConfig() - - for i in progressbar.progressbar(range(10)): - logging.error('Got %d', i) - time.sleep(0.2) - -Multiple (threaded) progressbars + with progressbar.ProgressBar( + total=24, + desc='Build', + fd=sys.stdout, + redirect_stdout=True, + line_breaks=False, + is_terminal=True, + enable_colors=True, + term_width=112, + ) as bar: + for step in range(24): + if step in {8, 16}: + print(f'log: completed step {step}') + bar.update(step + 1, force=True) + time.sleep(0.005) + +Multiple bars ============================================================================== -.. code:: python +.. image:: docs/_static/progressbar-multibar.svg + :alt: multiple progress bars updating together - import random - import threading - import time +.. code:: python + import io + import re import progressbar - BARS = 5 - N = 50 - - - def do_something(bar): - for i in bar(range(N)): - # Sleep up to 0.1 seconds - time.sleep(random.random() * 0.1) - - # print messages at random intervals to show how extra output works - if random.random() > 0.9: - bar.print('random message for bar', bar, i) - - - with progressbar.MultiBar() as multibar: - for i in range(BARS): - # Get a progressbar - bar = multibar[f'Thread label here {i}'] - # Create a thread and pass the progressbar - threading.Thread(target=do_something, args=(bar,)).start() - -Context wrapper + fd = io.StringIO() + multibar = progressbar.MultiBar( + fd=fd, + total=24, + enable_colors=True, + initial_format=None, + finished_format=None, + remove_finished=None, + sort_reverse=False, + term_width=112, + ) + build = multibar['build'] + test = multibar['test'] + terminal_control_re = re.compile(r'\x1b\[[0-9;]*[A-Za-ln-z]') + + def emit_frame(): + output = terminal_control_re.sub('', fd.getvalue()) + for line in output.split('\r'): + line = line.strip() + if line: + print(line) + print('\f', end='') + fd.seek(0) + fd.truncate(0) + + multibar.render(force=True, flush=True) + emit_frame() + + for step in range(24): + build.update(step + 1, force=True) + test_value = min(24, max(0, round((step - 3) * 1.2))) + test.update(test_value, force=True) + multibar.render(force=True, flush=True) + emit_frame() + +Unknown length and animated bars ============================================================================== -.. code:: python - import time - import progressbar +.. image:: docs/_static/progressbar-unknown-length.svg + :alt: unknown length progress with an animated marker - with progressbar.ProgressBar(max_value=10) as bar: - for i in range(10): - time.sleep(0.1) - bar.update(i) - -Combining progressbars with print output -============================================================================== .. code:: python - import time + import sys import progressbar - for i in progressbar.progressbar(range(100), redirect_stdout=True): - print('Some text', i) - time.sleep(0.1) - -Progressbar with unknown length + with progressbar.ProgressBar( + max_value=progressbar.UnknownLength, + fd=sys.stdout, + line_breaks=False, + is_terminal=True, + enable_colors=True, + term_width=112, + ) as bar: + for value in range(0, 120, 10): + bar.update(value, force=True) + +CLI usage ============================================================================== -.. code:: python - import time - import progressbar +.. code:: sh - bar = progressbar.ProgressBar(max_value=progressbar.UnknownLength) - for i in range(20): - time.sleep(0.1) - bar.update(i) + progressbar --progress --timer --eta --rate --bytes input.bin -o output.bin -Bar with custom widgets +Feature highlights ============================================================================== -.. code:: python - import time - import progressbar +* Works as an iterable wrapper or a manually updated progress bar. +* Supports custom widgets, colors, granular bars, animated markers, and labels. +* Handles unknown-length iterators. +* Supports multiple concurrent progress bars with ``MultiBar``. +* Redirects stdout/stderr so regular output does not corrupt the active bar. +* Includes a pipe-friendly ``progressbar`` command. +* Ships typed package metadata. - widgets=[ - ' [', progressbar.Timer(), '] ', - progressbar.Bar(), - ' (', progressbar.ETA(), ') ', - ] - for i in progressbar.progressbar(range(20), widgets=widgets): - time.sleep(0.1) - -Bar with wide Chinese (or other multibyte) characters +Known terminal caveats ============================================================================== -.. code:: python - - # vim: fileencoding=utf-8 - import time - import progressbar - +* JetBrains IDEs need "Enable terminal in output console" for advanced + terminal behavior such as ``MultiBar``. +* IDLE does not support terminal progress bars. +* Jupyter buffers stdout; call ``sys.stdout.flush()`` when output appears late. - def custom_len(value): - # These characters take up more space - characters = { - '进': 2, - '度': 2, - } - - total = 0 - for c in value: - total += characters.get(c, 1) - - return total - - - bar = progressbar.ProgressBar( - widgets=[ - '进度: ', - progressbar.Bar(), - ' ', - progressbar.Counter(format='%(value)02d/%(max_value)d'), - ], - len_func=custom_len, - ) - for i in bar(range(10)): - time.sleep(0.1) - -Showing multiple independent progress bars in parallel +Project history ============================================================================== -.. code:: python - - import random - import sys - import time - - import progressbar - - BARS = 5 - N = 100 - - # Construct the list of progress bars with the `line_offset` so they draw - # below each other - bars = [] - for i in range(BARS): - bars.append( - progressbar.ProgressBar( - max_value=N, - # We add 1 to the line offset to account for the `print_fd` - line_offset=i + 1, - max_error=False, - ) - ) - - # Create a file descriptor for regular printing as well - print_fd = progressbar.LineOffsetStreamWrapper(lines=0, stream=sys.stdout) +progressbar2 is based on the old Python progressbar package that was published +on the now defunct Google Code. Since that project was completely abandoned by +its developer and the developer did not respond to email, I decided to fork the +package. - # The progress bar updates, normally you would do something useful here - for i in range(N * BARS): - time.sleep(0.005) +This package is still backwards compatible with the original progressbar package +so you can safely use it as a drop-in replacement for existing projects. - # Increment one of the progress bars at random - bars[random.randrange(0, BARS)].increment() - - # Print a status message to the `print_fd` below the progress bars - print(f'Hi, we are at update {i+1} of {N * BARS}', file=print_fd) - - # Cleanup the bars - for bar in bars: - bar.finish() - - # Add a newline to make sure the next print starts on a new line - print() - -****************************************************************************** - -Naturally we can do this from separate threads as well: - -.. code:: python - - import random - import threading - import time - - import progressbar - - BARS = 5 - N = 100 - - # Create the bars with the given line offset - bars = [] - for line_offset in range(BARS): - bars.append(progressbar.ProgressBar(line_offset=line_offset, max_value=N)) - - - class Worker(threading.Thread): - def __init__(self, bar): - super().__init__() - self.bar = bar - - def run(self): - for i in range(N): - time.sleep(random.random() / 25) - self.bar.update(i) - - - for bar in bars: - Worker(bar).start() +Links +============================================================================== - print() +* Documentation: https://progressbar-2.readthedocs.org/en/latest/ +* Source: https://github.com/WoLpH/python-progressbar +* Bug reports: https://github.com/WoLpH/python-progressbar/issues +* Package homepage: https://pypi.python.org/pypi/progressbar2 diff --git a/docs/_static/progressbar-hero.svg b/docs/_static/progressbar-hero.svg new file mode 100644 index 0000000..64e8e9f --- /dev/null +++ b/docs/_static/progressbar-hero.svg @@ -0,0 +1,37 @@ + + + + + + + Progress with clean logs + Build: 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--Build: 4% (1 of 24) |## | Elapsed Time: 0:00:00 ETA: 0:00:00Build: 8% (2 of 24) |#### | Elapsed Time: 0:00:00 ETA: 0:00:00Build: 12% (3 of 24) |###### | Elapsed Time: 0:00:00 ETA: 0:00:00Build: 16% (4 of 24) |######## | Elapsed Time: 0:00:00 ETA: 0:00:00Build: 20% (5 of 24) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00Build: 29% (7 of 24) |############## | Elapsed Time: 0:00:00 ETA: 0:00:00Build: 33% (8 of 24) |################# | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 37% (9 of 24) |################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 41% (10 of 24) |#################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 45% (11 of 24) |###################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 50% (12 of 24) |######################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 54% (13 of 24) |########################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 58% (14 of 24) |############################# | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 62% (15 of 24) |############################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8Build: 66% (16 of 24) |################################# | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 70% (17 of 24) |################################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 75% (18 of 24) |##################################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 83% (20 of 24) |######################################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 87% (21 of 24) |########################################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 91% (22 of 24) |############################################# | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 95% (23 of 24) |############################################### | Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 100% (24 of 24) |##################################################| Elapsed Time: 0:00:00 ETA: 0:00:00log: completed step 8log: completed step 16Build: 100% (24 of 24) |##################################################| Elapsed Time: 0:00:00 Time: 0:00:00 + diff --git a/docs/_static/progressbar-multibar.svg b/docs/_static/progressbar-multibar.svg new file mode 100644 index 0000000..f3e2dd5 --- /dev/null +++ b/docs/_static/progressbar-multibar.svg @@ -0,0 +1,37 @@ + + + + + + + Multiple active jobs + build 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--test 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--build 4% (1 of 24) |# | Elapsed Time: 0:00:00 ETA: 0:00:00test 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--build 8% (2 of 24) |### | Elapsed Time: 0:00:00 ETA: 0:00:00test 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--build 12% (3 of 24) |#### | Elapsed Time: 0:00:00 ETA: 0:00:00test 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--build 16% (4 of 24) |###### | Elapsed Time: 0:00:00 ETA: 0:00:00test 0% (0 of 24) | | Elapsed Time: 0:00:00 ETA: --:--:--build 20% (5 of 24) |####### | Elapsed Time: 0:00:00 ETA: 0:00:00test 4% (1 of 24) |# | Elapsed Time: 0:00:00 ETA: 0:00:00build 25% (6 of 24) |######### | Elapsed Time: 0:00:00 ETA: 0:00:00test 8% (2 of 24) |### | Elapsed Time: 0:00:00 ETA: 0:00:00build 29% (7 of 24) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00test 16% (4 of 24) |###### | Elapsed Time: 0:00:00 ETA: 0:00:00build 33% (8 of 24) |############ | Elapsed Time: 0:00:00 ETA: 0:00:00test 20% (5 of 24) |####### | Elapsed Time: 0:00:00 ETA: 0:00:00build 37% (9 of 24) |############# | Elapsed Time: 0:00:00 ETA: 0:00:00test 25% (6 of 24) |######### | Elapsed Time: 0:00:00 ETA: 0:00:00build 41% (10 of 24) |############### | Elapsed Time: 0:00:00 ETA: 0:00:00test 29% (7 of 24) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00build 45% (11 of 24) |################ | Elapsed Time: 0:00:00 ETA: 0:00:00test 33% (8 of 24) |############ | Elapsed Time: 0:00:00 ETA: 0:00:00build 54% (13 of 24) |################### | Elapsed Time: 0:00:00 ETA: 0:00:00test 45% (11 of 24) |################ | Elapsed Time: 0:00:00 ETA: 0:00:00build 58% (14 of 24) |##################### | Elapsed Time: 0:00:00 ETA: 0:00:00test 50% (12 of 24) |################## | Elapsed Time: 0:00:00 ETA: 0:00:00build 62% (15 of 24) |###################### | Elapsed Time: 0:00:00 ETA: 0:00:00test 54% (13 of 24) |################### | Elapsed Time: 0:00:00 ETA: 0:00:00build 66% (16 of 24) |######################## | Elapsed Time: 0:00:00 ETA: 0:00:00test 58% (14 of 24) |##################### | Elapsed Time: 0:00:00 ETA: 0:00:00build 70% (17 of 24) |######################### | Elapsed Time: 0:00:00 ETA: 0:00:00test 66% (16 of 24) |######################## | Elapsed Time: 0:00:00 ETA: 0:00:00build 75% (18 of 24) |########################### | Elapsed Time: 0:00:00 ETA: 0:00:00test 70% (17 of 24) |######################### | Elapsed Time: 0:00:00 ETA: 0:00:00build 79% (19 of 24) |############################ | Elapsed Time: 0:00:00 ETA: 0:00:00test 75% (18 of 24) |########################### | Elapsed Time: 0:00:00 ETA: 0:00:00build 83% (20 of 24) |############################## | Elapsed Time: 0:00:00 ETA: 0:00:00test 79% (19 of 24) |############################ | Elapsed Time: 0:00:00 ETA: 0:00:00build 87% (21 of 24) |############################### | Elapsed Time: 0:00:00 ETA: 0:00:00test 83% (20 of 24) |############################## | Elapsed Time: 0:00:00 ETA: 0:00:00build 91% (22 of 24) |################################# | Elapsed Time: 0:00:00 ETA: 0:00:00test 91% (22 of 24) |################################# | Elapsed Time: 0:00:00 ETA: 0:00:00build 95% (23 of 24) |################################## | Elapsed Time: 0:00:00 ETA: 0:00:00test 95% (23 of 24) |################################## | Elapsed Time: 0:00:00 ETA: 0:00:00build 100% (24 of 24) |####################################| Elapsed Time: 0:00:00 ETA: 0:00:00test 100% (24 of 24) |####################################| Elapsed Time: 0:00:00 ETA: 0:00:00 + diff --git a/docs/_static/progressbar-unknown-length.svg b/docs/_static/progressbar-unknown-length.svg new file mode 100644 index 0000000..515970c --- /dev/null +++ b/docs/_static/progressbar-unknown-length.svg @@ -0,0 +1,37 @@ + + + + + + + Unknown length + / |# | 0 Elapsed Time: 0:00:00- |# | 0 Elapsed Time: 0:00:00\ |# | 10 Elapsed Time: 0:00:00| |# | 20 Elapsed Time: 0:00:00/ |# | 30 Elapsed Time: 0:00:00- |# | 40 Elapsed Time: 0:00:00\ |# | 50 Elapsed Time: 0:00:00| |# | 60 Elapsed Time: 0:00:00/ |# | 70 Elapsed Time: 0:00:00- |# | 80 Elapsed Time: 0:00:00\ |# | 90 Elapsed Time: 0:00:00| |# | 100 Elapsed Time: 0:00:00/ |# | 110 Elapsed Time: 0:00:00| |# | 110 Elapsed Time: 0:00:00 + diff --git a/docs/examples.rst b/docs/examples.rst index 39b974e..55091dc 100644 --- a/docs/examples.rst +++ b/docs/examples.rst @@ -1,4 +1,8 @@ Examples =================== +The :doc:`usage` guide and README show the generated overview demos. This page +keeps the full runnable example collection in sync with ``examples.py``. + + .. literalinclude:: ../examples.py diff --git a/docs/usage.rst b/docs/usage.rst index 6bbebe2..cdd530a 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -16,6 +16,21 @@ Wrapping an iterable for i in bar(range(100)): time.sleep(0.02) +Tqdm-style options +------------------------------------------------------------------------------ +:: + + import progressbar + + for item in progressbar.progressbar( + range(10), + desc='Items', + total=10, + unit='it', + postfix={'state': 'running'}, + ): + pass + Context wrapper ------------------------------------------------------------------------------ :: @@ -37,10 +52,25 @@ Combining progressbars with print output bar = progressbar.ProgressBar(redirect_stdout=True) for i in range(100): - print 'Some text', i + print('Some text', i) time.sleep(0.1) bar.update(i) +Logging integration +------------------------------------------------------------------------------ +:: + + import logging + import progressbar + + progressbar.streams.wrap_stderr() + progressbar.streams.wrap_logging() + logging.basicConfig() + + with progressbar.ProgressBar(total=10, redirect_stderr=True) as bar: + logging.warning('message above the bar') + bar.update(1) + Progressbar with unknown length ------------------------------------------------------------------------------ :: diff --git a/progressbar/__init__.py b/progressbar/__init__.py index 7f0e04f..00f1c8f 100644 --- a/progressbar/__init__.py +++ b/progressbar/__init__.py @@ -55,11 +55,13 @@ MultiRangeBar, Percentage, PercentageLabelBar, + Postfix, ReverseBar, RotatingMarker, SimpleProgress, SmoothingETA, Timer, + UnitProgress, Variable, VariableMixin, ) @@ -116,12 +118,14 @@ 'MultiProgressBar': 'widgets', 'MultiRangeBar': 'widgets', 'Percentage': 'widgets', + 'Postfix': 'widgets', 'PercentageLabelBar': 'widgets', 'ReverseBar': 'widgets', 'RotatingMarker': 'widgets', 'SimpleProgress': 'widgets', 'SmoothingETA': 'widgets', 'Timer': 'widgets', + 'UnitProgress': 'widgets', 'Variable': 'widgets', 'VariableMixin': 'widgets', } @@ -179,6 +183,7 @@ def __dir__() -> list[str]: 'NullBar', 'Percentage', 'PercentageLabelBar', + 'Postfix', 'ProgressBar', 'ReverseBar', 'RotatingMarker', @@ -187,6 +192,7 @@ def __dir__() -> list[str]: 'SmoothingETA', 'SortKey', 'Timer', + 'UnitProgress', 'UnknownLength', 'Variable', 'VariableMixin', diff --git a/progressbar/__main__.py b/progressbar/__main__.py index a59f7d5..9523544 100644 --- a/progressbar/__main__.py +++ b/progressbar/__main__.py @@ -4,6 +4,7 @@ import contextlib import pathlib import sys +import time import typing from pathlib import Path from typing import IO, BinaryIO, TextIO @@ -271,7 +272,82 @@ def create_argument_parser() -> argparse.ArgumentParser: return parser -def main(argv: list[str] | None = None) -> None: +def _default_widgets( + filesize_available: bool, +) -> list[progressbar.widgets.WidgetBase | str]: + if filesize_available: + return [ + progressbar.Percentage(), + ' ', + progressbar.Bar(), + ' ', + progressbar.Timer(), + ' ', + progressbar.FileTransferSpeed(), + ] + return [ + progressbar.SimpleProgress(), + ' ', + progressbar.DataSize(), + ' ', + progressbar.Timer(), + ] + + +def _append_widget_group( + widgets: list[progressbar.widgets.WidgetBase | str], + group: typing.Sequence[progressbar.widgets.WidgetBase | str], +) -> None: + if widgets: + widgets.append(' ') + widgets.extend(group) + + +def _build_widgets( + args: argparse.Namespace, + filesize_available: bool, +) -> list[progressbar.widgets.WidgetBase | str]: + if args.quiet: + return [] + + requested_widgets = [ + (args.progress, [progressbar.Percentage(), ' ', progressbar.Bar()]), + (args.bytes, [progressbar.DataSize()]), + (args.timer, [progressbar.Timer()]), + (args.eta, [progressbar.AdaptiveETA()]), + (args.fineta, [progressbar.AbsoluteETA()]), + (args.rate or args.average_rate, [progressbar.FileTransferSpeed()]), + ] + selected_widgets = [ + group for selected, group in requested_widgets if selected + ] + if not selected_widgets: + return _default_widgets(filesize_available) + + widgets: list[progressbar.widgets.WidgetBase | str] = [] + for group in selected_widgets: + _append_widget_group(widgets, group) + + return widgets + + +def _sleep_for_rate_limit( + rate_limit: int | None, + transferred: int, + started_at: float, + now: float | None = None, +) -> None: + if not rate_limit: + return + now = time.monotonic() if now is None else now + expected_elapsed = transferred / rate_limit + actual_elapsed = now - started_at + delay = expected_elapsed - actual_elapsed + if delay > 0: + time.sleep(delay) + + +def main(argv: list[str] | None = None) -> None: # noqa: C901 """ Main function for the `progressbar` command. @@ -289,162 +365,104 @@ def main(argv: list[str] | None = None) -> None: args.output, args.line_mode, stack ) - input_paths, total_size, filesize_available = _resolve_inputs( - args, parser - ) - widgets = _build_widgets(args, filesize_available) - - # Initialize the progress bar - bar = progressbar.ProgressBar( - widgets=widgets, - max_value=total_size if filesize_available else None, - max_error=False, - ) - - _transfer(bar, input_paths, output_stream, args, stack) - - -def _resolve_inputs( - args: argparse.Namespace, - parser: argparse.ArgumentParser, -) -> tuple[list[BinaryIO | TextIO | Path | IO[typing.Any]], int, bool]: - """ - Resolve the input arguments into concrete streams/paths and the total size. + input_paths: list[BinaryIO | TextIO | Path | IO[typing.Any]] = [] + total_size: int = 0 + filesize_available: bool = True + for filename in args.input: + input_path: typing.IO[typing.Any] | pathlib.Path + if filename == '-': + if args.line_mode: + input_path = sys.stdin + else: + input_path = sys.stdin.buffer - Returns the list of inputs (stdin streams or file paths), the total size in - bytes and whether that size is known (``filesize_available``). - """ - input_paths: list[BinaryIO | TextIO | Path | IO[typing.Any]] = [] - total_size: int = 0 - filesize_available: bool = True - for filename in args.input: - input_path: typing.IO[typing.Any] | pathlib.Path - if filename == '-': - if args.line_mode: - input_path = sys.stdin + filesize_available = False else: - input_path = sys.stdin.buffer - - filesize_available = False - else: - input_path = pathlib.Path(filename) - if not input_path.exists(): - parser.error(f'File not found: {filename}') - - if not args.size: - total_size += input_path.stat().st_size - - input_paths.append(input_path) - - # An explicit ``--size`` overrides the detected file sizes entirely. - if args.size: - total_size = size_to_bytes(args.size) - filesize_available = True - - return input_paths, total_size, filesize_available - - -def _build_widgets( - args: argparse.Namespace, - filesize_available: bool, -) -> list[typing.Any]: - """ - Select the widget set for the progress bar. - - When the total size is known a percentage/bar layout is used, otherwise a - size-based layout is used. An adaptive ETA is appended when requested. - """ - widgets: list[typing.Any] - if filesize_available: - # Create the progress bar components - widgets = [ - progressbar.Percentage(), - ' ', - progressbar.Bar(), - ' ', - progressbar.Timer(), - ' ', - progressbar.FileTransferSpeed(), - ] - else: - widgets = [ - progressbar.SimpleProgress(), - ' ', - progressbar.DataSize(), - ' ', - progressbar.Timer(), - ] + input_path = pathlib.Path(filename) + if not input_path.exists(): + parser.error(f'File not found: {filename}') - if args.eta: - widgets.append(' ') - widgets.append(progressbar.AdaptiveETA()) + if not args.size: + total_size += input_path.stat().st_size - return widgets + input_paths.append(input_path) + # Determine the size for the progress bar (if provided) + if args.size: + total_size = size_to_bytes(args.size) + filesize_available = True -def _transfer( - bar: progressbar.ProgressBar, - input_paths: list[BinaryIO | TextIO | Path | IO[typing.Any]], - output_stream: typing.IO[typing.Any], - args: argparse.Namespace, - stack: contextlib.ExitStack, -) -> None: - """ - Copy every input through the progress bar into ``output_stream``. + widgets = _build_widgets(args, filesize_available) + progress_bar_class: type[progressbar.ProgressBar] = ( + progressbar.NullBar if args.quiet else progressbar.ProgressBar + ) - Opened files are registered on ``stack`` so they are closed when the caller - exits its ``ExitStack`` context. - """ - # Data processing and updating the progress bar - buffer_size = size_to_bytes(args.buffer_size) if args.buffer_size else 1024 - total_transferred = 0 - - bar.start() - with contextlib.suppress(KeyboardInterrupt, BrokenPipeError): - for input_path in input_paths: - if isinstance(input_path, pathlib.Path): - if args.line_mode: - # newline='' disables universal-newline - # translation so the byte count matches the file - # size for CRLF files as well - input_stream = stack.enter_context( - input_path.open('r', newline=''), - ) - else: - input_stream = stack.enter_context( - input_path.open('rb'), - ) - else: - input_stream = input_path + # Initialize the progress bar + bar = progress_bar_class( + widgets=widgets, + max_value=total_size if filesize_available else None, + max_error=False, + line_breaks=True if args.numeric else None, + ) - while True: - data: str | bytes - if args.line_mode: - data = input_stream.readline(buffer_size) + # Data processing and updating the progress bar + buffer_size = ( + size_to_bytes(args.buffer_size) if args.buffer_size else 1024 + ) + rate_limit = ( + size_to_bytes(args.rate_limit) if args.rate_limit else None + ) + started_at = time.monotonic() + total_transferred = 0 + + bar.start() + with contextlib.suppress(KeyboardInterrupt, BrokenPipeError): + for input_path in input_paths: + if isinstance(input_path, pathlib.Path): + if args.line_mode: + # newline='' disables universal-newline + # translation so the byte count matches the file + # size for CRLF files as well + input_stream = stack.enter_context( + input_path.open('r', newline=''), + ) + else: + input_stream = stack.enter_context( + input_path.open('rb'), + ) else: - data = input_stream.read(buffer_size) - - if not data: - break - - output_stream.write(data) - if isinstance(data, str): - # The total size is measured in bytes, so progress - # must be tracked in bytes as well - encoding = ( - getattr(input_stream, 'encoding', None) or 'utf-8' - ) - total_transferred += len( - data.encode(encoding, errors='replace'), + input_stream = input_path + + while True: + data: str | bytes + if args.line_mode: + data = input_stream.readline(buffer_size) + else: + data = input_stream.read(buffer_size) + + if not data: + break + + output_stream.write(data) + if isinstance(data, str): + # The total size is measured in bytes, so progress + # must be tracked in bytes as well + encoding = ( + getattr(input_stream, 'encoding', None) or 'utf-8' + ) + total_transferred += len( + data.encode(encoding, errors='replace'), + ) + else: + total_transferred += len(data) + + bar.update(total_transferred) + _sleep_for_rate_limit( + rate_limit, + total_transferred, + started_at, ) - else: - total_transferred += len(data) - - bar.update(total_transferred) - # Inside the suppress block on purpose (matching the historical - # behavior): on interrupt/broken pipe the finish is skipped and a - # BrokenPipeError from a closed stderr cannot crash shutdown. bar.finish(dirty=True) diff --git a/progressbar/bar.py b/progressbar/bar.py index 4643b76..7cf1622 100644 --- a/progressbar/bar.py +++ b/progressbar/bar.py @@ -579,13 +579,18 @@ def update(self, value: NumberT | None = None): super().update(value=value) def finish(self, end='\n'): - super().finish(end=end) - utils.streams.stop_capturing(self) - if self.redirect_stdout: - utils.streams.unwrap_stdout() + try: + super().finish(end=end) + finally: + # Always release the global stream-wrapping state, even when + # the final render raises; a leaked listener would corrupt + # every later progressbar in the process. + utils.streams.stop_capturing(self) + if self.redirect_stdout: + utils.streams.unwrap_stdout() - if self.redirect_stderr: - utils.streams.unwrap_stderr() + if self.redirect_stderr: + utils.streams.unwrap_stderr() class ProgressBar( @@ -687,6 +692,11 @@ def __init__( suffix=None, variables=None, min_poll_interval=None, + desc: str | None = None, + total: ValueT = None, + unit: str = 'it', + unit_scale: bool = False, + postfix: typing.Any = None, **kwargs, ): """Initializes a progress bar with sane defaults.""" @@ -696,6 +706,13 @@ def __init__( max_value, poll_interval, kwargs ) + if max_value is None and total is not None: + # tqdm-style alias for `max_value` + max_value = total + if prefix is None and desc is not None: + # tqdm-style alias for `prefix` + prefix = f'{desc}: ' + if max_value and min_value > typing.cast(NumberT, max_value): raise ValueError( 'Max value needs to be bigger than the min value', @@ -708,6 +725,13 @@ def __init__( self.widgets = self._copy_widgets(widgets) + self.unit = unit + self.unit_scale = unit_scale + # Auto-append a Postfix widget in start() when `postfix` is used + # with the default widgets; explicit widget lists are left alone. + self._auto_postfix = widgets is None and postfix is not None + self._auto_postfix_added = False + self.prefix = prefix self.suffix = suffix self.widget_kwargs = widget_kwargs or {} @@ -720,6 +744,8 @@ def __init__( self._setup_poll_intervals(poll_interval, min_poll_interval) self._seed_variables(variables) + if postfix is not None: + self.variables['postfix'] = postfix def _apply_deprecated_aliases( self, @@ -964,6 +990,8 @@ def data(self) -> dict[str, typing.Any]: time_elapsed=elapsed, # Percentage as a float or `None` if no max_value is available percentage=self.percentage, + unit=self.unit, + unit_scale=self.unit_scale, # Dictionary of user-defined # :py:class:`progressbar.widgets.Variable`'s variables=self.variables, @@ -1368,9 +1396,13 @@ def start( if self.max_value is None: self.max_value = self._DEFAULT_MAXVAL - # Constructing the default widgets is only done when we know max_value + # Constructing the default widgets is only done when we know + # max_value if not self.widgets: self.widgets = self.default_widgets() + if self._auto_postfix and not self._auto_postfix_added: + self.widgets.append(_load_widgets().Postfix()) + self._auto_postfix_added = True self._init_prefix() self._init_suffix() @@ -1380,30 +1412,40 @@ def start( or not self.min_poll_interval ): self._gate_enabled = False - self._verify_max_value() - - # Timing state must be populated before `_started` becomes - # observable: a concurrent reader (MultiBar's render thread) that - # sees `started()` True calls `update(force=True)`, and `update()` - # re-enters `start()` whenever `start_time` is still None -- running - # the stream-capturing path twice. - now = datetime.now() - self.start_time = self.initial_start_time or now - self.last_update_time = now - self._last_update_timer = timeit.default_timer() - # Cooperative dispatch through the MRO - # (StdRedirectMixin -> DefaultFdMixin -> ProgressBarMixinBase); - # ResizableMixin/ProgressBarBase define no `start` and are skipped. - # This runs *after* all widget/state setup so that `_started` (set by - # ProgressBarMixinBase.start) only becomes observable once `widgets` - # is fully populated. Otherwise a concurrent reader (e.g. MultiBar's - # render thread) could see `started()` True with an empty widget list - # crash in `_label_bar`'s `assert bar.widgets`. The 0% draw below - # still happens at the same point, after stream/console setup. - super().start(max_value=max_value) - - self.update(self.min_value, force=True) + try: + self._verify_max_value() + + # Timing state must be populated before `_started` becomes + # observable: a concurrent reader (MultiBar's render thread) that + # sees `started()` True calls `update(force=True)`, and `update()` + # re-enters `start()` whenever `start_time` is still None -- + # running the stream-capturing path twice. + now = datetime.now() + self.start_time = self.initial_start_time or now + self.last_update_time = now + self._last_update_timer = timeit.default_timer() + + # Cooperative dispatch through the MRO + # (StdRedirectMixin -> DefaultFdMixin -> ProgressBarMixinBase); + # ResizableMixin/ProgressBarBase define no `start` and are + # skipped. This runs *after* all widget/state setup so that + # `_started` (set by ProgressBarMixinBase.start) only becomes + # observable once `widgets` is fully populated. Otherwise a + # concurrent reader (e.g. MultiBar's render thread) could see + # `started()` True with an empty widget list and crash in + # `_label_bar`'s `assert bar.widgets`. The 0% draw below still + # happens at the same point, after stream/console setup. + super().start(max_value=max_value) + + self.update(self.min_value, force=True) + except Exception: + # A failed start must not leak global stream-wrapping state + # (registered listeners, redirected stdout/stderr): run the + # finish chain suppressed and re-raise the original error. + with contextlib.suppress(Exception): + super().finish(end='') + raise return self @@ -1469,17 +1511,21 @@ def finish(self, end: str = '\n', dirty: bool = False): # state, so extra calls are no-ops return - if not dirty: - self.end_time = datetime.now() - self.update(self.max_value, force=True) - - # Cooperative dispatch through the MRO - # (StdRedirectMixin -> DefaultFdMixin -> ResizableMixin -> - # ProgressBarMixinBase). Ordering note: the SIGWINCH uninstall in - # ResizableMixin.finish now runs *before* the stream unwrap in - # StdRedirectMixin.finish (previously it ran after). The two - # subsystems are independent, so the observable result is unchanged. - super().finish(end=end) + try: + if not dirty: + self.end_time = datetime.now() + self.update(self.max_value, force=True) + finally: + # Run the finish chain even when the final render raises, so a + # failing widget cannot leak the global stream-wrapping state. + # Cooperative dispatch through the MRO + # (StdRedirectMixin -> DefaultFdMixin -> ResizableMixin -> + # ProgressBarMixinBase). Ordering note: the SIGWINCH uninstall in + # ResizableMixin.finish now runs *before* the stream unwrap in + # StdRedirectMixin.finish (previously it ran after). The two + # subsystems are independent, so the observable result is + # unchanged. + super().finish(end=end) @property def currval(self): diff --git a/progressbar/multi.py b/progressbar/multi.py index 8f1a0c2..0e0628d 100644 --- a/progressbar/multi.py +++ b/progressbar/multi.py @@ -106,8 +106,11 @@ class MultiBar(dict[str, bar.ProgressBar]): def __init__( self, - bars: collections.abc.Iterable[tuple[str, bar.ProgressBar]] - | None = None, + bars: ( + collections.abc.Mapping[str, bar.ProgressBar] + | collections.abc.Iterable[tuple[str, bar.ProgressBar]] + | None + ) = None, fd: typing.TextIO = sys.stderr, prepend_label: bool = True, append_label: bool = False, @@ -161,17 +164,36 @@ def __init__( self._thread_finished = threading.Event() self._thread_closed = threading.Event() - super().__init__(bars or {}) + super().__init__() + + bar_items: typing.Iterable[tuple[str, bar.ProgressBar]] + if bars is None: + bar_items = () + elif isinstance(bars, collections.abc.Mapping): + bar_items = typing.cast( + typing.Iterable[tuple[str, bar.ProgressBar]], + bars.items(), + ) + else: + bar_items = bars + + for key, progress in bar_items: + self[key] = progress def __setitem__(self, key: str, bar: bar.ProgressBar) -> None: """Add a progressbar to the multibar.""" if bar.label != key or not key: # pragma: no branch bar.label = key + + if not ( + isinstance(bar.fd, stream.LastLineStream) + and bar.fd.stream is self.fd + ): bar.fd = stream.LastLineStream(self.fd) - bar.paused = True - # Essentially `bar.print = self.print`, but `mypy` doesn't - # like that - bar.print = self.print # type: ignore + + bar.paused = True + # Essentially `bar.print = self.print`, but `mypy` doesn't like that + bar.print = self.print # type: ignore # Just in case someone is using a progressbar with a custom # constructor and forgot to call the super constructor @@ -282,7 +304,7 @@ def update( yield from self._render_finished_bar(bar_, now, expired, update) elif bar_.started(): - update() + yield update() else: if self.initial_format is None: bar_.start() @@ -315,7 +337,7 @@ def _render_finished_bar( if bar_.finished(): # pragma: no branch if self.finished_format is None: - update(force=False) + yield update(force=False) else: # pragma: no cover yield self.finished_format.format(label=bar_.label) diff --git a/progressbar/shortcuts.py b/progressbar/shortcuts.py index df72eba..459c0f7 100644 --- a/progressbar/shortcuts.py +++ b/progressbar/shortcuts.py @@ -24,14 +24,25 @@ def progressbar( prefix: str | None = None, suffix: str | None = None, fast: bool | None = None, + desc: str | None = None, + total: bar.ValueT = None, + unit: str = 'it', + unit_scale: bool = False, + postfix: typing.Any = None, **kwargs: typing.Any, ) -> collections.abc.Iterator[T]: # Auto-dispatch to the lean FastProgressBar for the simple, common case; - # anything that needs the full widget machinery uses ProgressBar. + # anything that needs the full widget machinery uses ProgressBar. The + # tqdm-style `desc` (a prefix) and `total` (a max_value) render fine on + # the fast path, but units and postfixes are widgets and need the full + # bar. use_fast = ( widgets is None and fast is not False and not kwargs.get('variables') + and unit == 'it' + and not unit_scale + and postfix is None and not os.environ.get('PROGRESSBAR_DISABLE_FASTPATH') ) cls = fast_module.FastProgressBar if use_fast else bar.ProgressBar @@ -41,6 +52,11 @@ def progressbar( widgets=widgets, prefix=prefix, suffix=suffix, + desc=desc, + total=total, + unit=unit, + unit_scale=unit_scale, + postfix=postfix, **kwargs, ) return iter(progressbar_(iterator)) diff --git a/progressbar/utils.py b/progressbar/utils.py index cb98994..affb15c 100644 --- a/progressbar/utils.py +++ b/progressbar/utils.py @@ -9,7 +9,7 @@ import re import sys import typing -from collections.abc import Callable, Iterable, Iterator +from collections.abc import Callable, Iterable, Iterator, Mapping from types import TracebackType from python_utils import types @@ -287,7 +287,9 @@ class StreamWrapper: ] wrapped_stdout: int = 0 wrapped_stderr: int = 0 + wrapped_logging: int = 0 wrapped_excepthook: int = 0 + logging_handlers: list[tuple[logging.StreamHandler[base.IO], base.IO]] capturing: int = 0 listeners: set[_ProgressListener] @@ -297,7 +299,9 @@ def __init__(self) -> None: self.original_excepthook = sys.excepthook self.wrapped_stdout = 0 self.wrapped_stderr = 0 + self.wrapped_logging = 0 self.wrapped_excepthook = 0 + self.logging_handlers = [] self.capturing = 0 self.listeners = set() @@ -363,6 +367,80 @@ def wrap_stderr(self) -> WrappingIO: return sys.stderr # type: ignore + def wrap_logging(self) -> None: + """Retarget stdout/stderr logging handlers to wrapped streams.""" + self.wrapped_logging += 1 + if self.wrapped_logging > 1: + return + + wrapped_streams = { + self.original_stdout: self.stdout, + self.original_stderr: self.stderr, + sys.stdout: self.stdout, + sys.stderr: self.stderr, + } + restore_streams: dict[object, base.IO] = {} + if isinstance(self.stdout, WrappingIO): + restore_streams[self.stdout] = self.original_stdout + if isinstance(self.stderr, WrappingIO): + restore_streams[self.stderr] = self.original_stderr + + seen: set[int] = set() + for logger_ in self._iter_loggers(): + for handler in tuple(logger_.handlers): + if id(handler) in seen: + continue + seen.add(id(handler)) + if not isinstance(handler, logging.StreamHandler): + continue + self._wrap_logging_handler( + handler, + wrapped_streams, + restore_streams, + ) + + def _wrap_logging_handler( + self, + handler: logging.StreamHandler[base.IO], + wrapped_streams: Mapping[types.Any, types.Any], + restore_streams: Mapping[types.Any, base.IO], + ) -> None: + stream = handler.stream + replacement = wrapped_streams.get(stream) + if replacement is not None and replacement is not stream: + if self._set_handler_stream(handler, replacement): + self.logging_handlers.append((handler, stream)) + elif (restore_stream := restore_streams.get(stream)) is not None: + self.logging_handlers.append((handler, restore_stream)) + + def unwrap_logging(self) -> None: + if self.wrapped_logging > 1: + self.wrapped_logging -= 1 + return + if not self.wrapped_logging: + return + + while self.logging_handlers: + handler, stream = self.logging_handlers.pop() + self._set_handler_stream(handler, stream) + self.wrapped_logging = 0 + + def _set_handler_stream( + self, + handler: logging.StreamHandler[base.IO], + stream: types.Any, + ) -> bool: + with contextlib.suppress(AttributeError, ValueError): + handler.setStream(stream) + return True + return False + + def _iter_loggers(self) -> types.Iterator[logging.Logger]: + yield logging.getLogger() + for logger_ in tuple(logging.Logger.manager.loggerDict.values()): + if isinstance(logger_, logging.Logger): + yield logger_ + def unwrap_excepthook(self) -> None: if self.wrapped_excepthook: self.wrapped_excepthook -= 1 diff --git a/progressbar/widgets.py b/progressbar/widgets.py index eae0b7b..73d33e7 100644 --- a/progressbar/widgets.py +++ b/progressbar/widgets.py @@ -1019,6 +1019,55 @@ def get_format( return self._apply_colors(output, data) +UNIT_PREFIXES = ('', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi') +DEFAULT_UNIT = object() + + +def format_unit_value(value, unit='it', unit_scale=False) -> str: + if value in (None, base.UnknownLength): + return 'N/A' + if unit_scale: + scaled, power = utils.scale_1024(float(value), len(UNIT_PREFIXES)) + prefix = UNIT_PREFIXES[int(power)] + return f'{scaled:.1f} {prefix}{unit}' + if isinstance(value, float): + return f'{value:g} {unit}' + return f'{value} {unit}' + + +class UnitProgress(WidgetBase): + """Displays progress as a count with an optional unit and 1024 scaling.""" + + def __init__( + self, + unit=DEFAULT_UNIT, + unit_scale=DEFAULT_UNIT, + **kwargs: typing.Any, + ): + self.use_progress_unit = unit is DEFAULT_UNIT + self.use_progress_unit_scale = unit_scale is DEFAULT_UNIT + self.unit: str = ( + 'it' if unit is DEFAULT_UNIT else typing.cast(str, unit) + ) + self.unit_scale: bool = ( + False + if unit_scale is DEFAULT_UNIT + else typing.cast(bool, unit_scale) + ) + WidgetBase.__init__(self, **kwargs) + + def __call__(self, progress: ProgressBarMixinBase, data: Data) -> str: + unit = typing.cast(str, data.get('unit', self.unit)) + unit_scale = typing.cast(bool, data.get('unit_scale', self.unit_scale)) + if not self.use_progress_unit: + unit = self.unit + if not self.use_progress_unit_scale: + unit_scale = self.unit_scale + value = format_unit_value(data.get('value'), unit, unit_scale) + max_value = format_unit_value(data.get('max_value'), unit, unit_scale) + return f'{value} of {max_value}' + + class SimpleProgress(FormatWidgetMixin, ColoredMixin, WidgetBase): """Returns progress as a count of the total (e.g.: "5 of 47").""" @@ -1289,6 +1338,38 @@ def __init__(self, name, **kwargs: typing.Any): super().__init__(**kwargs) +class Postfix(VariableMixin, WidgetBase): + """Displays a live postfix string or key-value mapping.""" + + def __init__( + self, + name='postfix', + prefix=' ', + separator=', ', + **kwargs: typing.Any, + ): + self.prefix = prefix + self.separator = separator + VariableMixin.__init__(self, name=name) + WidgetBase.__init__(self, **kwargs) + + def __call__(self, progress: ProgressBarMixinBase, data: Data) -> str: + value = data['variables'].get(self.name) + if value is None or ( + isinstance(value, (str, dict, list, set, tuple)) and not value + ): + return '' + if isinstance(value, str): + rendered = value + elif isinstance(value, dict): + rendered = self.separator.join( + f'{key}={value[key]}' for key in sorted(value) + ) + else: + rendered = str(value) + return f'{self.prefix}{rendered}' + + class MultiRangeBar(Bar, VariableMixin): """ A bar with multiple sub-ranges, each represented by a different symbol. diff --git a/scripts/__init__.py b/scripts/__init__.py new file mode 100644 index 0000000..6937790 --- /dev/null +++ b/scripts/__init__.py @@ -0,0 +1 @@ +"""Helper scripts for maintainers.""" diff --git a/scripts/render_readme_demos.py b/scripts/render_readme_demos.py new file mode 100644 index 0000000..f614350 --- /dev/null +++ b/scripts/render_readme_demos.py @@ -0,0 +1,482 @@ +from __future__ import annotations + +import argparse +import html +import os +import re +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +STATIC_DIR = ROOT / 'docs' / '_static' +TIMING_FIELD_RE = re.compile( + r'\b(Elapsed Time|ETA|Time):(\s+)\d+:\d{2}:\d{2}', +) +BAR_RE = re.compile(r'\|(?P(?:#+[\s#]*|))\|') +PERCENT_RE = re.compile(r'\b\d{1,3}%') +POSTFIX_RE = re.compile(r'\b[A-Za-z_][\w-]*=[^\s,]+') +LABEL_RE = re.compile(r'[A-Za-z][\w-]*:?') +ANSI_SGR_RE = re.compile(r'\x1b\[([0-9;]*)m') +ANIMATION_FRAME_SECONDS = 0.08 +MAX_ANIMATION_FRAMES = 24 +SVG_WIDTH = 1080 + + +@dataclass(frozen=True) +class Demo: + name: str + title: str + snippet: str + log_lines: int = 0 + + +DEMOS = [ + Demo( + 'hero', + 'Progress with clean logs', + """ +import sys +import time +import progressbar + +with progressbar.ProgressBar( + total=24, + desc='Build', + fd=sys.stdout, + redirect_stdout=True, + line_breaks=False, + is_terminal=True, + enable_colors=True, + term_width=112, +) as bar: + for step in range(24): + if step in {8, 16}: + print(f'log: completed step {step}') + bar.update(step + 1, force=True) + time.sleep(0.005) +""", + log_lines=2, + ), + Demo( + 'multibar', + 'Multiple active jobs', + """ +import io +import re +import progressbar + +fd = io.StringIO() +multibar = progressbar.MultiBar( + fd=fd, + total=24, + enable_colors=True, + initial_format=None, + finished_format=None, + remove_finished=None, + sort_reverse=False, + term_width=112, +) +build = multibar['build'] +test = multibar['test'] +terminal_control_re = re.compile(r'\\x1b\\[[0-9;]*[A-Za-ln-z]') + +def emit_frame(): + output = terminal_control_re.sub('', fd.getvalue()) + for line in output.split('\\r'): + line = line.strip() + if line: + print(line) + print('\\f', end='') + fd.seek(0) + fd.truncate(0) + +multibar.render(force=True, flush=True) +emit_frame() + +for step in range(24): + build.update(step + 1, force=True) + test_value = min(24, max(0, round((step - 3) * 1.2))) + test.update(test_value, force=True) + multibar.render(force=True, flush=True) + emit_frame() +""", + ), + Demo( + 'unknown-length', + 'Unknown length', + """ +import sys +import progressbar + +with progressbar.ProgressBar( + max_value=progressbar.UnknownLength, + fd=sys.stdout, + line_breaks=False, + is_terminal=True, + enable_colors=True, + term_width=112, +) as bar: + for value in range(0, 120, 10): + bar.update(value, force=True) +""", + ), +] + + +def capture_demo(demo: Demo) -> list[list[str]]: + env = os.environ.copy() + env['COLORFGBG'] = '15;0' + env['COLORTERM'] = 'truecolor' + env['PYTHONPATH'] = str(ROOT) + env['PYTHONIOENCODING'] = 'utf-8' + try: + result = subprocess.run( + [sys.executable, '-c', demo.snippet], + cwd=ROOT, + env=env, + text=True, + encoding='utf-8', + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + check=True, + timeout=5, + ) + except subprocess.TimeoutExpired as error: + raise SystemExit(f'timed out capturing demo: {demo.name}') from error + + frames = parse_frames(result.stdout) + if demo.log_lines: + frames = keep_recent_logs_with_progress(frames, demo.log_lines) + return limit_animation_frames(frames) or [['No output captured']] + + +def normalize_terminal_line(line: str) -> str: + return TIMING_FIELD_RE.sub( + lambda match: f'{match.group(1)}:{match.group(2)}0:00:00', + line, + ) + + +def frames_from_output(output: str) -> list[list[str]]: + return limit_animation_frames(parse_frames(output)) or [ + ['No output captured'] + ] + + +def parse_frames(output: str) -> list[list[str]]: + frames: list[list[str]] = [] + output = output.replace('\x1b[2K', '') + if '\f' in output: + for raw_frame in output.split('\f'): + lines = [ + normalize_terminal_line(line.strip()) + for line in raw_frame.splitlines() + if line.strip() + ] + if lines: + frames.append(lines) + return frames + + for raw_frame in output.splitlines(): + for part in raw_frame.split('\r'): + line = part.strip() + if line: + frames.append([normalize_terminal_line(line)]) + return frames + + +def keep_recent_logs_with_progress( + frames: list[list[str]], + log_lines: int, +) -> list[list[str]]: + logs: list[str] = [] + output: list[list[str]] = [] + + for frame in frames: + log_frame = [line for line in frame if line.startswith('log:')] + progress_frame = [ + line for line in frame if not line.startswith('log:') + ] + if log_frame: + logs.extend(log_frame) + logs = logs[-log_lines:] + if progress_frame: + output.append(logs + progress_frame) + + return output + + +def limit_animation_frames(frames: list[list[str]]) -> list[list[str]]: + if len(frames) <= MAX_ANIMATION_FRAMES: + return frames + + last_index = len(frames) - 1 + selected = [ + round(index * last_index / (MAX_ANIMATION_FRAMES - 1)) + for index in range(MAX_ANIMATION_FRAMES) + ] + return [frames[index] for index in selected] + + +def tspan( + text: str, + class_name: str | None = None, + style: str | None = None, +) -> str: + if not text: + return '' + escaped = html.escape(text) + if style is not None: + return f'{escaped}' + if class_name is None: + return escaped + return f'{escaped}' + + +def xterm_256_to_rgb(color: int) -> tuple[int, int, int]: + if color < 16: + palette = ( + (0, 0, 0), + (128, 0, 0), + (0, 128, 0), + (128, 128, 0), + (0, 0, 128), + (128, 0, 128), + (0, 128, 128), + (192, 192, 192), + (128, 128, 128), + (255, 0, 0), + (0, 255, 0), + (255, 255, 0), + (0, 0, 255), + (255, 0, 255), + (0, 255, 255), + (255, 255, 255), + ) + return palette[max(0, color)] + + if color < 232: + color -= 16 + levels = (0, 95, 135, 175, 215, 255) + return ( + levels[color // 36], + levels[(color // 6) % 6], + levels[color % 6], + ) + + shade = 8 + (color - 232) * 10 + return shade, shade, shade + + +def ansi_rgb_style(red: int, green: int, blue: int) -> str: + return f'fill: #{red:02x}{green:02x}{blue:02x}' + + +def ansi_sgr_style(parameters: str, current_style: str | None) -> str | None: + codes = [int(code) if code else 0 for code in parameters.split(';')] + index = 0 + while index < len(codes): + code = codes[index] + if code in {0, 39}: + current_style = None + elif code == 38 and index + 1 < len(codes): + mode = codes[index + 1] + if mode == 2 and index + 4 < len(codes): + current_style = ansi_rgb_style( + codes[index + 2], + codes[index + 3], + codes[index + 4], + ) + index += 4 + elif mode == 5 and index + 2 < len(codes): + current_style = ansi_rgb_style( + *xterm_256_to_rgb(codes[index + 2]), + ) + index += 2 + else: + index += 1 + index += 1 + + return current_style + + +def styled_ansi_terminal_line(line: str) -> str: + output: list[str] = [] + cursor = 0 + current_style: str | None = None + for match in ANSI_SGR_RE.finditer(line): + output.append(tspan(line[cursor : match.start()], style=current_style)) + current_style = ansi_sgr_style(match.group(1), current_style) + cursor = match.end() + + output.append(tspan(line[cursor:], style=current_style)) + return ''.join(output) + + +def styled_text_segment( + text: str, + absolute_start: int, + full_line: str, +) -> str: + ranges: list[tuple[int, int, str]] = [] + if absolute_start == 0 and text.startswith('log:'): + ranges.append((0, 4, 'terminal-log')) + elif ( + absolute_start == 0 + and '%' in full_line + and (label_match := LABEL_RE.match(text)) + ): + ranges.append( + (label_match.start(), label_match.end(), 'terminal-label') + ) + + ranges.extend( + (match.start(), match.end(), 'terminal-percent') + for match in PERCENT_RE.finditer(text) + ) + ranges.extend( + (match.start(), match.end(), 'terminal-postfix') + for match in POSTFIX_RE.finditer(text) + ) + + output: list[str] = [] + cursor = 0 + for start, end, class_name in sorted(ranges): + if start < cursor: + continue + output.append(tspan(text[cursor:start])) + output.append(tspan(text[start:end], class_name)) + cursor = end + output.append(tspan(text[cursor:])) + return ''.join(output) + + +def styled_bar_segment(inner: str) -> str: + output = [tspan('|', 'terminal-bar-frame')] + for match in re.finditer(r'#+|\s+|[^#\s]+', inner): + value = match.group(0) + if set(value) == {'#'}: + class_name = 'terminal-bar-fill' + elif value.isspace(): + class_name = 'terminal-bar-empty' + else: + class_name = 'terminal-bar-text' + output.append(tspan(value, class_name)) + output.append(tspan('|', 'terminal-bar-frame')) + return ''.join(output) + + +def styled_terminal_line(line: str) -> str: + if '\x1b[' in line: + return styled_ansi_terminal_line(line) + + output: list[str] = [] + cursor = 0 + for match in BAR_RE.finditer(line): + output.append( + styled_text_segment(line[cursor : match.start()], cursor, line) + ) + output.append(styled_bar_segment(match.group('inner'))) + cursor = match.end() + output.append(styled_text_segment(line[cursor:], cursor, line)) + return ''.join(output) + + +def svg_document(title: str, frames: list[list[str]]) -> str: + width = SVG_WIDTH + line_height = 24 + max_lines = max(len(frame) for frame in frames) + height = 72 + max_lines * line_height + duration = f'{max(len(frames), 1) * ANIMATION_FRAME_SECONDS:g}' + frame_groups = [] + for index, frame in enumerate(frames): + visible_values = ['0'] * len(frames) + visible_values[index] = '1' + visible_value_list = ';'.join(visible_values) + base_opacity = '1' if index == 0 else '0' + lines = [] + for row, line in enumerate(frame): + lines.append( + f'' + f'{styled_terminal_line(line)}' + ) + frame_groups.append( + f'' + '' + ''.join(lines) + '' + ) + + return f''' + + + + + + {html.escape(title)} + {''.join(frame_groups)} + +''' + + +def render_svg(path: Path, title: str, frames: list[list[str]]) -> None: + svg = svg_document(title, frames) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(svg, encoding='utf-8') + + +def check_svg(path: Path, expected: str) -> None: + if not path.exists(): + raise SystemExit(f'missing generated asset: {path}') + if path.read_text(encoding='utf-8') != expected: + raise SystemExit(f'outdated generated asset: {path}') + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument('--check', action='store_true') + args = parser.parse_args() + for demo in DEMOS: + output = STATIC_DIR / f'progressbar-{demo.name}.svg' + frames = capture_demo(demo) + if args.check: + check_svg(output, svg_document(demo.title, frames)) + else: + render_svg(output, demo.title, frames) + + +if __name__ == '__main__': + main() diff --git a/tests/api_surface_snapshot.json b/tests/api_surface_snapshot.json index 05405e2..86820c5 100644 --- a/tests/api_surface_snapshot.json +++ b/tests/api_surface_snapshot.json @@ -9,12 +9,12 @@ "Counter": "class(format=?, **kwargs)", "CurrentTime": "class(format=?, microseconds=?, **kwargs)", "DataSize": "class(variable=?, format=?, unit=?, prefixes=?, **kwargs)", - "DataTransferBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "DataTransferBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "DoubleExponentialMovingAverage": "class(alpha=?)", "DynamicMessage": "class(name, format=?, width=?, precision=?, **kwargs)", "ETA": "class(format_not_started=?, format_finished=?, format=?, format_zero=?, format_na=?, **kwargs)", "ExponentialMovingAverage": "class(alpha=?)", - "FastProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "FastProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "FileTransferSpeed": "class(format=?, inverse_format=?, unit=?, prefixes=?, **kwargs)", "FormatCustomText": "class(format, mapping=?, **kwargs)", "FormatLabel": "class(format, **kwargs)", @@ -25,10 +25,11 @@ "MultiBar": "class(bars=?, fd=?, prepend_label=?, append_label=?, label_format=?, initial_format=?, finished_format=?, update_interval=?, show_initial=?, show_finished=?, remove_finished=?, sort_key=?, sort_reverse=?, sort_keyfunc=?, *, join_timeout=?, **progressbar_kwargs)", "MultiProgressBar": "class(name, markers=?, **kwargs)", "MultiRangeBar": "class(name, markers, **kwargs)", - "NullBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "NullBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "Percentage": "class(format=?, na=?, **kwargs)", "PercentageLabelBar": "class(format=?, na=?, **kwargs)", - "ProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "Postfix": "class(name=?, prefix=?, separator=?, **kwargs)", + "ProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "ReverseBar": "class(marker=?, left=?, right=?, fill=?, fill_left=?, **kwargs)", "RotatingMarker": "class(markers=?, default=?, fill=?, marker_wrap=?, fill_wrap=?, **kwargs)", "SimpleProgress": "class(format=?, **kwargs)", @@ -36,13 +37,14 @@ "SmoothingETA": "class(smoothing_algorithm=?, smoothing_parameters=?, **kwargs)", "SortKey": "enum(CREATED,LABEL,VALUE,PERCENTAGE)", "Timer": "class(format=?, **kwargs)", + "UnitProgress": "class(unit=?, unit_scale=?, **kwargs)", "UnknownLength": "class()", "Variable": "class(name, format=?, width=?, precision=?, **kwargs)", "VariableMixin": "class(name, **kwargs)", "__author__": "str", "__version__": "str", "len_color": "callable(value)", - "progressbar": "callable(iterator, min_value=?, max_value=?, widgets=?, prefix=?, suffix=?, fast=?, **kwargs)", + "progressbar": "callable(iterator, min_value=?, max_value=?, widgets=?, prefix=?, suffix=?, fast=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "streams": "StreamWrapper" }, "progressbar.algorithms": { @@ -53,12 +55,12 @@ "timedelta": "re-export" }, "progressbar.bar": { - "DataTransferBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "DataTransferBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "DefaultFdMixin": "class(fd=?, is_terminal=?, line_breaks=?, enable_colors=?, line_offset=?, **kwargs)", "FrameType": "re-export", - "NullBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "NullBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "NumberT": "re-export", - "ProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "ProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "ProgressBarBase": "class(**kwargs)", "ProgressBarMixinBase": "class(**kwargs)", "ResizableMixin": "class(term_width=?, **kwargs)", @@ -89,7 +91,8 @@ "is_terminal": "callable(fd, is_terminal=?)" }, "progressbar.fast": { - "FastProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, **kwargs)", + "Callable": "re-export", + "FastProgressBar": "class(min_value=?, max_value=?, widgets=?, left_justify=?, initial_value=?, poll_interval=?, widget_kwargs=?, custom_len=?, max_error=?, prefix=?, suffix=?, variables=?, min_poll_interval=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)", "annotations": "_Feature", "datetime": "re-export", "timedelta": "re-export" @@ -104,7 +107,7 @@ "progressbar.shortcuts": { "T": "type-alias", "annotations": "_Feature", - "progressbar": "callable(iterator, min_value=?, max_value=?, widgets=?, prefix=?, suffix=?, fast=?, **kwargs)" + "progressbar": "callable(iterator, min_value=?, max_value=?, widgets=?, prefix=?, suffix=?, fast=?, desc=?, total=?, unit=?, unit_scale=?, postfix=?, **kwargs)" }, "progressbar.terminal": { "CLEAR_LINE": "callable()", @@ -128,6 +131,7 @@ "DOWN": "callable(*args)", "DummyColor": "class()", "ESC": "str", + "Generator": "re-export", "HIDE_CURSOR": "callable()", "HSL": "class(hue, saturation, lightness)", "Iterable": "re-export", @@ -447,6 +451,7 @@ "yellow4": "callable(value)" }, "progressbar.terminal.stream": { + "Generator": "re-export", "Iterable": "re-export", "Iterator": "re-export", "LastLineStream": "class(stream)", @@ -457,10 +462,13 @@ }, "progressbar.utils": { "AttributeDict": "classsignature-unavailable", + "Callable": "re-export", "Iterable": "re-export", "Iterator": "re-export", + "Mapping": "re-export", "StreamWrapper": "class()", "StringT": "type-alias", + "T": "type-alias", "TracebackType": "re-export", "WrappingIO": "class(target, capturing=?, listeners=?)", "annotations": "_Feature", @@ -487,6 +495,7 @@ "ColoredMixin": "class()", "Counter": "class(format=?, **kwargs)", "CurrentTime": "class(format=?, microseconds=?, **kwargs)", + "DEFAULT_UNIT": "object", "Data": "type-alias", "DataSize": "class(variable=?, format=?, unit=?, prefixes=?, **kwargs)", "DynamicMessage": "class(name, format=?, width=?, precision=?, **kwargs)", @@ -507,6 +516,7 @@ "MultiRangeBar": "class(name, markers, **kwargs)", "Percentage": "class(format=?, na=?, **kwargs)", "PercentageLabelBar": "class(format=?, na=?, **kwargs)", + "Postfix": "class(name=?, prefix=?, separator=?, **kwargs)", "ReverseBar": "class(marker=?, left=?, right=?, fill=?, fill_left=?, **kwargs)", "RotatingMarker": "class(markers=?, default=?, fill=?, marker_wrap=?, fill_wrap=?, **kwargs)", "SamplesMixin": "class(samples=?, key_prefix=?, **kwargs)", @@ -517,6 +527,8 @@ "TGradientColors": "type-alias", "TimeSensitiveWidgetBase": "class(*args, fixed_colors=?, gradient_colors=?, **kwargs)", "Timer": "class(format=?, **kwargs)", + "UNIT_PREFIXES": "tuple", + "UnitProgress": "class(unit=?, unit_scale=?, **kwargs)", "Variable": "class(name, format=?, width=?, precision=?, **kwargs)", "VariableMixin": "class(name, **kwargs)", "WidgetBase": "class(*args, fixed_colors=?, gradient_colors=?, **kwargs)", @@ -524,6 +536,7 @@ "annotations": "_Feature", "create_marker": "callable(marker, wrap=?)", "create_wrapper": "callable(wrapper)", + "format_unit_value": "callable(value, unit=?, unit_scale=?)", "logger": "Logger", "string_or_lambda": "callable(input_)", "wrapper": "callable(function, wrapper_)" diff --git a/tests/test_multibar.py b/tests/test_multibar.py index 08ca1ea..9b6d66f 100644 --- a/tests/test_multibar.py +++ b/tests/test_multibar.py @@ -284,6 +284,100 @@ def test_multibar_finished() -> None: multibar.render(force=True) +def test_multibar_render_writes_started_bar_text() -> None: + fd = io.StringIO() + multibar = progressbar.MultiBar( + fd=fd, + initial_format=None, + finished_format=None, + remove_finished=None, + sort_reverse=False, + total=3, + term_width=64, + ) + build = multibar['build'] + test = multibar['test'] + multibar.render(force=True, flush=True) + fd.seek(0) + fd.truncate(0) + + build.update(1, force=True) + test.update(1, force=True) + multibar.render(force=True, flush=True) + + output = fd.getvalue() + assert 'build' in output + assert 'test' in output + assert '(1 of 3)' in output + + +def test_multibar_wraps_pre_labeled_bar_stream() -> None: + fd = io.StringIO() + multibar = progressbar.MultiBar( + fd=fd, + initial_format=None, + finished_format=None, + remove_finished=None, + sort_reverse=False, + total=3, + term_width=64, + ) + bar = progressbar.ProgressBar(max_value=3) + bar.label = 'build' + multibar['build'] = bar + + bar.update(1, force=True) + multibar.render(force=True, flush=True) + + output = fd.getvalue() + assert 'build' in output + assert '(1 of 3)' in output + + +def test_multibar_constructor_wraps_external_bar_stream() -> None: + fd = io.StringIO() + bar = progressbar.ProgressBar(max_value=3) + multibar = progressbar.MultiBar( + [('build', bar)], + fd=fd, + initial_format=None, + finished_format=None, + remove_finished=None, + sort_reverse=False, + total=3, + term_width=64, + ) + + bar.update(1, force=True) + multibar.render(force=True, flush=True) + + output = fd.getvalue() + assert 'build' in output + assert '(1 of 3)' in output + + +def test_multibar_constructor_accepts_mapping() -> None: + fd = io.StringIO() + bar = progressbar.ProgressBar(max_value=3) + multibar = progressbar.MultiBar( + {'build': bar}, + fd=fd, + initial_format=None, + finished_format=None, + remove_finished=None, + sort_reverse=False, + total=3, + term_width=64, + ) + + bar.update(1, force=True) + multibar.render(force=True, flush=True) + + output = fd.getvalue() + assert 'build' in output + assert '(1 of 3)' in output + + def test_multibar_finished_format() -> None: multibar = progressbar.MultiBar( finished_format='Finished {label}', show_finished=True diff --git a/tests/test_progressbar.py b/tests/test_progressbar.py index e8de2c0..7accb58 100644 --- a/tests/test_progressbar.py +++ b/tests/test_progressbar.py @@ -11,7 +11,10 @@ import pytest import progressbar -from progressbar import utils +from progressbar import ( + bar as bar_module, + utils, +) # Import hack to allow for parallel Tox try: @@ -85,6 +88,77 @@ def test_negative_maximum() -> None: progress.start() +def test_progressbar_accepts_total_alias() -> None: + bar = progressbar.ProgressBar(total=5, fd=io.StringIO()) + assert bar.max_value == 5 + + +def test_progressbar_max_value_wins_over_total() -> None: + bar = progressbar.ProgressBar(max_value=7, total=5, fd=io.StringIO()) + assert bar.max_value == 7 + + +def test_progressbar_desc_maps_to_prefix() -> None: + stream = io.StringIO() + with progressbar.ProgressBar( + desc='Loading', + max_value=1, + fd=stream, + ) as bar: + bar.update(1, force=True) + assert 'Loading' in stream.getvalue() + + +def test_progressbar_postfix_updates_live() -> None: + stream = io.StringIO() + widgets = [progressbar.Postfix()] + with progressbar.ProgressBar( + max_value=2, + widgets=widgets, + postfix={'loss': 1.0}, + fd=stream, + ) as bar: + bar.update(1, postfix={'loss': 0.5}, force=True) + assert 'loss=0.5' in stream.getvalue() + + +def test_progressbar_postfix_preserves_default_widgets() -> None: + stream = io.StringIO() + with progressbar.ProgressBar( + max_value=2, + postfix='ok', + fd=stream, + ) as bar: + bar.update(2, force=True) + rendered = stream.getvalue() + assert 'ok' in rendered + assert '100%' in rendered or '(2 of 2)' in rendered + + +def test_progressbar_empty_desc_maps_to_prefix() -> None: + stream = io.StringIO() + with progressbar.ProgressBar(desc='', max_value=1, fd=stream) as bar: + bar.update(1, force=True) + assert stream.getvalue().startswith(': ') + + +def test_shortcut_passes_total_desc_and_postfix() -> None: + stream = io.StringIO() + values = list( + progressbar.progressbar( + range(2), + total=2, + desc='Items', + postfix='ok', + fd=stream, + ) + ) + assert values == [0, 1] + rendered = stream.getvalue() + assert 'Items' in rendered + assert 'ok' in rendered + + def test_elapsed_data_spans_days() -> None: # Regression: A1 - days_elapsed was computed from timedelta.seconds, # which only contains the sub-day component. @@ -186,14 +260,67 @@ def write(self, value: str) -> int: unraisable: list[object] = [] monkeypatch.setattr(sys, 'unraisablehook', unraisable.append) + baseline_capturing = utils.streams.capturing bar = progressbar.ProgressBar(max_value=5, fd=io.StringIO(), term_width=60) bar.start() + bar_id = id(bar) + # The listener registry deliberately owns running bars. Remove this + # test's artificial reference so the finalizer path is exercised. + utils.streams.listeners.discard(bar) bar.fd = ExplodingIO() del bar gc.collect() assert not unraisable + assert all(id(listener) != bar_id for listener in utils.streams.listeners) + assert utils.streams.capturing == baseline_capturing + + +def test_finish_cleans_stream_listener_when_render_fails() -> None: + class ExplodingIO(io.StringIO): + def write(self, value: str) -> int: + raise ValueError('I/O operation on closed file') + + bar = progressbar.ProgressBar(max_value=5, fd=io.StringIO(), term_width=60) + bar.start() + assert bar in utils.streams.listeners + + bar.fd = ExplodingIO() + with pytest.raises(ValueError, match='I/O operation on closed file'): + bar.finish() + + assert bar not in utils.streams.listeners + + +def test_start_cleans_stream_listener_when_validation_fails() -> None: + bar = progressbar.ProgressBar( + min_value=-2, + max_value=-1, + fd=io.StringIO(), + ) + + with pytest.raises(ValueError, match='max_value out of range'): + bar.update(-1) + + assert bar not in utils.streams.listeners + + +def test_start_preserves_original_error_when_base_cleanup_fails( + monkeypatch, +) -> None: + def fail_start(self, max_value=None): + raise ValueError('resize start failed') + + def fail_finish(self): + raise RuntimeError('base cleanup failed') + + monkeypatch.setattr(bar_module.ResizableMixin, 'start', fail_start) + monkeypatch.setattr(bar_module.ProgressBarBase, 'finish', fail_finish) + + bar = progressbar.ProgressBar(max_value=5, fd=io.StringIO()) + with pytest.raises(ValueError, match='resize start failed'): + bar.start() @pytest.mark.skipif(os.name == 'nt', reason='SIGWINCH is POSIX-only') diff --git a/tests/test_progressbar_command.py b/tests/test_progressbar_command.py index a277f8c..771fca4 100644 --- a/tests/test_progressbar_command.py +++ b/tests/test_progressbar_command.py @@ -20,6 +20,39 @@ def test_size_to_bytes() -> None: assert main.size_to_bytes('1024p') == 1152921504606846976 +def test_sleep_for_rate_limit_skips_when_unset(monkeypatch) -> None: + calls = [] + monkeypatch.setattr(main.time, 'sleep', calls.append) + main._sleep_for_rate_limit(None, transferred=1024, started_at=0, now=1) + assert calls == [] + + +def test_sleep_for_rate_limit_sleeps_when_ahead(monkeypatch) -> None: + calls = [] + monkeypatch.setattr(main.time, 'sleep', calls.append) + main._sleep_for_rate_limit(1024, transferred=2048, started_at=0, now=1) + assert calls == [1] + + +def test_sleep_for_rate_limit_skips_when_on_schedule(monkeypatch) -> None: + calls = [] + monkeypatch.setattr(main.time, 'sleep', calls.append) + main._sleep_for_rate_limit(1024, transferred=1024, started_at=0, now=1) + assert calls == [] + + +def test_main_passes_rate_limit(tmp_path, monkeypatch) -> None: + sleeps = [] + monkeypatch.setattr(main.time, 'sleep', sleeps.append) + monkeypatch.setattr(main.time, 'monotonic', lambda: 0) + file = tmp_path / 'data.bin' + file.write_bytes(b'x' * 2048) + main.main( + ['--rate-limit', '1k', str(file), '-o', str(tmp_path / 'out.bin')], + ) + assert sleeps + + def test_filename_to_bytes(tmp_path) -> None: file = tmp_path / 'test' file.write_text('test') @@ -155,10 +188,59 @@ def __init__(self, **kwargs) -> None: self.init_kwargs = kwargs super().__init__(**kwargs) + class RecordingNullBar(progressbar.NullBar): + def __init__(self, **kwargs) -> None: + created.append(self) + self.init_kwargs = kwargs + super().__init__(**kwargs) + monkeypatch.setattr(main.progressbar, 'ProgressBar', RecordingProgressBar) + monkeypatch.setattr(main.progressbar, 'NullBar', RecordingNullBar) return created +def test_build_widgets_honors_display_flags() -> None: + parser = main.create_argument_parser() + args = parser.parse_args( + ['--progress', '--timer', '--eta', '--rate', '--bytes'] + ) + widgets = main._build_widgets(args, filesize_available=True) + widget_types = tuple( + type(widget) for widget in widgets if not isinstance(widget, str) + ) + assert progressbar.Percentage in widget_types + assert progressbar.Bar in widget_types + assert progressbar.Timer in widget_types + assert progressbar.AdaptiveETA in widget_types + assert progressbar.FileTransferSpeed in widget_types + assert progressbar.DataSize in widget_types + + +def test_build_widgets_quiet_is_empty() -> None: + parser = main.create_argument_parser() + args = parser.parse_args(['--quiet']) + assert main._build_widgets(args, filesize_available=True) == [] + + +def test_main_quiet_uses_null_bar(tmp_path, recorded_bars) -> None: + file = tmp_path / 'data.bin' + file.write_bytes(b'x' * 16) + main.main(['--quiet', str(file), '-o', str(tmp_path / 'out.bin')]) + + assert isinstance(recorded_bars[0], progressbar.NullBar) + + +def test_numeric_output_uses_line_breaks(tmp_path, recorded_bars) -> None: + file = tmp_path / 'data.bin' + file.write_bytes(b'x' * 16) + main.main(['--numeric', str(file), '-o', str(tmp_path / 'out.bin')]) + assert recorded_bars[0].init_kwargs['line_breaks'] is True + assert any( + isinstance(widget, progressbar.Percentage) + for widget in recorded_bars[0].init_kwargs['widgets'] + ) + + def test_main_passes_widgets(tmp_path, recorded_bars) -> None: # Regression: E2 - the configured widgets were built but never passed # to the progress bar. diff --git a/tests/test_readme_demos.py b/tests/test_readme_demos.py new file mode 100644 index 0000000..6893526 --- /dev/null +++ b/tests/test_readme_demos.py @@ -0,0 +1,352 @@ +import os +import re +import sys +import textwrap +from pathlib import Path + +import pytest + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +import scripts.render_readme_demos as demos # noqa: E402 + + +def _bar_widths(frames: list[list[str]]) -> list[int]: + return [ + len(match.group('inner')) + for frame in frames + for line in frame + if (match := demos.BAR_RE.search(line)) + ] + + +def _indented_snippet(snippet: str) -> str: + return '\n'.join( + f' {line}' if line else '' + for line in textwrap.dedent(snippet).strip().splitlines() + ) + + +def _readme_demo_block(demo: demos.Demo, alt: str) -> str: + return ( + f'.. image:: docs/_static/progressbar-{demo.name}.svg\n' + f' :alt: {alt}\n\n' + '.. code:: python\n\n' + f'{_indented_snippet(demo.snippet)}' + ) + + +def test_render_svg_escapes_terminal_text(tmp_path: Path) -> None: + output = tmp_path / 'demo.svg' + demos.render_svg( + output, + title='Demo', + frames=[ + ['', 'progress 0%'], + ['done & clean', 'progress 100%'], + ], + ) + text = output.read_text(encoding='utf-8') + assert '<start>' in text + assert 'done & clean' in text + assert ' None: + output = tmp_path / 'demo.svg' + demos.render_svg( + output, + title='Demo', + frames=[ + [ + 'Build: 50% (1 of 2) |## | ETA: 0:00:00 loss=0.5', + 'log: completed step 1', + ], + ], + ) + + text = output.read_text(encoding='utf-8') + assert 'class="terminal-label">Build:' in text + assert 'class="terminal-percent">50%' in text + assert 'class="terminal-bar-fill">##' in text + assert 'class="terminal-bar-empty"> ' in text + assert 'class="terminal-postfix">loss=0.5' in text + assert 'class="terminal-log">log:' in text + + +def test_render_svg_preserves_actual_ansi_truecolor_segments( + tmp_path: Path, +) -> None: + output = tmp_path / 'demo.svg' + demos.render_svg( + output, + title='Demo', + frames=[ + ['\x1b[38;2;255;0;0m 0%\x1b[39m \x1b[38;2;0;255;0m100%\x1b[39m'], + ], + ) + + text = output.read_text(encoding='utf-8') + assert 'style="fill: #ff0000"> 0%' in text + assert 'style="fill: #00ff00">100%' in text + assert 'class="terminal-percent"' not in text + assert '\x1b' not in text + + +def test_styled_terminal_line_keeps_spinner_pipe_outside_bar() -> None: + styled = demos.styled_terminal_line('| |# | 31 Elapsed Time: 0:00:00') + + assert styled.startswith( + '| |' + '#' + ) + spinner_bar = ( + 'terminal-bar-empty"> ' + ) + assert spinner_bar not in styled + + +def test_demo_definitions_are_ordered_and_exercise_key_features() -> None: + assert [(demo.name, demo.title) for demo in demos.DEMOS] == [ + ('hero', 'Progress with clean logs'), + ('multibar', 'Multiple active jobs'), + ('unknown-length', 'Unknown length'), + ] + + snippets = {demo.name: demo.snippet for demo in demos.DEMOS} + assert 'redirect_stdout=True' in snippets['hero'] + assert 'progressbar.MultiBar' in snippets['multibar'] + assert 'io.StringIO()' in snippets['multibar'] + assert 'fd.getvalue()' in snippets['multibar'] + assert '.fd.line' not in snippets['multibar'] + assert 'build: {build.value}/3' not in snippets['multibar'] + assert 'progressbar.UnknownLength' in snippets['unknown-length'] + assert 'for value in range(0, 120, 10):' in snippets['unknown-length'] + assert 'for value in (' not in snippets['unknown-length'] + + +def test_readme_uses_branch_relative_demo_assets() -> None: + readme = (demos.ROOT / 'README.rst').read_text(encoding='utf-8') + + assert ( + 'raw.githubusercontent.com/WoLpH/python-progressbar/develop' + not in readme + ) + assert '.. image:: docs/_static/progressbar-hero.svg' in readme + assert '.. image:: docs/_static/progressbar-multibar.svg' in readme + assert '.. image:: docs/_static/progressbar-unknown-length.svg' in readme + assert 'progressbar-ergonomics.svg' not in readme + assert 'Tqdm-style ergonomic options' not in readme + + +def test_readme_omits_obsolete_gpg_release_verification() -> None: + readme = (demos.ROOT / 'README.rst').read_text(encoding='utf-8') + + assert 'Release verification' not in readme + assert 'GPG' not in readme + assert 'pgp.mit.edu' not in readme + assert '.tar.gz.asc' not in readme + + +def test_readme_places_exact_demo_code_after_each_animation() -> None: + readme = (demos.ROOT / 'README.rst').read_text(encoding='utf-8') + alt_by_name = { + 'hero': 'progressbar2 showing clean progress output with logs', + 'multibar': 'multiple progress bars updating together', + 'unknown-length': 'unknown length progress with an animated marker', + } + + for demo in demos.DEMOS: + assert _readme_demo_block(demo, alt_by_name[demo.name]) in readme + + +def test_render_svg_first_frame_is_visible_without_animation( + tmp_path: Path, +) -> None: + output = tmp_path / 'demo.svg' + demos.render_svg( + output, + title='Demo', + frames=[ + ['first'], + ['second'], + ], + ) + + text = output.read_text(encoding='utf-8') + assert text.count('') == 1 + assert text.count('') == 1 + assert ' None: + output = tmp_path / 'demo.svg' + demos.render_svg(output, title='Demo', frames=[['a'], ['b'], ['c']]) + + text = output.read_text(encoding='utf-8') + assert 'dur="0.24s"' in text + assert 'dur="3.6s"' not in text + assert '3.599999999' not in text + + +def test_render_svg_uses_wide_canvas_for_readable_bars( + tmp_path: Path, +) -> None: + output = tmp_path / 'demo.svg' + demos.render_svg(output, title='Demo', frames=[['a']]) + + text = output.read_text(encoding='utf-8') + assert 'width="1080"' in text + assert 'viewBox="0 0 1080 96"' in text + + +def test_multibar_demo_captures_rendered_progressbar_output() -> None: + demo = next(demo for demo in demos.DEMOS if demo.name == 'multibar') + frames = demos.capture_demo(demo) + text = '\n'.join(line for frame in frames for line in frame) + + assert frames + assert all(len(frame) == 2 for frame in frames) + assert 'build: 1/3 | test: 1/3' not in text + assert 'build' in text + assert 'test' in text + assert 'Elapsed Time:' in text + assert 'ETA:' in text + assert '(0 of 24)' in text + assert '(1 of 24)' in text or '(2 of 24)' in text + assert '(24 of 24)' in text + + +def test_multibar_demo_shows_independent_progress_values() -> None: + demo = next(demo for demo in demos.DEMOS if demo.name == 'multibar') + frames = demos.capture_demo(demo) + + mismatched_values = [] + for frame in frames: + values = [ + int(match.group(1)) + for line in frame + if (match := re.search(r'\((\d+) of 24\)', line)) + ] + if len(values) == 2 and values[0] != values[1]: + mismatched_values.append(values) + + assert mismatched_values + + +def test_capture_demo_timeout_raises_clear_error( + monkeypatch: pytest.MonkeyPatch, +) -> None: + def raise_timeout(*args: object, **kwargs: object) -> object: + raise demos.subprocess.TimeoutExpired(cmd=['python'], timeout=5) + + demo = demos.Demo('sample', 'Sample', '') + monkeypatch.setattr(demos.subprocess, 'run', raise_timeout) + + with pytest.raises(SystemExit) as error: + demos.capture_demo(demo) + + assert str(error.value) == 'timed out capturing demo: sample' + + +def test_capture_frames_splits_carriage_return_output() -> None: + frames = demos.frames_from_output('zero\rone\ntwo\rthree') + assert frames == [['zero'], ['one'], ['two'], ['three']] + + +def test_capture_frames_keeps_more_animation_states() -> None: + frames = demos.frames_from_output( + '\n'.join(f'frame {index}' for index in range(30)) + ) + + assert len(frames) == 24 + assert frames[0] == ['frame 0'] + assert frames[-1] == ['frame 29'] + + +def test_readme_demos_emit_enough_frames_to_look_responsive() -> None: + frame_counts = { + demo.name: len(demos.capture_demo(demo)) for demo in demos.DEMOS + } + + assert frame_counts['hero'] == 24 + assert frame_counts['multibar'] == 24 + assert frame_counts['unknown-length'] >= 8 + + +def test_readme_demos_show_wide_determinate_progress_bars() -> None: + frames_by_name = { + demo.name: demos.capture_demo(demo) for demo in demos.DEMOS + } + + for name in ('hero', 'multibar'): + text = '\n'.join( + line for frame in frames_by_name[name] for line in frame + ) + first_frame = '\n'.join(frames_by_name[name][0]) + last_frame = '\n'.join(frames_by_name[name][-1]) + assert '0%' in first_frame + assert '100%' in last_frame + assert '0%' in text + assert '100%' in text + assert max(_bar_widths(frames_by_name[name])) >= 32 + + +@pytest.mark.skipif( + os.name == 'nt', + reason='capture_demo renders in a subprocess; on Windows the color ' + 'detection reports the WINDOWS level whose color path is a no-op, so ' + 'truecolor escapes are never emitted (demos are generated on posix)', +) +def test_readme_demos_capture_real_percentage_color_changes() -> None: + demo = next(demo for demo in demos.DEMOS if demo.name == 'hero') + text = '\n'.join( + line for frame in demos.capture_demo(demo) for line in frame + ) + + assert '\x1b[38;2;255;0;0m' in text + assert '\x1b[38;2;0;255;0m' in text + + +def test_hero_demo_keeps_recent_logs_visible_above_progress() -> None: + demo = next(demo for demo in demos.DEMOS if demo.name == 'hero') + frames = demos.capture_demo(demo) + + assert max(len(frame) for frame in frames) >= 3 + assert any( + any(line.startswith('log: completed step 8') for line in frame) + and any('Build:' in line for line in frame) + for frame in frames + ) + + +def test_capture_frames_normalizes_variable_timing_text() -> None: + frames = demos.frames_from_output( + 'Build: 50% Elapsed Time: 0:01:23 ETA: 9:08:07\n' + 'Build: 100% Elapsed Time: 0:00:04 Time: 2:03:04' + ) + assert frames == [ + ['Build: 50% Elapsed Time: 0:00:00 ETA: 0:00:00'], + ['Build: 100% Elapsed Time: 0:00:00 Time: 0:00:00'], + ] + + +def test_check_mode_does_not_rewrite_mismatched_asset( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + demo = demos.Demo('sample', 'Sample', '') + output = tmp_path / 'progressbar-sample.svg' + output.write_text('stale asset', encoding='utf-8') + + monkeypatch.setattr(demos, 'DEMOS', [demo]) + monkeypatch.setattr(demos, 'STATIC_DIR', tmp_path) + monkeypatch.setattr(demos, 'capture_demo', lambda demo: [['fresh asset']]) + monkeypatch.setattr(sys, 'argv', ['render_readme_demos.py', '--check']) + + with pytest.raises(SystemExit) as error: + demos.main() + + assert 'outdated generated asset' in str(error.value) + assert output.read_text(encoding='utf-8') == 'stale asset' diff --git a/tests/test_stream.py b/tests/test_stream.py index 194310c..30bdaa9 100644 --- a/tests/test_stream.py +++ b/tests/test_stream.py @@ -1,4 +1,5 @@ import io +import logging import os import sys @@ -8,10 +9,28 @@ from progressbar import terminal +def reset_wrapped_streams() -> None: + while progressbar.streams.wrapped_logging: + progressbar.streams.unwrap_logging() + while ( + progressbar.streams.wrapped_stdout + or progressbar.streams.wrapped_stderr + ): + progressbar.streams.unwrap(stderr=True, stdout=True) + progressbar.streams.wrapped_logging = 0 + progressbar.streams.wrapped_stdout = 0 + progressbar.streams.wrapped_stderr = 0 + progressbar.streams.logging_handlers.clear() + for listener in list(progressbar.streams.listeners): + listener._finished = True + progressbar.streams.listeners.clear() + progressbar.streams.capturing = 0 + progressbar.streams.update_capturing() + + def test_nowrap() -> None: # Make sure we definitely unwrap - for _i in range(5): - progressbar.streams.unwrap(stderr=True, stdout=True) + reset_wrapped_streams() stdout = sys.stdout stderr = sys.stderr @@ -27,14 +46,12 @@ def test_nowrap() -> None: assert stderr == sys.stderr # Make sure we definitely unwrap - for _i in range(5): - progressbar.streams.unwrap(stderr=True, stdout=True) + reset_wrapped_streams() def test_wrap() -> None: # Make sure we definitely unwrap - for _i in range(5): - progressbar.streams.unwrap(stderr=True, stdout=True) + reset_wrapped_streams() stdout = sys.stdout stderr = sys.stderr @@ -54,8 +71,295 @@ def test_wrap() -> None: assert stderr == sys.stderr # Make sure we definitely unwrap - for _i in range(5): - progressbar.streams.unwrap(stderr=True, stdout=True) + reset_wrapped_streams() + + +def test_wrap_logging_retargets_existing_stderr_handler(monkeypatch) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + monkeypatch.setattr(sys, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-wrap-logging') + logger.handlers = [] + logger.propagate = False + handler = logging.StreamHandler(sys.stderr) + logger.addHandler(handler) + + progressbar.streams.wrap_stderr() + progressbar.streams.wrap_logging() + try: + assert handler.stream is progressbar.streams.stderr + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_unwrap_logging_restores_handler_stream(monkeypatch) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stderr', stream) + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-unwrap-logging') + logger.handlers = [] + logger.propagate = False + handler = logging.StreamHandler(sys.stderr) + logger.addHandler(handler) + + progressbar.streams.wrap_stderr() + progressbar.streams.wrap_logging() + progressbar.streams.unwrap_logging() + + try: + assert handler.stream is stream + finally: + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_unwrap_logging_restores_handler_created_after_stderr_wrap( + monkeypatch, +) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stderr', stream) + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-wrapped-stderr-handler') + logger.handlers = [] + logger.propagate = False + + progressbar.streams.wrap_stderr() + wrapped_stderr = progressbar.streams.stderr + handler = logging.StreamHandler(sys.stderr) + logger.addHandler(handler) + + try: + assert handler.stream is wrapped_stderr + + progressbar.streams.wrap_logging() + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + + assert handler.stream is stream + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_wrap_logging_handles_nested_calls(monkeypatch) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stderr', stream) + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-nested-wrap-logging') + logger.handlers = [] + logger.propagate = False + handler = logging.StreamHandler(sys.stderr) + logger.addHandler(handler) + + progressbar.streams.wrap_stderr() + progressbar.streams.wrap_logging() + progressbar.streams.wrap_logging() + try: + assert progressbar.streams.wrapped_logging == 2 + progressbar.streams.unwrap_logging() + assert progressbar.streams.wrapped_logging == 1 + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_wrap_logging_retargets_existing_stdout_handler(monkeypatch) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stdout', stream) + monkeypatch.setattr(progressbar.streams, 'original_stdout', stream) + monkeypatch.setattr(progressbar.streams, 'stdout', stream) + + logger = logging.getLogger('progressbar-test-wrap-stdout-logging') + logger.handlers = [] + logger.propagate = False + handler = logging.StreamHandler(sys.stdout) + logger.addHandler(handler) + + progressbar.streams.wrap_stdout() + progressbar.streams.wrap_logging() + try: + assert handler.stream is progressbar.streams.stdout + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stdout=True) + logger.handlers = [] + + +def test_wrap_logging_ignores_handlers_that_reject_streams( + monkeypatch, +) -> None: + class RejectingHandler(logging.StreamHandler): + def setStream(self, stream): # noqa: N802 + raise ValueError('stream rejected') + + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stderr', stream) + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-rejecting-handler') + logger.handlers = [] + logger.propagate = False + handler = RejectingHandler(sys.stderr) + logger.addHandler(handler) + + progressbar.streams.wrap_stderr() + try: + progressbar.streams.wrap_logging() + assert all( + logged_handler is not handler + for logged_handler, _stream in progressbar.streams.logging_handlers + ) + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_wrap_logging_uses_handler_snapshot(monkeypatch) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stderr', stream) + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-mutating-handlers') + logger.handlers = [] + logger.propagate = False + first_handler = logging.StreamHandler(sys.stderr) + late_handler = logging.StreamHandler(sys.stderr) + logger.addHandler(first_handler) + + wrapped_handlers: list[logging.StreamHandler] = [] + original_wrap_handler = progressbar.streams._wrap_logging_handler + + def mutating_wrap_handler(handler, wrapped_streams, restore_streams): + wrapped_handlers.append(handler) + if handler is first_handler: + logger.addHandler(late_handler) + original_wrap_handler(handler, wrapped_streams, restore_streams) + + monkeypatch.setattr( + progressbar.streams, + '_wrap_logging_handler', + mutating_wrap_handler, + ) + + progressbar.streams.wrap_stderr() + try: + progressbar.streams.wrap_logging() + assert first_handler in wrapped_handlers + assert late_handler not in wrapped_handlers + assert late_handler.stream is stream + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_unwrap_logging_ignores_dynamic_stderr_handler(monkeypatch) -> None: + reset_wrapped_streams() + + stream = io.StringIO() + monkeypatch.setattr(sys, 'stderr', stream) + monkeypatch.setattr(progressbar.streams, 'original_stderr', stream) + monkeypatch.setattr(progressbar.streams, 'stderr', stream) + + logger = logging.getLogger('progressbar-test-dynamic-stderr-handler') + logger.handlers = [] + logger.propagate = False + handler = logging._StderrHandler() # type: ignore[attr-defined] + logger.addHandler(handler) + + try: + progressbar.streams.wrap_stderr() + assert handler.stream is progressbar.streams.stderr + + progressbar.streams.wrap_logging() + progressbar.streams.unwrap_logging() + finally: + progressbar.streams.unwrap_logging() + progressbar.streams.unwrap(stderr=True) + logger.handlers = [] + + +def test_redirected_stdout_lines_are_flushed_above_bar(monkeypatch) -> None: + reset_wrapped_streams() + output = io.StringIO() + monkeypatch.setattr(progressbar.streams, 'original_stdout', output) + monkeypatch.setattr(progressbar.streams, 'stdout', output) + monkeypatch.setattr(sys, 'stdout', output) + + with progressbar.ProgressBar( + max_value=2, + fd=output, + redirect_stdout=True, + line_breaks=False, + is_terminal=True, + term_width=40, + ) as bar: + print('phase one') + bar.update(1, force=True) + print('phase two') + bar.update(2, force=True) + + rendered = output.getvalue() + assert 'phase one' in rendered + assert 'phase two' in rendered + assert '\r' + ' ' * 40 + '\rphase two\n' in rendered + assert rendered.endswith('\n') + + +def test_redirected_stderr_lines_are_flushed_above_bar(monkeypatch) -> None: + reset_wrapped_streams() + output = io.StringIO() + monkeypatch.setattr(progressbar.streams, 'original_stderr', output) + monkeypatch.setattr(progressbar.streams, 'stderr', output) + monkeypatch.setattr(sys, 'stderr', output) + + with progressbar.ProgressBar( + max_value=2, + fd=output, + redirect_stderr=True, + line_breaks=False, + is_terminal=True, + term_width=40, + ) as bar: + print('warning one', file=sys.stderr) + bar.update(1, force=True) + print('warning two', file=sys.stderr) + bar.update(2, force=True) + + rendered = output.getvalue() + assert 'warning one' in rendered + assert 'warning two' in rendered + assert '\r' + ' ' * 40 + '\rwarning two\n' in rendered + assert rendered.endswith('\n') def test_excepthook() -> None: diff --git a/tests/test_widgets.py b/tests/test_widgets.py index f31bc7c..03eb3a9 100644 --- a/tests/test_widgets.py +++ b/tests/test_widgets.py @@ -115,6 +115,112 @@ def test_widgets_large_values(max_value) -> None: p.finish() +def test_postfix_widget_renders_mapping_sorted() -> None: + bar = progressbar.ProgressBar( + widgets=[progressbar.Postfix()], + variables={'postfix': {'z': 2, 'a': 1}}, + fd=io.StringIO(), + term_width=80, + ) + bar.start() + output = ''.join(bar._format_widgets()) + assert output == ' a=1, z=2' + + +def test_postfix_widget_renders_string() -> None: + bar = progressbar.ProgressBar( + widgets=[progressbar.Postfix()], + variables={'postfix': 'loss=0.25'}, + fd=io.StringIO(), + term_width=80, + ) + bar.start() + output = ''.join(bar._format_widgets()) + assert output == ' loss=0.25' + + +def test_postfix_widget_renders_other_values() -> None: + bar = progressbar.ProgressBar( + widgets=[progressbar.Postfix()], + variables={'postfix': 3}, + fd=io.StringIO(), + term_width=80, + ) + bar.start() + output = ''.join(bar._format_widgets()) + assert output == ' 3' + + +@pytest.mark.parametrize( + ('value', 'expected'), + [ + (0, ' 0'), + (0.0, ' 0.0'), + (False, ' False'), + ], +) +def test_postfix_widget_renders_falsy_values(value, expected) -> None: + bar = progressbar.ProgressBar( + widgets=[progressbar.Postfix()], + variables={'postfix': value}, + fd=io.StringIO(), + term_width=80, + ) + bar.start() + output = ''.join(bar._format_widgets()) + assert output == expected + + +def test_postfix_widget_omits_empty_values() -> None: + bar = progressbar.ProgressBar( + widgets=[progressbar.Postfix()], + variables={'postfix': None}, + fd=io.StringIO(), + term_width=80, + ) + bar.start() + output = ''.join(bar._format_widgets()) + assert output == '' + + +def test_format_unit_value_special_cases() -> None: + assert progressbar.widgets.format_unit_value(None) == 'N/A' + unknown_value = progressbar.widgets.format_unit_value( + progressbar.UnknownLength, + ) + assert unknown_value == 'N/A' + assert progressbar.widgets.format_unit_value(1.5, unit='B') == '1.5 B' + assert progressbar.widgets.format_unit_value(2, unit='B') == '2 B' + + +def test_unit_progress_scales_values() -> None: + bar = progressbar.ProgressBar( + max_value=2048, + widgets=[progressbar.UnitProgress(unit='B', unit_scale=True)], + fd=io.StringIO(), + term_width=80, + ) + bar.start() + bar.update(1024, force=True) + output = ''.join(bar._format_widgets()) + assert output == '1.0 KiB of 2.0 KiB' + + +def test_unit_progress_uses_progress_units_by_default() -> None: + bar = progressbar.ProgressBar( + max_value=2048, + widgets=[progressbar.UnitProgress()], + unit='B', + unit_scale=True, + fd=io.StringIO(), + term_width=80, + ) + bar.start() + bar.update(1024, force=True) + output = ''.join(bar._format_widgets()) + assert output == '1.0 KiB of 2.0 KiB' + + def test_format_widget() -> None: widgets = [ progressbar.FormatLabel(f'%({mapping})r')