Merge branch 'master' into tool-init

Author: bdbaddog

CHANGES.txt

@@ -44,19 +44,61 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
       new contributors. (Using Gemini AI)
     - Fix Appveyor scripting to install unavailable python versions when needed and use them
       for testing.
+    - Subst: Fixed ListSubber.expanded(), which never detected an already-expanded
+      string (dead code since its 2019 introduction), so fully-expanded values are
+      no longer recursively re-processed during scons_subst_list().
+    - Subst: scons_subst() and scons_subst_list() no longer leak a __builtins__
+      key into the construction environment's dictionary if an exception is
+      raised during substitution.
+    - Subst: the result of the inspect.signature() check for callable
+      construction variables is now cached per callable, speeding up expansion
+      of function-valued variables. Callables whose signature cannot be
+      determined (some C/builtin callables) are now treated as not matching
+      the (target, source, env, for_signature) convention instead of raising.
+    - Subst: variable values which are plain strings with no further '$'
+      expansions are now returned directly, skipping an unneeded dict copy
+      and recursive substitution pass. Combined, the substitution speedups
+      measured on a representative command line (this from a micro benchmark
+      and not on a full build) are:
+      ('$CC $CCFLAGS $CPPDEFINES $GEN -c -o $TARGET $SOURCES'):
+                            old         new       improvement
+        scons_subst         20.7 us     12.8 us   ~38% faster
+        scons_subst_list    37.4 us     25.1 us   ~33% faster
+    - Subst: a NameError raised during scons_subst_list() now includes the
+      name of the unknown variable in the error message.
+    - Subst: the overrides argument to scons_subst()/scons_subst_list() no
+      longer mutates a caller-supplied lvars dictionary; also removed mutable
+      default arguments. Removed Literal.__neq__, a misspelled (and therefore
+      never-invoked) version of __ne__; Python derives inequality from
+      Literal.__eq__.
+
+  From Prabhu S. Khalsa:
+    - Fix typo in preface
+    - Remove word "below" when content is not actually located below
+    - Fix typos in preface, Chapter 6, Chapter 9 and Chapter 10 of User Guide
+    - Fix broken links in Chapter 1 of User Guide
+
 
   From Mats Wichmann:
     - Introduce some unit tests for the file locking utility routines
-    - More clarifications in manpage Builder Methods section.
+    - Subst: Improved variable substitution by consolidating dictionary
+      merging operations, reducing unnecessary copy operations when no
+      TARGET/SOURCE variables or overrides need to be applied. Combined
+      with other substitution improvements (lru_cache for callable signature
+      caching, action hashability, for_signature bug fix, f-string
+      modernization), typical builds see up to 8-12% improvement with larger gains
+      (up to 20-30%) on builds with many callable construction variables.
     - Reduce unneeded computation of overrides. The Mkdir builder used an
       unknown argument ('explain') on creation, causing it to be considered
       an override. Also, if override dict is empty, don't even call the
       Override factory function.
     - Handle the default (unset) ProgressObject differently for the sole
       purpose of avoiding Sphinx 9.0+ blowing up on it (it's been giving
       a warning for years, but now it's a fatal error).
-    - Improve covarage of API doc build by ignoring any setting of
+    - Improve coverage of "API docs" build by ignoring any setting of
       __all__ in a package and not showing inherited members from optparse.
+      Add a trick to make sure the build sees PathListCache while
+      generating docs (in normal operation, it's removed so was not visible).
     - Fix --debug=includes for case of multiple source files.
     - Unify internal "_null" sentinel usage.
     - All functions/classes/non-dunder methods in Environment have docstrings.
@@ -84,15 +126,25 @@ RELEASE  VERSION/DATE TO BE FILLED IN LATER
     - Don't duplicate the creation of Tool objects if "default" is part of
       the tool list.
     - Test suite: add support for Python 3.15 in the Action unit tests.
-    - Update documentation of the three file-finding functions in the API
-      (FindFile, FindInstalledFiles, FindSourceFiles) as well as FindPathDirs.
-      Also add a test for FindFile to be sure it locates non-existing
-      derived files as advertised.
+    - Add a test for FindFile to be sure it locates non-existing derived
+      files as advertised.
     - zip tool now uses zipfile module's "from_file" method to populate
       ZipInfo records, rather than doing manually.
-    - Update documentation for linking-related construction variables.
-      Add a few more live links.
-    - Update documentation for CacheDir and related option flags
+    - Reference manual improvements:
+      * More clarifications in Builder Methods section.
+      * Clarify VariantDir behavior when switching from duplicate=True (the
+        default) to duplicate=False, and tweak wording a bit.
+      * Update documentation of the three file-finding functions (FindFile,
+        FindInstalledFiles, FindSourceFiles) as well as FindPathDirs.
+      * Update linking-related construction variables.
+      * Add a few more live links.
+      * Clarify CacheDir and related option flags.
+      * Drop old example section from manpage (not a visible change: has been
+        commented out since v4.0, when the content moved to the new Cookbook)
+      * Reword IMPLICIT_COMMAND_DEPENDENCIES construction variable.
+      * Add info on the type of an option variable to GetOption and SetOption.
+      * Tighten up the doc for Variables-Add() and add a note on file contents
+        not being preserved to Variables-Save()
 
 
 RELEASE 4.10.1 - Sun, 16 Nov 2025 10:51:57 -0700

RELEASE.txt

@@ -63,6 +63,18 @@ IMPROVEMENTS
   an override. Also, if override dict is empty, don't even call the
   Override factory function.
 
+- Subst: Multiple substitution optimizations providing up to 8-12% performance
+  improvement on typical builds, with larger gains (up to 20-30%) on builds with
+  many callable construction variables:
+  * Consolidate dictionary merging operations to reduce unnecessary copies
+  * Use functools.lru_cache for callable signature inspection with bounded
+    memory (256 entries max, ~1MB)
+  * Make Action classes (CommandAction, FunctionAction, ListAction) hashable
+    to enable cache optimization
+  * Fix for_signature logic bug in ListSubber.expand() for correct signature
+    generation
+  * Modernize string formatting to f-strings
+
 - Test runner reworked to use Python logging; the portion of the test suite
   which tests the runner was adjusted to match.
 
@@ -112,8 +124,10 @@ DOCUMENTATION
   a warning for years, but now it's a fatal error). Affects only the
   API doc build.
 
-- Improve coverage of API doc build by ignoring any setting of
+- Improve coverage of "API doc" build by ignoring any setting of
   __all__ in a package and not showing inherited members from optparse.
+  Add a trick to make sure build sees PathListCache while generating
+  docs (in normal operation, it's removed so was not visible).
 
 - All functions/classes/non-dunder methods in Environment now have docstrings.
 
@@ -132,6 +146,23 @@ DOCUMENTATION
 
 - Update documentation for CacheDir and related option flags
 
+- Drop old example section from manpage. This has been commented out since
+  release 4.0, when the content was used to populate the new SCons Cookbook.
+
+- Update documentation for IMPLICIT_COMMAND_DEPENDENCIES construction variable.
+
+- Fix typo in preface
+- Remove word "below" when content is not actually located below
+
+- Fix typos in preface, Chapter 6, Chapter 9 and Chapter 10 of User Guide
+
+- Fix broken links in Chapter 1 of User Guide
+
+- Add info on the type of an option variable to GetOption and SetOption.
+
+- Tighten up the doc for Variables-Add() and add a note on file contents
+  not being preserved to Variables-Save()
+
 DEVELOPMENT
 -----------
 

SCons/Action.py

@@ -1000,6 +1000,9 @@ def __str__(self) -> str:
             return ' '.join(map(str, self.cmd_list))
         return str(self.cmd_list)
 
+    def __hash__(self) -> int:
+        """ Added so CommandAction can be used as a key in a dict and/or cached. """
+        return id(self)
 
     def process(self, target, source, env, executor: Executor | None = None, overrides: dict | None = None) -> tuple[list, bool, bool]:
         if executor:
@@ -1421,6 +1424,10 @@ def __str__(self) -> str:
             return str(self.execfunction)
         return "%s(target, source, env)" % name
 
+    def __hash__(self) -> int:
+        """ Added so FunctionAction can be used as a key in a dict and/or cached. """
+        return id(self)
+
     def execute(self, target, source, env, executor: Executor | None = None):
         exc_info = (None,None,None)
         try:
@@ -1482,6 +1489,10 @@ def genstring(self, target, source, env, executor: Executor | None = None) -> st
     def __str__(self) -> str:
         return '\n'.join(map(str, self.list))
 
+    def __hash__(self) -> int:
+        """ Added so ListAction can be used as a key in a dict and/or cached. """
+        return id(self)
+
     def presub_lines(self, env):
         return SCons.Util.flatten_sequence(
             [a.presub_lines(env) for a in self.list])

SCons/Action.xml

@@ -27,109 +27,66 @@ This file is processed by the bin/SConsDoc.py module.
 <cvar name="IMPLICIT_COMMAND_DEPENDENCIES">
 <summary>
 <para>
-Controls whether or not &SCons; will
-add implicit dependencies for the commands
-executed to build targets.
+Controls how &SCons; adds implicit dependencies from action strings
+(the commands executed to build targets).
+These implicit dependencies allow &SCons; to trigger target rebuilds
+when the compilers, tools, or scripts that affect build outcomes change
+(for example, when a new version of a compiler is installed).
+By default, &SCons; adds an implicit dependency on the executable
+represented by the first argument of the command line.
+This executable is first checked as a path;
+if not found, it is searched for in the execution environment's search path
+(<literal>env['ENV']['PATH']</literal>).
 </para>
 
 <para>
-By default, &SCons; will add to each target
-an implicit dependency on the command
-represented by the first argument of any
-command line it executes (which is typically
-the command itself). By setting such
-a dependency, &SCons; can determine that
-a target should be rebuilt if the command changes,
-such as when a compiler is upgraded to a new version.
-The specific file for the dependency is
-found by searching the
-<varname>PATH</varname>
-variable in the
-<varname>ENV</varname> dictionary
-in the &consenv; used to execute the command.
-The default is the same as
-setting the &consvar;
+If &cv-IMPLICIT_COMMAND_DEPENDENCIES; is set to a falsy value
+(<literal>"none"</literal>, <literal>"false"</literal>,
+<literal>"no"</literal>, <literal>"off"</literal>,
+<literal>"0"</literal> or integer <literal>0</literal>),
+no implicit dependencies from action strings are added.
+If set to an integer value greater than <literal>1</literal>,
+or to <literal>"all"</literal>,
+additional command-line arguments are checked.
+A numeric value <emphasis>N</emphasis>
+checks the first <emphasis>N</emphasis> arguments,
+while <literal>"all"</literal> checks every argument.
+A checked argument beyond the first is added as a dependency
+if it is an absolute path or it refers to
+an existing file in the filesystem;
+unlike the first argument, these additional arguments are not searched
+using the execution environment's search path.
+All other values of &cv-IMPLICIT_COMMAND_DEPENDENCIES;
+are treated the same as the default.
+The value of
 &cv-IMPLICIT_COMMAND_DEPENDENCIES;
-to a True-like value (<quote>true</quote>,
-<quote>yes</quote>,
-or <quote>1</quote> - but not a number
-greater than one, as that has a different meaning).
+is subject to substitution before it is used,
+so it can contain a &consvar; reference.
 </para>
 
 <para>
-Action strings can be segmented by the
-use of an AND operator, <literal>&amp;&amp;</literal>.
-In a segmented string, each segment is a separate
-<quote>command line</quote>, these are run
-sequentially until one fails, or the entire
-sequence has been executed. If an
-action string is segmented, then the selected
-behavior of &cv-IMPLICIT_COMMAND_DEPENDENCIES;
-is applied to each segment.
+Action strings may be segmented with a logical AND operator
+(<literal>&amp;&amp;</literal>),
+indicating each segment is executed sequentially,
+but only if the previous segment succeeded.
+If &cv-IMPLICIT_COMMAND_DEPENDENCIES; is set to
+<literal>"all"</literal> or an integer greater than <literal>1</literal>,
+each segment is treated as a separate command line
+for computing dependencies;
+otherwise, only the first segment is checked.
 </para>
-
-<para>
-If &cv-IMPLICIT_COMMAND_DEPENDENCIES;
-is set to a False-like value
-(<quote>none</quote>,
-<quote>false</quote>,
-<quote>no</quote>,
-<quote>0</quote>,
-etc.),
-then the implicit dependency will
-not be added to the targets
-built with that &consenv;.
-</para>
-
-<para>
-If &cv-IMPLICIT_COMMAND_DEPENDENCIES;
-is set to <quote>2</quote> or higher,
-then that number of arguments in the command line
-will be scanned for relative or absolute paths.
-If any are present, they will be added as
-implicit dependencies to the targets built
-with that &consenv;.
-The first argument in the command line will be
-searched for using the <varname>PATH</varname>
-variable in the <varname>ENV</varname> dictionary
-in the &consenv; used to execute the command.
-The other arguments will only be found if they
-are absolute paths or valid paths relative
-to the working directory.
-</para>
-
-<para>
-If &cv-IMPLICIT_COMMAND_DEPENDENCIES;
-is set to <quote>all</quote>,
-then all arguments in the command line will be
-scanned for relative or absolute paths.
-If any are present, they will be added as
-implicit dependencies to the targets built
-with that &consenv;.
-The first argument in the command line will be
-searched for using the <varname>PATH</varname>
-variable in the <varname>ENV</varname> dictionary
-in the &consenv; used to execute the command.
-The other arguments will only be found if they
-are absolute paths or valid paths relative
-to the working directory.
-</para>
-
-<example_commands>
-env = Environment(IMPLICIT_COMMAND_DEPENDENCIES=False)
-</example_commands>
 </summary>
 </cvar>
 
 <cvar name="PRINT_CMD_LINE_FUNC">
 <summary>
 <para>
-A Python function used to print the command lines as they are executed
-(assuming command printing is not disabled by the
-<option>-q</option>
+A &Python; function used to print the command lines as they are executed
+(unless command printing is disabled by the
+<link linkend="opt-question"><option>-q</option>/<option>--question</option></link>
 or
-<option>-s</option>
-options or their equivalents).
+<link linkend="opt-silent"><option>-s</option>/<option>--silent</option></link>
+options).
 The function must accept four arguments:
 <varname>s</varname>,
 <varname>target</varname>,

SCons/Defaults.xml

@@ -266,8 +266,8 @@ directly into &cv-link-CCFLAGS; or &cv-link-CXXFLAGS;
 as the result will be non-portable
 and the directories will not be searched by the dependency scanner.
 &cv-CPPPATH; should be a list of path strings,
-or a single string, not a pathname list joined by
-Python's <systemitem>os.pathsep</systemitem>.
+or a single string, not a pathname list joined by the
+&Python; <systemitem>os.pathsep</systemitem> character.
 </para>
 
 <para>
@@ -527,7 +527,7 @@ The list of directories that will be searched for libraries
 specified by the &cv-link-LIBS; &consvar;.
 &cv-LIBPATH; should be a list of path strings,
 or a single string, not a pathname list joined by
-Python's <systemitem>os.pathsep</systemitem>.
+the &Python; <systemitem>os.pathsep</systemitem> character.
 <!-- XXX OLD
 The implicit dependency scanner will search these
 directories for include files. Don't explicitly put include directory

SCons/PathList.py

@@ -30,6 +30,7 @@
 """
 
 import os
+import sys
 
 import SCons.Memoize
 import SCons.Node
@@ -216,6 +217,7 @@ def PathList(self, pathlist, split=True):
 
 PathList = PathListCache().PathList
 
-# TODO: removing the class object here means Sphinx doesn't pick up its
-#   docstrings: they're fine for reading here, but are not in API Docs.
-del PathListCache
+# Some trickery here so Sphinx can still see the class when building API docs
+_SPHINX_BUILD = os.environ.get("SPHINX_BUILD") == "1"
+if not _SPHINX_BUILD:
+    del PathListCache