gh-141376: make smelly: List all symbols, repeat smelly ones at end

[encukou]

Without colours, the script's output hard to scan for my spoiled eyes.

The messages are adjusted to no longer imply a lack of exceptions to the rules.

(This PR is not tightly related to #141376 but I think it's best to attach it to that issue.)

[encukou]

@AA-Turner @hugovk How can I inform precommit that _colorize is part of the stdandard library?

[vstinner]

Instead of colors, I would prefer to not print each tested binary unless if has "smelly" symbols.

[hugovk]

@AA-Turner @hugovk How can I inform precommit that _colorize is part of the stdandard library?

Add this to Tools/build/.ruff.toml:

diff --git a/Tools/build/.ruff.toml b/Tools/build/.ruff.toml
index 996f725fdcb..e7852abe131 100644
--- a/Tools/build/.ruff.toml
+++ b/Tools/build/.ruff.toml
@@ -31,6 +31,9 @@ ignore = [
     "PYI025",  # Use `from collections.abc import Set as AbstractSet`
 ]
 
+[lint.isort]
+extra-standard-library = ["_colorize"]
+
 [lint.per-file-ignores]
 "{check_extension_modules,freeze_modules}.py" = [
     "UP031",  # Use format specifiers instead of percent format

Docs: https://docs.astral.sh/ruff/settings/#lint_isort_extra-standard-library

And note it'll stick it first in the stdlib import list.

[encukou]

@hugovk, thanks! Maybe I'll not need it here but hopefully I'll remember next time.

@vstinner, would you be OK with something like https://github.com/encukou/cpython/blob/make-smelly-refactor/Tools/build/smelly.py ?

[vstinner]

would you be OK with something like https://github.com/encukou/cpython/blob/make-smelly-refactor/Tools/build/smelly.py ?

Yes, something like that 👍

[encukou]

OK; I've changed the script to instead:

• List all symbols as they're discovered (which is useful to have in the logs, for checking what the script does)
• List smelly symbols at the end (so you don't have to dig for them in the verbose output)

This way it's fine without colours!

[encukou]

This version includes removing exceptions (as in #141392).

[vstinner]

List all symbols as they're discovered (which is useful to have in the logs, for checking what the script does)

Oh. I expected the opposite: only display smelly symbols and be silent if a binary has no smelly symbols.

make smelly|wc -l now returns 3,666 lines. It's way more verbose.

[encukou]

It's more verbose, but that part scrolls up on a terminal; the important part is at the end so it stays visible.
I think it's good to have the full list in logs. But if you disagree strongly I can remove it, of course.

[vstinner]

Would it be possible/make sense to add a command line option to control if Python symbols are displayed or not? And don't show up Python symbols by default?

[encukou]

OK, here it is.

[vstinner]

I tested the change with:

diff --git a/Objects/object.c b/Objects/object.c
index 4fc692bb029..9176f3522f4 100644
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -893,7 +893,7 @@ PyObject_Bytes(PyObject *v)
     return PyBytes_FromObject(v);
 }
 
-static void
+void
 clear_freelist(struct _Py_freelist *freelist, int is_finalization,
                freefunc dofree)
 {
@@ -908,7 +908,7 @@ clear_freelist(struct _Py_freelist *freelist, int is_finalization,
     }
 }
 
-static void
+void
 free_object(void *obj)
 {
     PyObject *op = (PyObject *)obj;

The script works as expected:

./python ./Tools/build/smelly.py

Found 2 smelly symbols in total!
    - clear_freelist from libpython3.15d.a
    - free_object from libpython3.15d.a
make: *** [Makefile:3491: smelly] Error 1

Build FAILED with exit code 2 (1.897 sec)

The (default) output is now short and readable!

The verbose output is, well, more verbose :-)

$ ./python ./Tools/build/smelly.py -v|wc -l
3682

[vstinner]

LGTM