gh-95853: Add script to automate WASM build (GH-95828)

[tiran]

Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

configure now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
wasm64-emscripten.

• Issue: Add script to automate WebAssembly builds #95853

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit a7aa566 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @tiran for commit fb38556 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[brettcannon]

Getting failures on MacOS.

Built with ./Tools/wasm/wasm_build.py wasi compile > wasi_compile.txt 2>&1 (python is Python 3.10): wasi_compile.txt

Testing with ./Tools/wasm/wasm_build.py wasi test:

❯ ./Tools/wasm/wasm_build.py wasi test
make: realpath: Command not found
The following modules are *disabled* in configure script:
_sqlite3

The necessary bits to build these optional modules were not found:
_bz2                  _ctypes               _hashlib
_lzma                 _ssl                  _testimportmultiple
_testmultiphase       _uuid                 readline
xxlimited             xxlimited_35          zlib
To find the necessary bits, look in configure.ac and config.log.

Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer

Checked 109 modules (75 built-in, 0 shared, 21 n/a on wasi-wasm32, 1 disabled, 12 missing, 0 failed on import)
make: realpath: Command not found
_PYTHON_HOSTRUNNER='wasmtime run --env PYTHONPATH=//build/lib.wasi-wasm32-3.12:/Lib --mapdir /::/Users/brettcannon/Repositories/python/cpython/builddir/wasi/../.. --' _PYTHON_PROJECT_BASE=/Users/brettcannon/Repositories/python/cpython/builddir/wasi _PYTHON_HOST_PLATFORM=wasi-wasm32 PYTHONPATH=/Users/brettcannon/Repositories/python/cpython/builddir/wasi/build/lib.wasi-wasm32-3.12:/Users/brettcannon/Repositories/python/cpython/builddir/wasi/../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__wasi_wasm32-wasi /Users/brettcannon/Repositories/python/cpython/builddir/build/python.exe  /Users/brettcannon/Repositories/python/cpython/builddir/wasi/../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=1200 -i '*.test_srcdir' -j2
/Users/brettcannon/Repositories/python/cpython/builddir/build/python.exe -u -W default -bb -m test -r -w --python 'wasmtime run --env PYTHONPATH=//build/lib.wasi-wasm32-3.12:/Lib --mapdir /::/Users/brettcannon/Repositories/python/cpython/builddir/wasi/../.. -- python.wasm' -j 1 -u all -W --slowest --fail-env-changed --timeout=1200 -i '*.test_srcdir' -j2
Raised RLIMIT_NOFILE: 256 -> 1024
== CPython 3.12.0a0 (heads/wasm_build-dirty:66f6fd8897, Aug 12 2022, 15:51:21) [Clang 12.0.5 (clang-1205.0.22.9)]
== macOS-12.3.1-x86_64-i386-64bit little-endian
== cwd: /Users/brettcannon/Repositories/python/cpython/builddir/wasi/build/test_python_90919æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
Using random seed 5054115
0:00:00 load avg: 7.71 Run tests in parallel using 2 child processes (timeout: 20 min, worker timeout: 25 min)
0:00:06 load avg: 7.73 [  1/435/1] test_uuid crashed (Exit code 1)
Could not find platform independent libraries <prefix>
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Lib/test/regrtest.py", line 11, in <module>
    from test.libregrtest import main
  File "/Lib/test/libregrtest/__init__.py", line 1, in <module>
    from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES, ALL_RESOURCES
  File "/Lib/test/libregrtest/cmdline.py", line 5, in <module>
    from test.support import os_helper
  File "/Lib/test/support/__init__.py", line 1229, in <module>
    not sysconfig.get_config_var('WITH_DOC_STRINGS'))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Lib/sysconfig.py", line 715, in get_config_var
    return get_config_vars().get(name)
           ^^^^^^^^^^^^^^^^^
  File "/Lib/sysconfig.py", line 670, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/Lib/sysconfig.py", line 531, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata__wasi_wasm32-wasi'

Then that import failure repeats.

[brettcannon]

Why the one compile example but the two test examples? I would just choose a single build target (e.g. wasi), show the end-to-end, and start the example by saying you're using wasi as an example.

[tiran]

Because emscripten-browser target does not support test verb. We do not have a test runner for browsers.

[tiran]

The test failure is caused by the fact that macOS does not have realpath command make: realpath: Command not found. The HOSTRUNNER has wrong PYTHONPATH env --env PYTHONPATH=//build/lib.wasi-wasm32-3.12:/Lib. It should be /builddir/wasi/build/lib.wasi-wasm32-3.12:/Lib.

[tiran]

Happy type checker is happy

$ mypy Tools/wasm/wasm_build.py 
Success: no issues found in 1 source file

[tiran]

I have opened #95952 to track the test issue on macOS.

[miss-islington]

Thanks @tiran for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

[bedevere-bot]

GH-95968 is a backport of this pull request to the 3.11 branch.