Unstable C API tier (PEP 689)

[encukou]

Add an Unstable C API tier as per PEP 689.

• Update CPython docs
• Update the Devguide (PEP 689: Add docs for unstable C API devguide#1060)
• Add initial unstable API from the PEP

Other candidates for inclusion are below.
These usually need discussion first. This is a checklist for having the discussion.
Please try to not hold the individual discussions in this issue.

• Dict watching API
• Functions from PEP-523 not specified in PEP-689 (_PyEval_EvalFrameDefault & co.) -- this is a can of worms tho
• FrameStack API (Add "unstable" frame stack api #91371)
• _Py_HashDouble, see PEP 689 -- Add an unstable C-API tier #91744 (comment)
• PyBytesObject.ob_shash sounds like a good candidate: see https://discuss.python.org/t/15108 (guess fields need a slightly different naming convention though. Sigh. That should have been in the PEP.)
• non-opaque access to frame structs and any other key APIs needed to implement alternate eval loops with comparable performance to the default eval loop (unless & until we can figure out stable public APIs that can deliver equivalent performance) - see Nick's reply
• C APIs that provide access to compiled code whether in AST or opcode form (the API itself may be stable, but the compiled code isn't) - see Nick's reply
• PyLong_FromByteArray/PyUnstable_LongToBase30Digits: https://discuss.python.org/t/20045 [there's now `PyLong_{From,To}NativeBytes
• Anything that starts with an underscore and is documented
• Fast access to PyLong contents [PyLong_Export]

Got any more?
I offer to move other API to the unstable tier myself, if there's a good usage example to base docs & regression tests on [edit 2026:] and a C API WG decision. Please discuss new ideas on Discourse.

Linked PRs

• gh-101101: Unstable C API tier (PEP 689) #101102
• gh-101101: Fix test_code_extra to reset value for refleak test #102350

[corona10]

@encukou #101102 caused the refleak test failure with my local machine
I submitted the related patch: #102350
See also: https://buildbot.python.org/all/#/builders/840/builds/390

[CAM-Gerlach]

Hey @encukou , with the feature freeze coming up, do you have an update on the status here?

[encukou]

Changes described the PEP are in. Waiting for SC approval for a change before I mark it Final: python/steering-council#185

I won't be able to start all the discussions I want before beta, so the issue will stay open.

[vstinner]

https://peps.python.org/pep-0689/ status is now Final. Can this issue be closed?

[encukou]

Sadly I haven't had as much time for the C API as I hoped. If you want to work on one of the items above, or move this to the C API WG, go ahead.

[mbechard]

I would like to request that the AST parsing be exposed as an Unstable C API as well. I use this to parse simple expressions that run code native to my app, rather than going through the Python interpreter which ultimately ends up back in C module code I've made using the Python C API. This is useful for common used expressions, as it gives a large speedup for the simple/common cases. The code falls back to the full interpreter when an expression includes items my parser doesn't handle.
In particular, I use
PyParser_ASTFromString
And then traverse the AST using the various enums and structs from pycore_ast.h