Merge remote-tracking branch 'upstream/main' into specialize_overridden

Author: Fidget-Spinner

.gitattributes

@@ -28,8 +28,8 @@ Lib/test/cjkencodings/*                    noeol
 Lib/test/coding20731.py                    noeol
 Lib/test/decimaltestdata/*.decTest         noeol
 Lib/test/test_email/data/*.txt             noeol
-Lib/test/test_importlib/data01/*           noeol
-Lib/test/test_importlib/namespacedata01/*  noeol
+Lib/test/test_importlib/resources/data01/*           noeol
+Lib/test/test_importlib/resources/namespacedata01/*  noeol
 Lib/test/xmltestdata/*                     noeol
 
 # CRLF files

.github/CODEOWNERS

@@ -61,7 +61,7 @@ Python/pythonrun.c            @iritkatriel
 # bytecode.
 **/*import*.c                 @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw
 **/*import*.py                @brettcannon @encukou @ericsnowcurrently @ncoghlan @warsaw
-**/importlib/resources/*      @jaraco @warsaw @brettcannon
+**/*importlib/resources/*      @jaraco @warsaw @brettcannon
 **/importlib/metadata/*       @jaraco @warsaw
 
 # Dates and times

.github/CONTRIBUTING.rst

@@ -6,19 +6,19 @@ Build Status
 
 - main
 
-  + `Stable buildbots <http://buildbot.python.org/3.x.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.x.stable/>`_
 
 - 3.9
 
-  + `Stable buildbots <http://buildbot.python.org/3.9.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.9.stable/>`_
 
 - 3.8
 
-  + `Stable buildbots <http://buildbot.python.org/3.8.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.8.stable/>`_
 
 - 3.7
 
-  + `Stable buildbots <http://buildbot.python.org/3.7.stable/>`_
+  + `Stable buildbots <https://buildbot.python.org/3.7.stable/>`_
 
 
 Thank You

.github/ISSUE_TEMPLATE/bug.md

@@ -15,12 +15,12 @@ labels: "type-bug"
     your problem has already been reported
 -->
 
-**Bug report**
+# Bug report
 
 A clear and concise description of what the bug is.
 Include a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example), if possible.
 
-**Your environment**
+# Your environment
 
 <!-- Include as many relevant details as possible about the environment you experienced the bug in -->
 

.github/ISSUE_TEMPLATE/crash.md

@@ -13,15 +13,15 @@ labels: "type-crash"
   For CPython, a "crash" is when Python itself fails, leading to a traceback in the C stack.
 -->
 
-**Crash report**
+# Crash report
 
 Tell us what happened, ideally including a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example).
 
-**Error messages**
+# Error messages
 
 Enter any relevant error message caused by the crash, including a core dump if there is one.
 
-**Your environment**
+# Your environment
 
 <!-- Include as many relevant details as possible about the environment you experienced the bug in -->
 

.github/ISSUE_TEMPLATE/documentation.md

@@ -4,6 +4,6 @@ about: Report a problem with the documentation
 labels: "docs"
 ---
 
-**Documentation**
+# Documentation
 
 (A clear and concise description of the issue.)

.github/ISSUE_TEMPLATE/feature.md

@@ -4,16 +4,16 @@ about: Submit a proposal for a new CPython feature or enhancement
 labels: "type-feature"
 ---
 
-**Feature or enhancement**
+# Feature or enhancement
 
 (A clear and concise description of your proposal.)
 
-**Pitch**
+# Pitch
 
 (Explain why this feature or enhancement should be implemented and how it would be used.
  Add examples, if applicable.)
 
-**Previous discussion**
+# Previous discussion
 
 <!--
   New features to Python should first be discussed elsewhere before creating issues on GitHub,

.github/workflows/doc.yml

@@ -78,4 +78,4 @@ jobs:
       run: make -C Doc/ PYTHON=../python venv
     # Use "xvfb-run" since some doctest tests open GUI windows
     - name: 'Run documentation doctest'
-      run: xvfb-run make -C Doc/ PYTHON=../python SPHINXOPTS="-q" SPHINXERRORHANDLING="-W --keep-going" doctest
+      run: xvfb-run make -C Doc/ PYTHON=../python SPHINXERRORHANDLING="-W --keep-going" doctest

Doc/about.rst

@@ -6,7 +6,7 @@ About these documents
 These documents are generated from `reStructuredText`_ sources by `Sphinx`_, a
 document processor specifically written for the Python documentation.
 
-.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _reStructuredText: https://docutils.sourceforge.io/rst.html
 .. _Sphinx: http://sphinx-doc.org/
 
 .. In the online version of these documents, you can submit comments and suggest
@@ -21,7 +21,7 @@ Many thanks go to:
 
 * Fred L. Drake, Jr., the creator of the original Python documentation toolset
   and writer of much of the content;
-* the `Docutils <http://docutils.sourceforge.net/>`_ project for creating
+* the `Docutils <https://docutils.sourceforge.io/>`_ project for creating
   reStructuredText and the Docutils suite;
 * Fredrik Lundh for his Alternative Python Reference project from which Sphinx
   got many good ideas.

Doc/c-api/call.rst

@@ -57,6 +57,15 @@ This bears repeating:
    A class supporting vectorcall **must** also implement
    :c:member:`~PyTypeObject.tp_call` with the same semantics.
 
+.. versionchanged:: 3.12
+
+   The :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is now removed from a class
+   when the class's :py:meth:`~object.__call__` method is reassigned.
+   (This internally sets :c:member:`~PyTypeObject.tp_call` only, and thus
+   may make it behave differently than the vectorcall function.)
+   In earlier Python versions, vectorcall should only be used with
+   :const:`immutable <Py_TPFLAGS_IMMUTABLETYPE>` or static types.
+
 A class should not implement vectorcall if that would be slower
 than *tp_call*. For example, if the callee needs to convert
 the arguments to an args tuple and kwargs dict anyway, then there is no point

Doc/c-api/code.rst

@@ -90,3 +90,28 @@ bound into a function.
 
    .. versionadded:: 3.11
 
+.. c:function:: PyObject* PyCode_GetVarnames(PyCodeObject *co)
+
+   Equivalent to the Python code ``getattr(co, 'co_varnames')``.
+   Returns a new reference to a :c:type:`PyTupleObject` containing the names of
+   the local variables. On error, ``NULL`` is returned and an exception
+   is raised.
+
+   .. versionadded:: 3.11
+
+.. c:function:: PyObject* PyCode_GetCellvars(PyCodeObject *co)
+
+   Equivalent to the Python code ``getattr(co, 'co_cellvars')``.
+   Returns a new reference to a :c:type:`PyTupleObject` containing the names of
+   the local variables that are referenced by nested functions. On error, ``NULL``
+   is returned and an exception is raised.
+
+   .. versionadded:: 3.11
+
+.. c:function:: PyObject* PyCode_GetFreevars(PyCodeObject *co)
+
+   Equivalent to the Python code ``getattr(co, 'co_freevars')``.
+   Returns a new reference to a :c:type:`PyTupleObject` containing the names of
+   the free variables. On error, ``NULL`` is returned and an exception is raised.
+
+   .. versionadded:: 3.11

Doc/c-api/long.rst

@@ -84,14 +84,15 @@ distinguished from a number.  Use :c:func:`PyErr_Occurred` to disambiguate.
 .. c:function:: PyObject* PyLong_FromString(const char *str, char **pend, int base)
 
    Return a new :c:type:`PyLongObject` based on the string value in *str*, which
-   is interpreted according to the radix in *base*.  If *pend* is non-``NULL``,
-   *\*pend* will point to the first character in *str* which follows the
-   representation of the number.  If *base* is ``0``, *str* is interpreted using
-   the :ref:`integers` definition; in this case, leading zeros in a
-   non-zero decimal number raises a :exc:`ValueError`. If *base* is not ``0``,
-   it must be between ``2`` and ``36``, inclusive.  Leading spaces and single
-   underscores after a base specifier and between digits are ignored.  If there
-   are no digits, :exc:`ValueError` will be raised.
+   is interpreted according to the radix in *base*, or ``NULL`` on failure.  If
+   *pend* is non-``NULL``, *\*pend* will point to the end of *str* on success or
+   to the first character that could not be processed on error.  If *base* is ``0``,
+   *str* is interpreted using the :ref:`integers` definition; in this case, leading
+   zeros in a non-zero decimal number raises a :exc:`ValueError`.  If *base* is not
+   ``0``, it must be between ``2`` and ``36``, inclusive.  Leading and trailing
+   whitespace and single underscores after a base specifier and between digits are
+   ignored.  If there are no digits or *str* is not NULL-terminated following the
+   digits and trailing whitespace, :exc:`ValueError` will be raised.
 
 
 .. c:function:: PyObject* PyLong_FromUnicodeObject(PyObject *u, int base)

Doc/c-api/object.rst

@@ -126,6 +126,14 @@ Object Protocol
    A generic implementation for the getter of a ``__dict__`` descriptor. It
    creates the dictionary if necessary.
 
+   This function may also be called to get the :py:attr:`~object.__dict__`
+   of the object *o*. Pass ``NULL`` for *context* when calling it.
+   Since this function may need to allocate memory for the
+   dictionary, it may be more efficient to call :c:func:`PyObject_GetAttr`
+   when accessing an attribute on the object.
+
+   On failure, returns ``NULL`` with an exception set.
+
    .. versionadded:: 3.3
 
 
@@ -137,6 +145,16 @@ Object Protocol
    .. versionadded:: 3.3
 
 
+.. c:function:: PyObject** _PyObject_GetDictPtr(PyObject *obj)
+
+   Return a pointer to :py:attr:`~object.__dict__` of the object *obj*.
+   If there is no ``__dict__``, return ``NULL`` without setting an exception.
+
+   This function may need to allocate memory for the
+   dictionary, so it may be more efficient to call :c:func:`PyObject_GetAttr`
+   when accessing an attribute on the object.
+
+
 .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid)
 
    Compare the values of *o1* and *o2* using the operation specified by *opid*,

Doc/c-api/typeobj.rst

@@ -135,7 +135,7 @@ Quick Reference
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
    | [:c:member:`~PyTypeObject.tp_cache`]           | :c:type:`PyObject` *              |                   |   |   |       |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
-   | [:c:member:`~PyTypeObject.tp_subclasses`]      | :c:type:`PyObject` *              | __subclasses__    |   |   |       |
+   | [:c:member:`~PyTypeObject.tp_subclasses`]      | void *                            | __subclasses__    |   |   |       |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
    | [:c:member:`~PyTypeObject.tp_weaklist`]        | :c:type:`PyObject` *              |                   |   |   |       |
    +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
@@ -720,29 +720,29 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    with the *vectorcallfunc* function.
    This can be done by setting *tp_call* to :c:func:`PyVectorcall_Call`.
 
-   .. warning::
-
-      It is not recommended for :ref:`mutable heap types <heap-types>` to implement
-      the vectorcall protocol.
-      When a user sets :attr:`__call__` in Python code, only *tp_call* is updated,
-      likely making it inconsistent with the vectorcall function.
-
    .. versionchanged:: 3.8
 
       Before version 3.8, this slot was named ``tp_print``.
       In Python 2.x, it was used for printing to a file.
       In Python 3.0 to 3.7, it was unused.
 
+   .. versionchanged:: 3.12
+
+      Before version 3.12, it was not recommended for
+      :ref:`mutable heap types <heap-types>` to implement the vectorcall
+      protocol.
+      When a user sets :attr:`~type.__call__` in Python code, only *tp_call* is
+      updated, likely making it inconsistent with the vectorcall function.
+      Since 3.12, setting ``__call__`` will disable vectorcall optimization
+      by clearing the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag.
+
    **Inheritance:**
 
    This field is always inherited.
    However, the :const:`Py_TPFLAGS_HAVE_VECTORCALL` flag is not
-   always inherited. If it's not, then the subclass won't use
+   always inherited. If it's not set, then the subclass won't use
    :ref:`vectorcall <vectorcall>`, except when
    :c:func:`PyVectorcall_Call` is explicitly called.
-   This is in particular the case for types without the
-   :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set (including subclasses defined in
-   Python).
 
 
 .. c:member:: getattrfunc PyTypeObject.tp_getattr
@@ -1178,12 +1178,18 @@ and :c:type:`PyType_Type` effectively act as defaults.)
 
       **Inheritance:**
 
-      This bit is inherited for types with the
-      :const:`Py_TPFLAGS_IMMUTABLETYPE` flag set, if
-      :c:member:`~PyTypeObject.tp_call` is also inherited.
+      This bit is inherited if :c:member:`~PyTypeObject.tp_call` is also
+      inherited.
 
       .. versionadded:: 3.9
 
+      .. versionchanged:: 3.12
+
+         This flag is now removed from a class when the class's
+         :py:meth:`~object.__call__` method is reassigned.
+
+         This flag can now be inherited by mutable classes.
+
    .. data:: Py_TPFLAGS_IMMUTABLETYPE
 
       This bit is set for type objects that are immutable: type attributes cannot be set nor deleted.
@@ -1709,18 +1715,11 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    :c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is
    at the very end of the structure.
 
-   The real dictionary offset in an instance can be computed from a negative
-   :c:member:`~PyTypeObject.tp_dictoffset` as follows::
-
-      dictoffset = tp_basicsize + abs(ob_size)*tp_itemsize + tp_dictoffset
-      if dictoffset is not aligned on sizeof(void*):
-          round up to sizeof(void*)
-
-   where :c:member:`~PyTypeObject.tp_basicsize`, :c:member:`~PyTypeObject.tp_itemsize` and :c:member:`~PyTypeObject.tp_dictoffset` are
-   taken from the type object, and :attr:`ob_size` is taken from the instance.  The
-   absolute value is taken because ints use the sign of :attr:`ob_size` to
-   store the sign of the number.  (There's never a need to do this calculation
-   yourself; it is done for you by :c:func:`_PyObject_GetDictPtr`.)
+   The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only.
+   To get the pointer to the dictionary call :c:func:`PyObject_GenericGetDict`.
+   Calling :c:func:`PyObject_GenericGetDict` may need to allocate memory for the
+   dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr`
+   when accessing an attribute on the object.
 
    **Inheritance:**
 
@@ -1928,9 +1927,17 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    This field is not inherited.
 
 
-.. c:member:: PyObject* PyTypeObject.tp_subclasses
+.. c:member:: void* PyTypeObject.tp_subclasses
+
+   A collection of subclasses.  Internal use only.  May be an invalid pointer.
+
+   To get a list of subclasses, call the Python method
+   :py:meth:`~class.__subclasses__`.
 
-   List of weak references to subclasses.  Internal use only.
+   .. versionchanged:: 3.12
+
+      For some types, this field does not hold a valid :c:expr:`PyObject*`.
+      The type was changed to :c:expr:`void*` to indicate this.
 
    **Inheritance:**
 
@@ -1942,6 +1949,13 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    Weak reference list head, for weak references to this type object.  Not
    inherited.  Internal use only.
 
+   .. versionchanged:: 3.12
+
+      Internals detail: For the static builtin types this is always ``NULL``,
+      even if weakrefs are added.  Instead, the weakrefs for each are stored
+      on ``PyInterpreterState``.  Use the public C-API or the internal
+      ``_PyObject_GET_WEAKREFS_LISTPTR()`` macro to avoid the distinction.
+
    **Inheritance:**
 
    This field is not inherited.

Doc/c-api/unicode.rst

@@ -477,9 +477,6 @@ APIs:
    |                   |                     | :c:func:`PyObject_Repr`.         |
    +-------------------+---------------------+----------------------------------+
 
-   An unrecognized format character causes all the rest of the format string to be
-   copied as-is to the result string, and any extra arguments discarded.
-
    .. note::
       The width formatter unit is number of characters rather than bytes.
       The precision formatter unit is number of bytes for ``"%s"`` and
@@ -500,6 +497,11 @@ APIs:
       Support width and precision formatter for ``"%s"``, ``"%A"``, ``"%U"``,
       ``"%V"``, ``"%S"``, ``"%R"`` added.
 
+   .. versionchanged:: 3.12
+      An unrecognized format character now sets a :exc:`SystemError`.
+      In previous versions it caused all the rest of the format string to be
+      copied as-is to the result string, and any extra arguments discarded.
+
 
 .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs)
 
@@ -819,7 +821,7 @@ wchar_t Support
    most C functions. If *size* is ``NULL`` and the :c:type:`wchar_t*` string
    contains null characters a :exc:`ValueError` is raised.
 
-   Returns a buffer allocated by :c:func:`PyMem_Alloc` (use
+   Returns a buffer allocated by :c:func:`PyMem_New` (use
    :c:func:`PyMem_Free` to free it) on success. On error, returns ``NULL``
    and *\*size* is undefined. Raises a :exc:`MemoryError` if memory allocation
    is failed.