Python 3.11 support

[cheesycod]

Feature request: support Python 3.11 (right now it fails to compile)

[cheesycod]

Ok it works, but only with Cynthon from github

[cheesycod]

There should be a way to disable the cython version check with a Env var or smth

[tehmufifnman]

Python 3.11 is still in Alpha release stages and is likely far from feature complete. Additionally the chest is there because it is not terribly uncommon for their to be breaking changes from version to version.

I imagine once 3.11 is released we will see a new release of asyncpg

[AMDmi3]

I second this issue: support for alpha versions of python is required for working on compatibility of whole infrastructures with the upcoming versions. I currently work on python 3.11 support in FreeBSD ports, so this is critical to me. As @cheesycod says, it builds fine with cython from git. For that to work you need to remove cython version limitation from setup.py: #856. No other changes are needed.

[elprans]

Unpinning the critical dependency this way is not the right way to solve this. There is no guarantee of perfect forward compatibility in Cython and this might prevent older versions of asyncpg from building correctly in the future. Please use the 0.29.x Cython branch instead of master, it also carries the necessary 3.11 patches. The next version there is 0.29.5, which satisfies the version selector.

[eirnym]

@AMDmi3 It's great that these works are already in progress even at early stages. With python-only code there's no problem for it. However, I don't see many binary packages supporting early stages of a language version at least before feature-freeze

[AMDmi3]

@eirnym not sure what your remark relates to. From the packaging standpoint there's not much of a problem, as packages are built from source, and when building from source, it's trivial to remove generated files and regenerate them with actual version of cython. The only problem is when upstream deliberately makes this harder than it should be, such as by limiting dependency versions from above, or calling cython in unusual ways, but this can be as well fixed in the packaging process. If you refer to wheel users, they are naturally on their own.

[musicinmybrain]

For example, Fedora Linux is already doing test builds of all of its Python packages with an alpha version of 3.11 in preparation for Fedora 37 in late 2022, and filing bugs on the downstream packages where there are build or test failures. For a fast-moving distribution, it’s just not possible to provide the latest Python interpreter without dealing with incompatibilities in Python packages before the final release of a new Python version. That said, it’s still very early in the Python 3.11 development cycle, and we won’t switch the development version of Fedora over from 3.10 for more than six months. By then, feature changes will be done in Python 3.11 and it will be well into the beta phase.

In Fedora, we can’t respect versions pinned by upstreams. We need to build with the versions that are in the distribution, which are typically the latest stable ones. At best, we can update an out-of-date dependency, or temporarily delay an upgrade that breaks a lot of dependent packages. We also have a policy requirement to re-generate all Cythonized sources. In practice, this mostly works out fine, and where it doesn’t, we usually find out quickly because the build fails or some of the tests don’t pass (which fails our package build).

Currently, too many of asyncpg’s dependencies are still failing to build with Python 3.11 in Fedora, so there’s no telling whether asyncpg would work without patches in our environment or not. I’m the maintainer of the python-asyncpg package in Fedora Linux, so at some point in the next year I’ll get an automated bug report and start looking into any remaining issues.

[elprans]

The only problem is when upstream deliberately makes this harder than it should be

I'm all for making asyncpg easier to handle in distro packaging, but not at the expense of possibly breaking builds for everybody else. asyncpg is not a trivial Cython project, and we've been burned by bugs and incompatibilities in new Cython releases before. The ability to build historical asyncpg releases from source without hunting for correct versions of build dependencies is important!

[AMDmi3]

I'm all for making asyncpg easier to handle in distro packaging, but not at the expense of possibly breaking builds for everybody else

Do I understand correctly that the problem is that there's no way to differentiate the case where asyncpg developers are doing a source release (for which you want to use stable cython version which provides generated code compatible both ways to generate the code for tarball) and the case when someone packages asyncpg (for which limiting cython version does not make sense)?

[elprans]

It's not just about source releases on PyPI, it's about the ability to build any version of asyncpg from source. Package maintainers have the luxury of working in a well-defined and controlled environment, others do not. If all you need is the ability to override the version pin, and patching the line in setup.py is somehow hard, then I'll be happy to accept a PR with an environment variable override.

[Yun-Kim]

Hi there! Just wondering if there's a timeline for asyncpg to release a Python 3.11 compatible version, considering Python 3.11 is scheduled to be released in a few weeks. Thanks!

[elprans]

Will make a release soon, waiting for #954 to land.

[JP-Globality]

Any updates on here? #954 looks to have landed 🎉

[ddelange]

this issue can be closed, cp311 wheels are up: https://pypi.org/project/asyncpg/0.27.0/#files