Skip to content

bpo-21536: C extensions are no longer linked to libpython #12946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 25, 2019
Merged

bpo-21536: C extensions are no longer linked to libpython #12946

merged 1 commit into from
Apr 25, 2019

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Apr 25, 2019 •
edited by bedevere-bot
Loading

On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTDL_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"shared" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.

https://bugs.python.org/issue21536

@vstinner vstinner requested a review from pitrou April 25, 2019 01:28
@vstinner
Copy link
Member Author

This PR allows to load a C extension built by a shared libpython with a statically linked Python: https://bugs.python.org/issue21536#msg340819

It also allows to load C extension built in release mode with a Python built in debug mode: https://bugs.python.org/issue21536#msg340821

@vstinner
Copy link
Member Author

One potential issue: linking an application to libpython to embed Python, see discussion at https://bugs.python.org/issue21536#msg340837

pitrou
pitrou approved these changes Apr 25, 2019
View reviewed changes
Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you!

skoslowski
skoslowski reviewed Apr 25, 2019
View reviewed changes
@vstinner
bpo-21536: C extensions are no longer linked to libpython
14e4843 
On Unix, C extensions are no longer linked to libpython.

It is now possible to load a C extension built using a shared library
Python with a statically linked Python.

When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.

distutils, python-config and python-config.py have been modified.
@vstinner vstinner merged commit 8c3ecc6 into python:master Apr 25, 2019
@vstinner vstinner deleted the omit_libpython branch April 25, 2019 18:13
This was referenced May 24, 2019
Merged
Merged
@yarda yarda mentioned this pull request May 19, 2020
Closed
@sanjayankur31 sanjayankur31 mentioned this pull request May 28, 2020
Closed
gferon added a commit to gferon/libArcus that referenced this pull request Jun 2, 2020
@gferon
Don't link to libpython for Python >= 3.8
4ebc453 
As of Python 3.8, python C extensions modules should not link to
libpython, unless they embed the interpreter in their code.

Relevant upstream PR: python/cpython#12946

Signed-off-by: Gabriel Féron <[email protected]>
@gferon gferon mentioned this pull request Jun 2, 2020
Merged
gferon added a commit to gferon/libSavitar that referenced this pull request Jun 2, 2020
@gferon
Don't link to libpython for Python >= 3.8
fcea518 
As of Python 3.8, python C extensions modules should not link to
libpython, unless they embed the interpreter in their code.

Relevant upstream PR: python/cpython#12946

Signed-off-by: Gabriel Féron <[email protected]>
@gferon gferon mentioned this pull request Jun 2, 2020
Closed
@froody
Copy link

froody commented Dec 7, 2020 •
edited
Loading

This change causes strange bugs on Mac OS X due to the two-level namespace symbol resolution for dynamic libraries. A C extension that links against libpython will resolve symbols from libpython instead of the python executable (for example, PDAL/python#75 ). I believe linking C extensions on Mac OS X with "-flat_namespace" would be a reliable fix, as it would allow the python executable symbols to take precedence over those from libpython.

@vstinner
Copy link
Member Author

vstinner commented Dec 7, 2020

Please don't comment closed PRs, but open a new issue at https://bugs.python.org/ Otherwise, your comment will simply be ignored.

@krsfrodaslz krsfrodaslz mentioned this pull request May 2, 2022
Open
@pitrou pitrou mentioned this pull request Sep 2, 2022
Closed
@DownerCase DownerCase mentioned this pull request Feb 26, 2024
Merged
@messense messense mentioned this pull request Apr 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@skoslowski skoslowski skoslowski left review comments

@pitrou pitrou pitrou approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@vstinner @froody @skoslowski @pitrou @the-knights-who-say-ni @bedevere-bot