manylinux1 packages contain unstripped, but unstrippable, shared libraries

[coreywright]

There are manylinux1 packages on PyPI that contain unstripped shared libraries that cause strip to fail.

dev@devenv:~$ virtualenv manylinux1_relocatable_test
New python executable in /home/dev/manylinux1_relocatable_test/bin/python
Installing setuptools, pip, wheel...done.
dev@devenv:~$ . manylinux1_relocatable_test/bin/activate
(manylinux1_relocatable_test) dev@devenv:~$ pip install cffi numpy
Collecting cffi
  Using cached cffi-1.7.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting numpy
  Using cached numpy-1.11.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pycparser (from cffi)
Installing collected packages: pycparser, cffi, numpy
Successfully installed cffi-1.7.0 numpy-1.11.1 pycparser-2.14
(manylinux1_relocatable_test) dev@devenv:~$ file manylinux1_relocatable_test/lib/python2.7/site-packages/_cffi_backend.so manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so
manylinux1_relocatable_test/lib/python2.7/site-packages/_cffi_backend.so:                             ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=d272756d64640cc3e6584d1a3db4ae0ae2990b48, not stripped
manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=85e6780ba62dd077bd2fbe1e765763d30ae25f10, not stripped
(manylinux1_relocatable_test) dev@devenv:~$ strip manylinux1_relocatable_test/lib/python2.7/site-packages/_cffi_backend.so manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so
BFD: manylinux1_relocatable_test/lib/python2.7/site-packages/stG8EFkj: Not enough room for program headers, try linking with -N
strip:manylinux1_relocatable_test/lib/python2.7/site-packages/stG8EFkj[.note.gnu.build-id]: Bad value
BFD: manylinux1_relocatable_test/lib/python2.7/site-packages/stG8EFkj: Not enough room for program headers, try linking with -N
strip:manylinux1_relocatable_test/lib/python2.7/site-packages/stG8EFkj: Bad value
BFD: manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/stZ7PY7C: Not enough room for program headers, try linking with -N
strip:manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/stZ7PY7C[.note.gnu.build-id]: Bad value
BFD: manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/stZ7PY7C: Not enough room for program headers, try linking with -N
strip:manylinux1_relocatable_test/lib/python2.7/site-packages/numpy/.libs/stZ7PY7C: Bad value

This behavior has been observed on both Debian 8, Ubuntu 14.04, and Ubuntu 16.04.

If this is not a general artifact of the manylinux build process (and therefor not applicable to this project specifically), but a common (mis)configuration that affects multiple packages' manylinux1 build configuration/settings (eg cffi, numpy), then please let me know and I'll close this issue (and reopen it against the relevant projects).

[njsmith]

I suspect that the reason they can't be stripped is a side effect of the way auditwheel uses patchelf to patch binaries when it's bundling libraries.

I think the solution is probably for auditwheel to be taught how to automatically/optionally strip binaries.

[njsmith]

Do you know how much stripping actually affects these binaries?

[coreywright]

i wondered the same question, "how much is stripping these binaries saving", but realized after a second or two that without being able to strip the binaries (the very thing that is failing and i tried on ubuntu 14.04 and 16.04 with no success) that i'm not sure of an easy method to calculate the size reduction (as i don't look forward to manually reviewing the symbols in the shared libraries, resolving which are not necessary, and calculating their collective size; that's what strip is for 😉).

and it's not so much that these specific libraries can't be stripped, regardless of their size as they are just prominent examples, but that stripping fails on manylinux1 packages, which is quite the oddity/irregularity/abnormality. i can alter my predeployment-packaging step to ignore this particular failure while stripping shared libraries, but figured i would report it "upstream" first because it is definitely "unique".

thanks for the tip; i'll look into auditwheel (if only out of curiosity and enlightenment).

[coreywright]

okay, after reading through auditwheel, specifically repair.py and noting it's strong dependency on patchelf (as you mentioned, @njsmith), i found this similar-sounding patchelf issue: NixOS/patchelf#10.

[ogrisel]

I did some stripping tests on numpy / scipy / scikit-learn in the past (before manylinux existed) and the size reduction can be very large: a typical scipy stack venv would go from ~250MB down to less than 50MB.

This was for instance documented in: https://serverlesscode.com/post/deploy-scikitlearn-on-lamba/

[ogrisel]

@njsmith I think the solution is probably for auditwheel to be taught how to automatically/optionally strip binaries.

I agree if the problem cannot be easily fixed at the patchelf level.

[njsmith]

It would be very difficult to fix this at the patchelf level.

…

On Jan 7, 2017 3:58 AM, "Olivier Grisel" ***@***.***> wrote: @njsmith <https://github.com/njsmith> I think the solution is probably for auditwheel to be taught how to automatically/optionally strip binaries. I agree if the problem cannot be easily fixed at the patchelf level. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAlOaML5wiQoqvWGiWFEWQl4gTf_hAO6ks5rP33hgaJpZM4JUsG5> .

[ogrisel]

@njsmith I tried to build this suggested fix from NixOS/patchelf#10 (comment) but I cannot get the bootstrap.sh script to work on our manylinux1 docker container. How did you generate the distribution files in the https://nipy.bic.berkeley.edu/manylinux/patchelf-0.9njs2.tar.gz tarball?

[matthew-brett]

Olivier - I think that's me. I just ran the bootstrap.sh script on another machine with more recent autoconf tools, and then packed up the archive with the resulting configure file. Is that what you mean?

[ogrisel]

Yes I tried something like that but also need other files no? Like build-aux or something.

[matthew-brett]

I just checked out that commit, and ran bootstrap.sh on it, then started the manylinux docker container pointing to the resulting directory, and did ./configure and make - worked OK for me. What problems are you having?

[ogrisel]

I tried again and it worked. I don't understand what I did wrong the first time. Sorry for the noise.

[sciyoshi]

NixOS/patchelf#117 has been merged - it'd be great if the Docker images could include this fix soon, since this breaks the Debian virtualenv packaging workflow: spotify/dh-virtualenv#150

[njsmith]

@sciyoshi: any interest in working on a patch?

[sciyoshi]

@njsmith opened #109