From e6c058e18c7a63e831a98b41d0d9df0e0c4901b4 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Thu, 21 May 2015 16:45:38 +0100 Subject: [PATCH 1/5] Rename 'link-args' to 'advanced-linking', add intro --- src/doc/trpl/SUMMARY.md | 2 +- .../trpl/{link-args.md => advanced-linking.md} | 15 +++++++++++---- 2 files changed, 12 insertions(+), 5 deletions(-) rename src/doc/trpl/{link-args.md => advanced-linking.md} (62%) diff --git a/src/doc/trpl/SUMMARY.md b/src/doc/trpl/SUMMARY.md index ca3381ffba465..f1e51591aea0e 100644 --- a/src/doc/trpl/SUMMARY.md +++ b/src/doc/trpl/SUMMARY.md @@ -63,7 +63,7 @@ * [No stdlib](no-stdlib.md) * [Intrinsics](intrinsics.md) * [Lang items](lang-items.md) - * [Link args](link-args.md) + * [Advanced linking](advanced-linking.md) * [Benchmark Tests](benchmark-tests.md) * [Box Syntax and Patterns](box-syntax-and-patterns.md) * [Slice Patterns](slice-patterns.md) diff --git a/src/doc/trpl/link-args.md b/src/doc/trpl/advanced-linking.md similarity index 62% rename from src/doc/trpl/link-args.md rename to src/doc/trpl/advanced-linking.md index cdaef6cd9b510..f54799d9e4a38 100644 --- a/src/doc/trpl/link-args.md +++ b/src/doc/trpl/advanced-linking.md @@ -1,6 +1,13 @@ -% Link args +% Advanced Linking -There is one other way to tell rustc how to customize linking, and that is via +The common cases of linking with Rust have been covered earlier in this book, +but supporting the range of linking possibilities made available by other +languages is important for Rust to achieve seamless interaction with native +libraries. + +# Link args + +There is one other way to tell `rustc` how to customize linking, and that is via the `link_args` attribute. This attribute is applied to `extern` blocks and specifies raw flags which need to get passed to the linker when producing an artifact. An example usage would be: @@ -14,9 +21,9 @@ extern {} ``` Note that this feature is currently hidden behind the `feature(link_args)` gate -because this is not a sanctioned way of performing linking. Right now rustc +because this is not a sanctioned way of performing linking. Right now `rustc` shells out to the system linker, so it makes sense to provide extra command line -arguments, but this will not always be the case. In the future rustc may use +arguments, but this will not always be the case. In the future `rustc` may use LLVM directly to link native libraries, in which case `link_args` will have no meaning. From e5c1884c26cf41a338e903ec60f58bdad49af886 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Thu, 21 May 2015 16:47:41 +0100 Subject: [PATCH 2/5] Note possiblities of empty extern blocks (based on #12575) --- src/doc/reference.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/doc/reference.md b/src/doc/reference.md index a37e1c146681e..26fd2fd8d20d6 100644 --- a/src/doc/reference.md +++ b/src/doc/reference.md @@ -1636,6 +1636,10 @@ The type of a function declared in an extern block is `extern "abi" fn(A1, ..., An) -> R`, where `A1...An` are the declared types of its arguments and `R` is the declared return type. +It is valid to add the `link` attribute on an empty extern block. You can use +this to satisfy the linking requirements of extern blocks elsewhere in your code +(including upstream crates) instead of adding the attribute to each extern block. + ## Visibility and Privacy These two terms are often used interchangeably, and what they are attempting to From 7af865b33348730aacc02afeb3233cf8d90bb1f2 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Thu, 21 May 2015 19:55:51 +0100 Subject: [PATCH 3/5] Static linking --- src/doc/trpl/advanced-linking.md | 127 +++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) diff --git a/src/doc/trpl/advanced-linking.md b/src/doc/trpl/advanced-linking.md index f54799d9e4a38..ea5238c381466 100644 --- a/src/doc/trpl/advanced-linking.md +++ b/src/doc/trpl/advanced-linking.md @@ -30,3 +30,130 @@ meaning. It is highly recommended to *not* use this attribute, and rather use the more formal `#[link(...)]` attribute on `extern` blocks instead. +# Static linking + +Static linking refers to the process of creating output that contain all +required libraries and so don't need libraries installed on every system where +you want to use your compiled project. Pure-Rust dependencies are statically +linked by default so you can use created binaries and libraries without +installing the Rust everywhere. By contrast, native libraries +(e.g. `libc` and `libm`) usually dynamically linked, but it is possible to +change this and statically link them as well. + +Linking is a very platform dependent topic - on some platforms, static linking +may not be possible at all! This section assumes some basic familiarity with +linking on your platform on choice. + +## Linux + +By default, all Rust programs on Linux will link to the system `libc` along with +a number of other libraries. Let's look at an example on a 64-bit linux machine +with GCC and `glibc` (by far the most common `libc` on Linux): + +``` text +$ cat example.rs +fn main() {} +$ rustc example.rs +$ ldd example + linux-vdso.so.1 => (0x00007ffd565fd000) + libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa81889c000) + libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa81867e000) + librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa818475000) + libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa81825f000) + libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa817e9a000) + /lib64/ld-linux-x86-64.so.2 (0x00007fa818cf9000) + libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa817b93000) +``` + +Dynamic linking on Linux can be undesirable if you wish to use new library +features on old systems or target systems which do not have the required +dependencies for your program to run. + +The first step in using static linking is examining the Rust linking arguments +with an option to rustc. Newlines have been added for readability: + +``` text +$ rustc example.rs -Z print-link-args +"cc" + "-Wl,--as-needed" + "-m64" + [...] + "-o" "example" + "example.o" + "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" + "-Wl,--gc-sections" + "-pie" + "-nodefaultlibs" + [...] + "-Wl,--whole-archive" "-Wl,-Bstatic" + "-Wl,--no-whole-archive" "-Wl,-Bdynamic" + "-ldl" "-lpthread" "-lrt" "-lgcc_s" "-lpthread" "-lc" "-lm" "-lcompiler-rt" +``` + +Arguments with a `-L` before them set up the linker search path and arguments +ending with `.rlib` are linking Rust crates statically into your application. +Neither of these are relevent for static linking so have been ommitted. + +The first step in being able to statically link is to obtain an object file. +This can be achieved with `rustc --emit obj example.rs`, and creates a file +called `example.o`, which you can see being passed in the command line above - +rustc automatically deletes it when finished with it by default. As you now have +the object file, you should be able to run the link command obtained with +`print-link-args` to create perform the linking stage yourself. + +In order to statically link, there are a number of changes you must make. Below +is the command required to perform a static link; we will go through them each +in turn. + +``` text +$ rustc example.rs -Z print-link-args +"cc" + "-static" + "-m64" + [...] + "-o" "example" + "example.o" + "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" + "-Wl,--gc-sections" + "-nodefaultlibs" + [...] + "-Wl,--whole-archive" + "-Wl,--no-whole-archive" + "-ldl" "-lpthread" "-lrt" "-lgcc_eh" "-lpthread" "-lc" "-lm" "-lcompiler-rt" +``` + + - `-static` was added - this is the signal to the compiler to use a static + glibc, among other things + - `-Wl,--as-needed` was removed - this can be left in, but is unnecessary + as it only applies to dynamic librares + - `-pie` was removed - this is not compatible with static binaries + - both `-Wl,-B*` options were removed - everything will be linked statically, + so informing the linker of how certain libraries should be linked is not + appropriate + - `-lgcc_s` was changed to `-lgcc_eh` - `gcc_s` is the GCC support library, + which Rust uses for unwinding support. This is only available as a dynamic + library, so we must specify the static version of the library providing + unwinding support. + +By running this command, you will likely see some warnings like + +``` text +warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking +``` + +These should be considered carefully! They indicate calls in glibc which +*cannot* be statically linked without significant extra effort. An application +using these calls will find it is not as portable as 'static binary' would imply. +Rust supports targeting musl as an alternative libc to be able to fully +statically link these calls. + +As we are confident that our code does not use these calls, we can now see the +fruits of our labour: + +``` +$ ldd example + not a dynamic executable +``` + +This binary can be copied to virtually any 64-bit Linux machine and work +without requiring external libraries. From a8a49fbfc9c8433a03c67007fb4ad30672cfb483 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Mon, 15 Jun 2015 18:54:37 +0100 Subject: [PATCH 4/5] musl static linking not glibc --- src/doc/trpl/advanced-linking.md | 152 +++++++++++++++---------------- 1 file changed, 71 insertions(+), 81 deletions(-) diff --git a/src/doc/trpl/advanced-linking.md b/src/doc/trpl/advanced-linking.md index ea5238c381466..614b7893e969e 100644 --- a/src/doc/trpl/advanced-linking.md +++ b/src/doc/trpl/advanced-linking.md @@ -40,14 +40,14 @@ installing the Rust everywhere. By contrast, native libraries (e.g. `libc` and `libm`) usually dynamically linked, but it is possible to change this and statically link them as well. -Linking is a very platform dependent topic - on some platforms, static linking +Linking is a very platform dependent topic — on some platforms, static linking may not be possible at all! This section assumes some basic familiarity with -linking on your platform on choice. +linking on your platform of choice. ## Linux By default, all Rust programs on Linux will link to the system `libc` along with -a number of other libraries. Let's look at an example on a 64-bit linux machine +a number of other libraries. Let's look at an example on a 64-bit Linux machine with GCC and `glibc` (by far the most common `libc` on Linux): ``` text @@ -69,91 +69,81 @@ Dynamic linking on Linux can be undesirable if you wish to use new library features on old systems or target systems which do not have the required dependencies for your program to run. -The first step in using static linking is examining the Rust linking arguments -with an option to rustc. Newlines have been added for readability: - -``` text -$ rustc example.rs -Z print-link-args -"cc" - "-Wl,--as-needed" - "-m64" - [...] - "-o" "example" - "example.o" - "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" - "-Wl,--gc-sections" - "-pie" - "-nodefaultlibs" - [...] - "-Wl,--whole-archive" "-Wl,-Bstatic" - "-Wl,--no-whole-archive" "-Wl,-Bdynamic" - "-ldl" "-lpthread" "-lrt" "-lgcc_s" "-lpthread" "-lc" "-lm" "-lcompiler-rt" -``` - -Arguments with a `-L` before them set up the linker search path and arguments -ending with `.rlib` are linking Rust crates statically into your application. -Neither of these are relevent for static linking so have been ommitted. - -The first step in being able to statically link is to obtain an object file. -This can be achieved with `rustc --emit obj example.rs`, and creates a file -called `example.o`, which you can see being passed in the command line above - -rustc automatically deletes it when finished with it by default. As you now have -the object file, you should be able to run the link command obtained with -`print-link-args` to create perform the linking stage yourself. - -In order to statically link, there are a number of changes you must make. Below -is the command required to perform a static link; we will go through them each -in turn. - -``` text -$ rustc example.rs -Z print-link-args -"cc" - "-static" - "-m64" - [...] - "-o" "example" - "example.o" - "-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" - "-Wl,--gc-sections" - "-nodefaultlibs" - [...] - "-Wl,--whole-archive" - "-Wl,--no-whole-archive" - "-ldl" "-lpthread" "-lrt" "-lgcc_eh" "-lpthread" "-lc" "-lm" "-lcompiler-rt" +Static linking is supported via an alternative `libc`, `musl` - this must be +enabled at Rust compile-time with some prerequisites available. You can compile +your own version of Rust with `musl` enabled and install it into a custom +directory with the instructions below: + +```text +$ mkdir musldist +$ PREFIX=$(pwd)/musldist +$ +$ # Build musl +$ wget http://www.musl-libc.org/releases/musl-1.1.10.tar.gz +[...] +$ tar xf musl-1.1.10.tar.gz +$ cd musl-1.1.10/ +musl-1.1.10 $ ./configure --disable-shared --prefix=$PREFIX +[...] +musl-1.1.10 $ make +[...] +musl-1.1.10 $ make install +[...] +musl-1.1.10 $ cd .. +$ du -h musldist/lib/libc.a +2.2M musldist/lib/libc.a +$ +$ # Build libunwind.a +$ wget http://llvm.org/releases/3.6.1/llvm-3.6.1.src.tar.xz +$ tar xf llvm-3.6.1.src.tar.xz +$ cd llvm-3.6.1.src/projects/ +llvm-3.6.1.src/projects $ svn co http://llvm.org/svn/llvm-project/libcxxabi/trunk/ libcxxabi +llvm-3.6.1.src/projects $ svn co http://llvm.org/svn/llvm-project/libunwind/trunk/ libunwind +llvm-3.6.1.src/projects $ sed -i 's#^\(include_directories\).*$#\0\n\1(../libcxxabi/include)#' libunwind/CMakeLists.txt +llvm-3.6.1.src/projects $ mkdir libunwind/build +llvm-3.6.1.src/projects $ cd libunwind/build +llvm-3.6.1.src/projects/libunwind/build $ cmake -DLLVM_PATH=../../.. -DLIBUNWIND_ENABLE_SHARED=0 .. +llvm-3.6.1.src/projects/libunwind/build $ make +llvm-3.6.1.src/projects/libunwind/build $ cp lib/libunwind.a $PREFIX/lib/ +llvm-3.6.1.src/projects/libunwind/build $ cd cd ../../../../ +$ du -h musldist/lib/libunwind.a +164K musldist/lib/libunwind.a +$ +$ # Build musl-enabled rust +$ git clone https://github.com/rust-lang/rust.git muslrust +$ cd muslrust +muslrust $ ./configure --target=x86_64-unknown-linux-musl --musl-root=$PREFIX --prefix=$PREFIX +muslrust $ make +muslrust $ make install +muslrust $ cd .. +$ du -h musldist/bin/rustc +12K musldist/bin/rustc ``` - - `-static` was added - this is the signal to the compiler to use a static - glibc, among other things - - `-Wl,--as-needed` was removed - this can be left in, but is unnecessary - as it only applies to dynamic librares - - `-pie` was removed - this is not compatible with static binaries - - both `-Wl,-B*` options were removed - everything will be linked statically, - so informing the linker of how certain libraries should be linked is not - appropriate - - `-lgcc_s` was changed to `-lgcc_eh` - `gcc_s` is the GCC support library, - which Rust uses for unwinding support. This is only available as a dynamic - library, so we must specify the static version of the library providing - unwinding support. - -By running this command, you will likely see some warnings like +You now have a build of a `musl`-enabled Rust! Because we've installed it to a +custom prefix we need to make sure our system can the binaries and appropriate +libraries when we try and run it: -``` text -warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking +```text +$ export PATH=$PREFIX/bin:$PATH +$ export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH ``` -These should be considered carefully! They indicate calls in glibc which -*cannot* be statically linked without significant extra effort. An application -using these calls will find it is not as portable as 'static binary' would imply. -Rust supports targeting musl as an alternative libc to be able to fully -statically link these calls. +Let's try it out! -As we are confident that our code does not use these calls, we can now see the -fruits of our labour: - -``` +```text +$ echo 'fn main() { println!("hi!"); panic!("failed"); }' > example.rs +$ rustc --target=x86_64-unknown-linux-musl example.rs $ ldd example not a dynamic executable +$ ./example +hi! +thread '
' panicked at 'failed', example.rs:1 ``` -This binary can be copied to virtually any 64-bit Linux machine and work -without requiring external libraries. +Success! This binary can be copied to almost any Linux machine with the same +machine architecture and run without issues. + +`cargo build` also permits the `--target` option so you should be able to build +your crates as normal. However, you may need to recompile your native libraries +against `musl` before they can be linked against. From b6a0d9e17843a2b078e491e10f64a4005b3a7791 Mon Sep 17 00:00:00 2001 From: Aidan Hobson Sayers Date: Mon, 15 Jun 2015 18:58:28 +0100 Subject: [PATCH 5/5] Additional notes to link-args --- src/doc/trpl/advanced-linking.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/doc/trpl/advanced-linking.md b/src/doc/trpl/advanced-linking.md index 614b7893e969e..6d37043354282 100644 --- a/src/doc/trpl/advanced-linking.md +++ b/src/doc/trpl/advanced-linking.md @@ -22,10 +22,12 @@ extern {} Note that this feature is currently hidden behind the `feature(link_args)` gate because this is not a sanctioned way of performing linking. Right now `rustc` -shells out to the system linker, so it makes sense to provide extra command line +shells out to the system linker (`gcc` on most systems, `link.exe` on MSVC), +so it makes sense to provide extra command line arguments, but this will not always be the case. In the future `rustc` may use LLVM directly to link native libraries, in which case `link_args` will have no -meaning. +meaning. You can achieve the same effect as the `link-args` attribute with the +`-C link-args` argument to `rustc`. It is highly recommended to *not* use this attribute, and rather use the more formal `#[link(...)]` attribute on `extern` blocks instead.