1.31.0 source dist fails to build miri

[dtolnay]

I am seeing the following failure in what should be a clean build.

$ wget https://static.rust-lang.org/dist/rustc-1.31.0-src.tar.gz
$ tar xf rustc-1.31.0-src.tar.gz
$ cd rustc-1.31.0-src
$ sed 's/#extended = false/extended = true/' config.toml.example > config.toml
$ ./x.py build

...
Building stage2 tool miri (x86_64-unknown-linux-gnu)
   Compiling num-traits v0.2.5                                                                                      
   Compiling num-integer v0.1.39                                                                                    
   Compiling bitflags v1.0.4                                                                                        
   Compiling byteorder v1.2.3                                                                                       
   Compiling time v0.1.40                                                                                           
   Compiling failure v0.1.2                                                                                         
   Compiling chrono v0.4.4                                                                                          
   Compiling vergen v3.0.3                                                                                          
   Compiling miri v0.1.0 (rustc-1.31.0-src/src/tools/miri)               
error: failed to run custom build command for `miri v0.1.0 (rustc-1.31.0-src/src/tools/miri)`
process didn't exit successfully: `rustc-1.31.0-src/build/x86_64-unknown-linux-gnu/stage2-tools/release/build/miri-ffcc0501a1e0abe7/build-script-build` (exit code: 101)
--- stdout
cargo:rustc-env=PROFILE=release
cargo:rerun-if-changed=build.rs
cargo:rustc-env=VERGEN_SEMVER_LIGHTWEIGHT=0.1.0
cargo:rustc-env=VERGEN_TARGET_TRIPLE=x86_64-unknown-linux-gnu
cargo:rustc-env=VERGEN_SHA_SHORT=
cargo:rustc-env=VERGEN_SEMVER=0.1.0
cargo:rustc-env=VERGEN_BUILD_TIMESTAMP=2018-12-06T22:22:03.560705685+00:00
cargo:rustc-env=VERGEN_COMMIT_DATE=
cargo:rustc-env=VERGEN_SHA=
cargo:rustc-env=VERGEN_BUILD_DATE=2018-12-06

--- stderr
thread 'main' panicked at 'Unable to generate vergen keys!: Os { code: 2, kind: NotFound, message: "No such file or directory" }', libcore/result.rs:1009:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

command did not execute successfully: "rustc-1.31.0-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "28" "--release" "--manifest-path" "rustc-1.31.0-src/src/tools/miri/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101
Build completed successfully in 0:06:23

Mentioning @oli-obk in case you know of an easy fix.
Mentioning @zachreizner who hit this upgrading Chrome OS to Rust 1.31.

[oli-obk]

cc @RalfJung you added vergen to miri, do you have any idea what kind of file it tries to read and consequently fails due to "No such file or directory"?

[RalfJung]

Yes, it tries to determine the current git version. It reads .git/HEAD, figures out the current branch, and then reads e.g. .git/refs/heads/master.

Maybe this does not work for detached branches, like you often have with submodules?

[RalfJung]

Oh, this is from a source dist. I am not sure how the tarball is created, but it likely contains the miri sources but without the .git?

[oli-obk]

I am not sure how the tarball is created, but it likely contains the miri sources but without the .git?

Jup.

[oli-obk]

so... easy fix for now is to run git init, while we try to figure out how to make it work in the future.

[RalfJung]

Hm, I locally tried just not having a .git dir and then miri builds fine (and VERGEN_SHA is empty).

[oli-obk]

A .git dir in src/tools/miri or in the root?

[RalfJung]

Ah the problem is that locally I use vergen 3.0.4 but rustc still has vergen 3.0.3. I think a vergen update should do it.

[dtolnay]

@rust-lang/release -- if we end up doing a patch release I would like to propose pulling in this fix. It seems bad that we can't build Cargo etc from a clean rustc source dist.

[zachreizner]

If you do upgrade vergen, you'll probably encounter another issue in the source tarball version of miri like I did. The commit cbe6b22 removed the is_null method that is used by miri. You'll need a version of miri with at least rust-lang/miri@d890a70

[cuviper]

Miri fails, yes, but that's not a blocker for everything else:

Build completed successfully in 0:06:23

As I recall, it's been this way for some time in Fedora builds, but it's fine because we don't ship miri.

[zachreizner]

@cuviper: are you saying that extended builds from the souce tarball don't require miri to build successfully?

[RalfJung]

@zachreizner correct.

[cuviper]

@zachreizner right, you can see this in Fedora's build.log files here:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1170053

... miri fails, but is otherwise ignored, and everything else proceeds normally.