Streampager build failure due to flaky dependency failure

[markbt]

Crate name: streampager
Build failure link: https://docs.rs/crate/streampager/0.9.2/builds/336810

Additional details:

Version 0.9.2 failed to build, despite this being a minor cfg-attribute change over 0.9.1 which built fine.

It looks like the problem was during building of the pest_meta 2.1.3 dependency, which built fine in the previous version, and also built fine itself.

[jyn514]

SIGKILL most likely means your crate ran out of memory when compiling. How much memory does it take to compile streampager locally with cargo doc? The default limit is 3 GB.

[markbt]

Max RSS from time says it uses about 1.3GB. It's not a particularly complex library, so I'd be surprised if it hit any limits.

The failure actually happens before it even gets to the streampager library - it was killed compiling a dependency. The previous version (0.9.1) also compiled fine (and the only difference between the two is a missing cfg attribute).

I suspect some noisy neighbour caused the build to fail (perhaps by exhausting host memory), and if you tried the build again it would work fine. I don't know if you can kick off another build. It seems the only way I could do it myself would be to release 0.9.3.

[Nemo157]

I tested building locally and saw the same spike and SIGKILL happen. The crate which I saw spike the most was vtparse, testing just that it's hitting 3.6GB resident memory running cargo check, interestingly it only takes 290MB while runnning cargo doc, so this only affects dependers of it. (I really wish we had #878 to see how much memory https://docs.rs/crate/termwiz/0.11.0/builds took, maybe it just snuck in under the limit, while streampager's build-tree happens to compile some other larger crates in parallel with vtparse).

[markbt]

Ok. So it looks like it's a bit non-deterministic depending on what happens in parallel. For example 0.9.1 (https://docs.rs/crate/streampager/0.9.1/builds) built fine despite being essentially the same code.

I noticed that wezterm has a 5GB limit rather than the default 3GB. Maybe streampager needs a higher limit, too? Can that be changed?

Alternatively we could build fewer things in parallel. Is there a setting I can change for that?

[jyn514]

I would rather fix this upstream in rustc rather than having to special case every crate that depends on wezterm. I opened rust-lang/rust#81124.

[jyn514]

Alternatively we could build fewer things in parallel. Is there a setting I can change for that?

docs.rs currently always uses -j2; since cargo check uses so much memory I wouldn't expect less parallelism to help much. It's weird that 0.9.1 built successfully.

[jyn514]

Requeued and rebuilt: https://docs.rs/streampager/0.9.2/streampager/

I guess the memory improvements are working :)