Skip to content

[DO NOT MERGE] experiment CI with doctest-xcompile #119147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

ehuss
Copy link
Contributor

@ehuss ehuss commented Dec 20, 2023

No description provided.

@ehuss ehuss self-assigned this Dec 20, 2023
@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Dec 20, 2023
@rust-log-analyzer
Copy link
Collaborator

The job arm-android failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
......ii................................................................................  704/1137
...................FFFF.................................................................  792/1137
...i................................i...................................................  880/1137
........................................................................................  968/1137
FF...F..FF.............................................................................. 1056/1137

failures:

---- library/std/src/os/net/linux_ext/addr.rs - os::net::linux_ext::addr::SocketAddrExt::as_abstract_name (line 49) stdout ----
---- library/std/src/os/net/linux_ext/addr.rs - os::net::linux_ext::addr::SocketAddrExt::as_abstract_name (line 49) stdout ----
error[E0433]: failed to resolve: could not find `linux` in `os`
  |
4 | use std::os::linux::net::SocketAddrExt;
4 | use std::os::linux::net::SocketAddrExt;
  |              ^^^^^ could not find `linux` in `os`
error[E0599]: no function or associated item named `from_abstract_name` found for struct `std::os::unix::net::SocketAddr` in the current scope
##[error]   --> library/std/src/os/net/linux_ext/addr.rs:55:33
    |
    |
8   |     let name_addr = SocketAddr::from_abstract_name(name)?;
    |                                 ^^^^^^^^^^^^^^^^^^ function or associated item not found in `SocketAddr`
    |
note: if you're trying to build a new `std::os::unix::net::SocketAddr` consider using one of the following associated functions:
      std::os::unix::net::SocketAddr::from_parts
      std::os::unix::net::SocketAddr::from_pathname
   --> /checkout/library/std/src/os/unix/net/addr.rs:94:5
    |
    |
94  | /     pub(super) fn new<F>(f: F) -> io::Result<SocketAddr>
95  | |     where
96  | |         F: FnOnce(*mut libc::sockaddr, *mut libc::socklen_t) -> libc::c_int,
...
106 | /     pub(super) fn from_parts(
107 | |         addr: libc::sockaddr_un,
107 | |         addr: libc::sockaddr_un,
108 | |         mut len: libc::socklen_t,
109 | |     ) -> io::Result<SocketAddr> {
...
...
152 | /     pub fn from_pathname<P>(path: P) -> io::Result<SocketAddr>
153 | |     where
154 | |         P: AsRef<Path>,
    = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
    |
3   + use std::os::android::net::SocketAddrExt;
3   + use std::os::android::net::SocketAddrExt;
    |
help: there is an associated function with a similar name
    |
8   |     let name_addr = SocketAddr::from_pathname(name)?;

error[E0599]: no method named `as_abstract_name` found for struct `std::os::unix::net::SocketAddr` in the current scope
##[error]  --> library/std/src/os/net/linux_ext/addr.rs:58:27
   |
   |
11 |     assert_eq!(local_addr.as_abstract_name(), Some(&name[..]));
   |
  ::: /checkout/library/std/src/os/net/linux_ext/addr.rs:63:8
   |
   |
63 |     fn as_abstract_name(&self) -> Option<&[u8]>;
   |        ---------------- the method is available for `std::os::unix::net::SocketAddr` here
   = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
   |
3  + use std::os::android::net::SocketAddrExt;
3  + use std::os::android::net::SocketAddrExt;
   |
help: there is a method with a similar name
   |
11 |     assert_eq!(local_addr.as_pathname(), Some(&name[..]));

error: aborting due to 3 previous errors

Some errors have detailed explanations: E0433, E0599.
Some errors have detailed explanations: E0433, E0599.
For more information about an error, try `rustc --explain E0433`.
Couldn't compile the test.
---- library/std/src/os/net/linux_ext/addr.rs - os::net::linux_ext::addr::SocketAddrExt::from_abstract_name (line 24) stdout ----
error[E0433]: failed to resolve: could not find `linux` in `os`
  |
4 | use std::os::linux::net::SocketAddrExt;
4 | use std::os::linux::net::SocketAddrExt;
  |              ^^^^^ could not find `linux` in `os`
error[E0599]: no function or associated item named `from_abstract_name` found for struct `std::os::unix::net::SocketAddr` in the current scope
##[error]   --> library/std/src/os/net/linux_ext/addr.rs:29:28
    |
    |
7   |     let addr = SocketAddr::from_abstract_name(b"hidden")?;
    |                            ^^^^^^^^^^^^^^^^^^ function or associated item not found in `SocketAddr`
    |
note: if you're trying to build a new `std::os::unix::net::SocketAddr` consider using one of the following associated functions:
      std::os::unix::net::SocketAddr::from_parts
      std::os::unix::net::SocketAddr::from_pathname
   --> /checkout/library/std/src/os/unix/net/addr.rs:94:5
    |
    |
94  | /     pub(super) fn new<F>(f: F) -> io::Result<SocketAddr>
95  | |     where
96  | |         F: FnOnce(*mut libc::sockaddr, *mut libc::socklen_t) -> libc::c_int,
...
106 | /     pub(super) fn from_parts(
107 | |         addr: libc::sockaddr_un,
107 | |         addr: libc::sockaddr_un,
108 | |         mut len: libc::socklen_t,
109 | |     ) -> io::Result<SocketAddr> {
...
...
152 | /     pub fn from_pathname<P>(path: P) -> io::Result<SocketAddr>
153 | |     where
154 | |         P: AsRef<Path>,
    = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
    |
3   + use std::os::android::net::SocketAddrExt;
3   + use std::os::android::net::SocketAddrExt;
    |
help: there is an associated function with a similar name
    |
7   |     let addr = SocketAddr::from_pathname(b"hidden")?;

error: aborting due to 2 previous errors

Some errors have detailed explanations: E0433, E0599.
Some errors have detailed explanations: E0433, E0599.
For more information about an error, try `rustc --explain E0433`.
Couldn't compile the test.
---- library/std/src/os/net/linux_ext/tcp.rs - os::net::linux_ext::tcp::TcpStreamExt::set_quickack (line 26) stdout ----
error[E0433]: failed to resolve: could not find `linux` in `os`
  |
6 | use std::os::linux::net::TcpStreamExt;
6 | use std::os::linux::net::TcpStreamExt;
  |              ^^^^^ could not find `linux` in `os`

error[E0599]: no method named `set_quickack` found for struct `TcpStream` in the current scope
   |
   |
10 | stream.set_quickack(true).expect("set_quickack call failed");
   |        ^^^^^^^^^^^^ method not found in `TcpStream`
  ::: /checkout/library/std/src/os/net/linux_ext/tcp.rs:36:8
   |
36 |     fn set_quickack(&self, quickack: bool) -> io::Result<()>;
36 |     fn set_quickack(&self, quickack: bool) -> io::Result<()>;
   |        ------------ the method is available for `TcpStream` here
   = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
   |
4  + use std::os::android::net::TcpStreamExt;
4  + use std::os::android::net::TcpStreamExt;
   |

error: aborting due to 2 previous errors

Some errors have detailed explanations: E0433, E0599.
For more information about an error, try `rustc --explain E0433`.
Couldn't compile the test.
---- library/std/src/os/net/linux_ext/tcp.rs - os::net::linux_ext::tcp::TcpStreamExt::quickack (line 44) stdout ----
error[E0433]: failed to resolve: could not find `linux` in `os`
  |
6 | use std::os::linux::net::TcpStreamExt;
6 | use std::os::linux::net::TcpStreamExt;
  |              ^^^^^ could not find `linux` in `os`

error[E0599]: no method named `set_quickack` found for struct `TcpStream` in the current scope
   |
   |
10 | stream.set_quickack(true).expect("set_quickack call failed");
   |        ^^^^^^^^^^^^ method not found in `TcpStream`
  ::: /checkout/library/std/src/os/net/linux_ext/tcp.rs:36:8
   |
36 |     fn set_quickack(&self, quickack: bool) -> io::Result<()>;
36 |     fn set_quickack(&self, quickack: bool) -> io::Result<()>;
   |        ------------ the method is available for `TcpStream` here
   = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
   |
4  + use std::os::android::net::TcpStreamExt;
4  + use std::os::android::net::TcpStreamExt;
   |

error[E0599]: no method named `quickack` found for struct `TcpStream` in the current scope
   |
   |
11 | assert_eq!(stream.quickack().unwrap_or(false), true);
   |                   ^^^^^^^^ method not found in `TcpStream`
  ::: /checkout/library/std/src/os/net/linux_ext/tcp.rs:55:8
   |
55 |     fn quickack(&self) -> io::Result<bool>;
55 |     fn quickack(&self) -> io::Result<bool>;
   |        -------- the method is available for `TcpStream` here
   = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
   |
4  + use std::os::android::net::TcpStreamExt;
---
---- library/std/src/process.rs - process::ExitStatusError (line 1721) stdout ----
Test executable failed (exit status: 101).

stdout:
uploaded "/tmp/rustdoctestIaABWt/rust_out", waiting for result
stderr:
thread 'main' panicked at library/std/src/process.rs:9:32:
called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }
stack backtrace:
---
---- library/std/src/process.rs - process::Stdio::from (line 1509) stdout ----
Test executable failed (exit status: 101).

stdout:
uploaded "/tmp/rustdoctestl1TDfb/rust_out", waiting for result
stderr:
thread 'main' panicked at library/std/src/process.rs:18:12:
called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }
stack backtrace:
---
    library/std/src/process.rs - process::Stdio::from (line 1540)

test result: FAILED. 1111 passed; 9 failed; 17 ignored; 0 measured; 0 filtered out; finished in 44.59s

error: doctest failed, to rerun pass `-p std --doc`
  local time: Wed Dec 20 01:45:10 UTC 2023
  network time: Wed, 20 Dec 2023 01:45:10 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

@ehuss ehuss closed this Dec 21, 2023
github-merge-queue bot pushed a commit to rust-lang/cargo that referenced this pull request May 14, 2025
This stabilizes the doctest-xcompile feature by unconditionally enabling
it.

Closes #7040
Closes #12118

## What is being stabilized?

This changes it so that cargo will run doctests when using the
`--target` flag for a target that is not the host. Previously, cargo
would ignore doctests (and show a note if passing `--verbose`).

A wrapper for running the doctest can be specified with the
[`target.*.runner`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplerunner)
configuration option (which is powered by the `--test-runtool` rustdoc
flag). This would typically be something like qemu to run under
emulation. It is my understanding that this should work just like
running other kinds of tests.

Additionally, the
[`target.*.linker`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker)
config option is honored for using a custom linker.

Already stabilized in rustdoc is the ability to [ignore tests
per-target](https://doc.rust-lang.org/nightly/rustdoc/write-documentation/documentation-tests.html#ignoring-targets).

## Motivation

The lack of doctest cross-compile support has always been simply due to
the lack of functionality in rustdoc to support this. Rustdoc gained the
ability to cross-compile doctests some time ago, but there were
additional flags like the test runner that were not stabilized until
just recently.

This is intended to ensure that projects have full test coverage even
when doing cross-compilation. It can be
[surprising](#12118) to some
that this was not happening, particularly since cargo is silent about
it.

## Risks

The cargo team had several conversations about how to roll out this
feature. Ultimately we decided to enable it unconditionally with the
understanding that most projects will probably want to have their
doctests covered, and that any breakage will be a local concern that can
be resolved by either fixing the test or ignoring the target.

Tests in rust-lang/rust run into this issue, [particularly on
android](rust-lang/rust#119147 (comment)),
and those will need to be fixed before this reaches beta. This is
something I am looking into.

Some cross-compiling scenarios may need codegen flags that are not
supported. It's not clear how common this will be, or if ignoring will
be a solution, or how difficult it would be to update rustdoc and cargo
to support these. Additionally, the split between RUSTFLAGS and
RUSTDOCFLAGS can be cumbersome.

## Implementation history

- rust-lang/rust#60387 -- Support added to
rustdoc to support the `--target` flag and runtool and
per-target-ignores.
- #6892 -- Initial support in
cargo.
- #7391 -- Added unstable
documentation.
- #8094 -- Fix target for doc
test cross compilation
- #8358 -- Fixed regression with
`--target=HOST` not working on stable.
- #10132 -- Added note about
doctests not running (under `--verbose`).
- rust-lang/rust#112751 -- Fixed
`--test-run-directory` interaction with `--test-runtool`.
- rust-lang/rust#137096 -- Stabilization (and
rename) of the rustdoc `--test-runtool` and `--test-runtool-arg` CLI
args, and drops `--enable-per-target-ignores` unconditionally enabling
it.

## Test coverage

Cargo tests:
-
[artifact_dep::cross_doctests_works_with_artifacts](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/artifact_dep.rs#L1248-L1326)
-- Checks that doctest has access to the artifact dependencies.
-
[build_script::duplicate_script_with_extra_env](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/build_script.rs#L5514-L5614)
-- Checks that build-script env and cfg values are correctly handled on
host versus target when cross running doctests.
-
[cross_compile::cross_tests](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L416-L502)
-- Basic test that cross-compiled tests work.
-
[cross_compile::doctest_xcompile_linker](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L1139-L1182)
-- Checks that the linker config argument works.
-
[custom_target::custom_target_minimal](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/custom_target.rs#L39-L71)
-- Checks that `.json` targets work with rustdoc cross tests.
-
[test::cargo_test_doctest_xcompile_ignores](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/test.rs#L4743-L4777)
-- Checks the `ignore-*` syntax works.
-
[test::cargo_test_doctest_xcompile_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4783-L4863)
-- Checks runner with cross doctests.
-
[test::cargo_test_doctest_xcompile_no_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4869-L4907)
-- Checks cross doctests without a runner.

Rustdoc tests:
-
[run-make/doctest-runtool](https://github.com/rust-lang/rust/tree/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/run-make/doctests-runtool)
-- Tests behavior of `--test-run-directory` with relative paths of the
runner.
-
[rustdoc/doctest/doctest-runtool](https://github.com/rust-lang/rust/blob/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/rustdoc/doctest/doctest-runtool.rs)
-- Tests for `--test-runtool` and `--test-runtool-arg`.

## Future concerns

There have been some discussions
(rust-lang/testing-devex-team#5) about
changing how doctests are driven. My understanding is that stabilizing
this should not affect those plans, since if cargo becomes the driver,
it will simply need to build things with `--target` and use the
appropriate runner.

## Change notes

This PR changed tests a little:
- artifact_dep::no_cross_doctests_works_with_artifacts was changed now
that doctests actually work.
- cross_compile::cross_tests was changed to properly check doctests.
- cross_compile::no_cross_doctests dropped since it is no longer
relevant.
- standard_lib::doctest didn't need `-Zdoctest-xcompile` since
`-Zbuild-std` no longer uses a target.
- test::cargo_test_doctest_xcompile was removed since it is a duplicate
of cross_compile::cross_tests

I think this should probably wait until the next release cutoff, moving
this to 1.89 (will update the PR accordingly if that happens).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants