Skip to content

Fix last_green build after disable_autoloads flip #1738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mbland
Copy link
Contributor

@mbland mbland commented May 17, 2025

Description

Explicitly loads all symbols affected by --incompatible_disable_autoloads_in_main_repo to fix last_green Bazel builds. Adds rules_shell as a dev dependency and updates files to fix test_bzlmod_macros.sh and test_dependency_versions.sh.

As a result of using Buildifier to add the missing load statements, Buildifier reordered the load statements as well.

Fixes the following breakage when using the last_green build from bazelbuild/bazel@f08d561 (and other breakages blocked by this one):

$ bazel run //tools:lint_check

ERROR: scala/private/rules/scala_test.bzl:130:21:
  name 'JavaInfo' is not defined

ERROR: .../external/+dev_deps+com_github_bazelbuild_buildtools/buildifier/BUILD.bazel:3:10:
  While resolving toolchains for target
  @@+dev_deps+com_github_bazelbuild_buildtools//buildifier:buildifier
  (b5bbebe): invalid registered toolchain
  '//test/proto:scalapb_toolchain':
  error loading package 'test/proto':
  at /Users/mbland/src/bazel-contrib/rules_scala/scala/scala.bzl:35:5:
  compilation of module 'scala/private/rules/scala_test.bzl' failed

ERROR: Analysis of target '//tools:lint_check' failed; build aborted:
  Analysis failed

Fixed using Buildifier 8.2.0 by first running the following to get the list of appropriate warnings:

buildifier --lint=warn -r . 2>&1 | grep -- native- |
  awk '{print $2}' | sort | uniq

then running:

buildifier --lint=fix \
  --warnings=native-java-common,native-java-info,native-sh-test,native-sh-binary \
  -r .

I also ran it separately on the changed files in deps/test and scala/private/macros/test, since Buildifier didn't automatically find them.

Motivation

bazelbuild/bazel@d87eaf5 from 2025-05-08 at 8:03am EDT flipped
--incompatible_disable_autoloads_in_main_repo to true to prepare for Bazel 9. I did confirm that setting
--noincompatible_disable_autoloads_in_main_repo resolved the breakage before using Buildifier to add the necessary load statements.

Pull request #1735 was the last to pass the last_green CI build using bazelbuild/bazel@94e0b44 from 2025-05-08 at 7:39am EDT. This was two commits before the commit that flipped --incompatible_disable_autoloads_in_main_repo:

Pull request #1736 failed the last_green CI build using bazelbuild/bazel@cac54c4 from 2025-05-09 at 6:14am EDT. That job ran on 2025-05-09 at 6:47am EDT. The last_green configuration in .bazelci/presubmit.yml emits a warning if the build fails. This warning was present on the build page, but we've overlooked it until now:

Part of this change copies more files from test/jmh into the test_dependency_versions.sh test directory. It also replicates targets from //test/jmh in deps/test/BUILD.bazel.test to replace the previous dependency on @rules_scala//test/jmh:add_numbers.

This is because //test/jmh now has a load dependency on @rules_shell, which is a dev_dependency of @rules_scala. Since @rules_shell isn't visible to @rules_scala when it isn't the root module, this broke the build within test_dependency_versions.sh.

Explicitly loads all symbols affected by
`--incompatible_disable_autoloads_in_main_repo` to fix `last_green`
Bazel builds. Adds `rules_shell` as a dev dependency and updates files
to fix `test_bzlmod_macros.sh` and `test_dependency_versions.sh`.

As a result of using Buildifier to add the missing `load` statements,
Buildifier reordered the `load` statements as well.

Fixes the following breakage when using the `last_green` build from
bazelbuild/bazel@f08d561 (and other
breakages blocked by this one):

```sh
$ bazel run //tools:lint_check

ERROR: scala/private/rules/scala_test.bzl:130:21:
  name 'JavaInfo' is not defined

ERROR: .../external/+dev_deps+com_github_bazelbuild_buildtools/buildifier/BUILD.bazel:3:10:
  While resolving toolchains for target
  @@+dev_deps+com_github_bazelbuild_buildtools//buildifier:buildifier
  (b5bbebe): invalid registered toolchain
  '//test/proto:scalapb_toolchain':
  error loading package 'test/proto':
  at /Users/mbland/src/bazel-contrib/rules_scala/scala/scala.bzl:35:5:
  compilation of module 'scala/private/rules/scala_test.bzl' failed

ERROR: Analysis of target '//tools:lint_check' failed; build aborted:
  Analysis failed
```

Fixed using Buildifier 8.2.0 by first running the following to get the
list of appropriate warnings:

```txt
buildifier --lint=warn -r . 2>&1 | grep -- native- |
  awk '{print $2}' | sort | uniq
```

then running:

```txt
buildifier --lint=fix \
  --warnings=native-java-common,native-java-info,native-sh-test,native-sh-binary \
  -r .
```

I also ran it separately on the changed files in `deps/test` and
`scala/private/macros/test`, since Buildifier didn't automatically find
them.

---

bazelbuild/bazel@d87eaf5 from
2025-05-08 at 8:03am EDT flipped
`--incompatible_disable_autoloads_in_main_repo` to `true` to prepare for
Bazel 9. I did confirm that setting
`--noincompatible_disable_autoloads_in_main_repo` resolved the breakage
before using Buildifier to add the necessary `load` statements.

Pull request bazel-contrib#1735 was the last to pass the `last_green` CI build using
bazelbuild/bazel@94e0b44 from
2025-05-08 at 7:39am EDT. This was two commits before the commit that
flipped `--incompatible_disable_autoloads_in_main_repo`:

- https://buildkite.com/bazel/rules-scala-scala/builds/5596#0196afc2-4af3-4202-8232-d5f5fe113349

Pull request bazel-contrib#1736 failed the `last_green` CI build using
bazelbuild/bazel@cac54c4 from
2025-05-09 at 6:14am EDT. That job ran on 2025-05-09 at 6:47am EDT. The
`last_green` configuration in `.bazelci/presubmit.yml` emits a warning
if the build fails. This warning was present on the build page, but
we've overlooked it until now:

- https://buildkite.com/bazel/rules-scala-scala/builds/5598#0196b4a8-e593-431a-ab4c-8d8471d9b08b

Part of this change copies more files from `test/jmh` into the
`test_dependency_versions.sh` test directory. It also replicates targets
from `//test/jmh` in `deps/test/BUILD.bazel.test` to replace
the previous dependency on `@rules_scala//test/jmh:add_numbers`.

This is because `//test/jmh` now has a `load` dependency on
`@rules_shell`, which is a `dev_dependency` of `@rules_scala`. Since
`@rules_shell` isn't visible to `@rules_scala` when it isn't the root
module, this broke the build within `test_dependency_versions.sh`.
@mbland mbland requested review from liucijus and simuons as code owners May 17, 2025 20:03
The following build experienced this seemingly transient error in
`./test_coverage on macOS` (all other jobs in the same build passed):

- https://buildkite.com/bazel/rules-scala-scala/builds/5608#0196dfd9-3da9-4fbd-b284-e63e849f9d0a/89-128

```txt
WARNING: Download from
https://github.com/protocolbuffers/protobuf/releases/download/v30.2/protoc-30.2-osx-x86_64.zip
failed: class java.io.IOException GET returned 500 Internal Server Error
```
mbland added a commit to EngFlow/example that referenced this pull request May 17, 2025
Explicitly loads all symbols affected by
`--incompatible_disable_autoloads_in_main_repo` to fix `last_green`
Bazel builds. Adds the flag to `.bazelrc`, adds `rules_shell` as a
dependency, and contains lots of other Buildifier formatting touch ups.

bazelbuild/bazel@d87eaf5 from
2025-05-08 at 8:03am EDT flipped
`--incompatible_disable_autoloads_in_main_repo` to `true` to prepare for
Bazel 9. See also bazel-contrib/rules_scala#1738.

Before this change, setting the flag caused these errors:

```txt
$ bazel build --incompatible_disable_autoloads_in_main_repo //...

ERROR: java/com/engflow/notificationqueue/demoserver/BUILD:5:1:
  name 'proto_library' is not defined
ERROR: /demoserver/BUILD:10:1:
  name 'java_proto_library' is not defined (did you mean 'java_grpc_library'?)
ERROR: docker/network/BUILD:19:10: //docker/network:docker-network-test:
  no such attribute 'srcs' in 'java_test' rule
ERROR: docker/network/BUILD:19:10: //docker/network:docker-network-test:
  no such attribute 'test_class' in 'java_test' rule
ERROR: docker/sandbox/BUILD:15:12: //docker/sandbox:hello-java:
  no such attribute 'srcs' in 'java_binary' rule
ERROR: docker/sandbox/BUILD:15:12: //docker/sandbox:hello-java:
  no such attribute 'main_class' in 'java_binary' rule
ERROR: docker/sysbox/dind_test/BUILD:15:1:
  name 'sh_test' is not defined (did you mean 'cc_test'?)
ERROR: package contains errors: docker/sysbox/dind_test
ERROR: python/pytest/BUILD:3:1:
  name 'py_library' is not defined (did you mean 'cc_library'?)
ERROR: package contains errors: docker/sandbox
ERROR: package contains errors: java/com/engflow/notificationqueue/demoserver
ERROR: docker/network/BUILD:39:13: //docker/network:jdk:
  no such attribute 'java_home' in 'java_runtime' rule
ERROR: package contains errors: docker/network
ERROR: package contains errors: python/pytest
ERROR: java/com/engflow/example/BUILD:3:13: //java/com/engflow/example:example:
  no such attribute 'srcs' in 'java_library' rule
ERROR: java/com/engflow/example/BUILD:8:10: //java/com/engflow/example:ExampleTest:
  no such attribute 'srcs' in 'java_test' rule
ERROR: java/com/engflow/example/BUILD:8:10: //java/com/engflow/example:ExampleTest:
  no such attribute 'deps' in 'java_test' rule
ERROR: package contains errors: java/com/engflow/example
ERROR: java/com/engflow/notificationqueue/BUILD:5:12: //java/com/engflow/notificationqueue:client:
  no such attribute 'srcs' in 'java_binary' rule
ERROR: java/com/engflow/notificationqueue/BUILD:5:12: //java/com/engflow/notificationqueue:client:
  no such attribute 'main_class' in 'java_binary' rule
ERROR: java/com/engflow/notificationqueue/BUILD:5:12: //java/com/engflow/notificationqueue:client:
  no such attribute 'deps' in 'java_binary' rule
ERROR: package contains errors: java/com/engflow/notificationqueue
WARNING: Target pattern parsing failed.
ERROR: Skipping '//...': Error evaluating '//...':
  error loading package 'docker/network':
  Package 'docker/network' contains errors
ERROR: Error evaluating '//...':
  error loading package 'docker/network':
  Package 'docker/network' contains errors
INFO: Elapsed time: 2.821s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
```

Applied the fixes using Buildifier 8.2.0 with the following script:

```bash

WARNINGS=(
  $(buildifier --lint=warn -r . 2>&1 | grep -- native- |
    awk '{print $2}' | sort | uniq | sed -e 's/:$//')
)

IFS=, WARNINGS="${WARNINGS[*]}"
echo "warnings: ${WARNINGS}"
buildifier --lint=fix --warnings="$WARNINGS" -r .
```
@mbland
Copy link
Contributor Author

mbland commented May 17, 2025

@simuons I've also got another change teed up to land after this PR that contains some version bumps and small documentation fixes. Maybe after this pull request and that, it could be time for v7.0.1?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant