Skip to content

ci: Build gccjit from a git archive #122658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 24, 2024
Merged

ci: Build gccjit from a git archive #122658

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9d66b1b
ci: Build gccjit from a git archive
cuviper Mar 17, 2024
7936e18
Use the rust-lang/gcc repo directly
cuviper Mar 17, 2024
f512f9e
Use `pushd` and `popd`
cuviper Mar 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions src/ci/docker/host-x86_64/dist-x86_64-linux/build-gccjit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,22 @@
#!/usr/bin/env bash

GIT_REPO="https://github.com/rust-lang/gcc"

# This commit hash needs to be updated to use a more recent gcc fork version.
GIT_COMMIT="78dc50f0e50e6cd1433149520bd512a4e0eaa1bc"

set -ex

cd $1

source shared.sh

# Setting up folders for GCC
git clone https://github.com/antoyo/gcc gcc-src
cd gcc-src
# This commit hash needs to be updated to use a more recent gcc fork version.
git checkout 78dc50f0e50e6cd1433149520bd512a4e0eaa1bc
curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably also retry the download if it fails for some reason.

Suggested change
curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" |
retry curl -L "$GIT_REPO/archive/$GIT_COMMIT.tar.gz" |

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not that simple -- if a partial download fails and restarts, the latter | tar is not likely to fare well. We could download to disk first, or wrap the pipeline in a bash function to be retried. But I wonder if this is useful, because if a GitHub CI runner can't download from github.com, we likely have bigger issues...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue here is not so much the download but the transient DNS failures, they happened in the past, even for github.com (can't find a example right now).

Sure, we may want to download first to disk; I didn't look to closely at the retry function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't worry about it. The whole docker build is already retried 5 times IIRC. This is maybe nice cleanup in the future but isn't worth it for now.

tar -xz --transform "s/gcc-$GIT_COMMIT/gcc-src/"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If for some reason we do want git metadata on disk, AFAIK shallow clones are not possible with specific commit hashes, but you can get there with git init; git fetch --depth 1 $repo $commit; git checkout $commit.


mkdir ../gcc-build ../gcc-install
cd ../gcc-build
mkdir gcc-build gcc-install
pushd gcc-build

# Building GCC.
hide_output \
Expand All @@ -28,6 +31,7 @@ hide_output \
hide_output make -j$(nproc)
hide_output make install

rm -rf ../gcc-src
popd
rm -rf gcc-src gcc-build
ln -s /scripts/gcc-install/lib/libgccjit.so /usr/lib/x86_64-linux-gnu/libgccjit.so
ln -s /scripts/gcc-install/lib/libgccjit.so /usr/lib/x86_64-linux-gnu/libgccjit.so.0