Docker images with arm64 support #86

mcbex · 2022-05-02T21:36:18Z

I am still working on the image for the dev container but the testing process is slow so I will make a separate PR once that is ready.

mcbex · 2022-05-02T21:43:59Z

The build error may be related to this issue: golang/go#51253

mcbex · 2022-05-02T21:54:25Z

Also need to look more closely at these issues which were brought to my attention by Aron:

actions/checkout#760
actions/checkout#766

jonyoder · 2022-05-03T12:05:58Z

docker/bionic/Dockerfile

@@ -51,8 +51,11 @@ RUN chmod +x /usr/local/bin/docker-compose

 # Install Go. Keep in sync with other Dockerfiles.
 ENV GOLANG_VERSION 1.18
-ENV GOLANG_DOWNLOAD_SHA256 e85278e98f57cdb150fe8409e6e5df5343ecb13cebf03a5d5ff12bd55a80264f
-RUN curl -fsSL "https://dl.google.com/go/go${GOLANG_VERSION}.linux-amd64.tar.gz" -o golang.tar.gz \
+RUN ARCH=$(if [ $(uname -m) = "aarch64" ]; then echo "arm64"; else echo "amd64"; fi) \


Here, and above, I wonder if we should consider simply using an ARG ARCH=amd64 and optionally passing the architecture to the build. For example, see the Launcher dockerfile: https://github.com/rstudio/launcher/blob/main/jenkins/docker/Dockerfile.bionic#L1.

Using an ARG would keep the Dockerfile a bit cleaner and let us handle the shell logic elsewhere.

I think the issue would be that when we install awscli and Go the URLs contain the architecture but it's not consistent: aarch64 and arm64 vs x86_64 and amd64. So passing in a single variable would not remove the need to resolve those URLs dynamically.... unless you're suggesting the part where we build the URLs should move out of the Dockerfile?

A default architecture hard-coded in the image would also mean that anyone not on an amd64 machine has to remember to pass the parameter every time. That really only affects me at the moment but it feels kind of clunky to require from users.

Maybe another solution is to have two Dockerfiles and separate build targets for each architecture?

I was thinking more along the lines of delegating the responsibility of determining architecture to the caller. In other words, instead of determining architecture in the docker file, you'd determine it in the justfile or in a script called by the justfile, and then you'd pass the ARCH arg to the docker build command.

I comment this above before I saw this comment. Docker BuildKit will set TARGETARCH to the architecture you're building for, so you won't have to pass any extra parameters. The info is pretty far down the page:

https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

BUILDPLATFORM — matches the current machine. (e.g. linux/amd64) BUILDOS — os component of BUILDPLATFORM, e.g. linux BUILDARCH — e.g. amd64, arm64, riscv64 BUILDVARIANT — used to set ARM variant, e.g. v7 TARGETPLATFORM — The value set with --platform flag on build TARGETOS - OS component from --platform, e.g. linux TARGETARCH - Architecture from --platform, e.g. arm64 TARGETVARIANT

Whoever is building this Dockerfile would then use the BuildKit --platform argument which would take care of all of the architecture-related stuff, e.g. setting the environment variable and choosing the right multi-platform base image.

You would want to remove the --platform option in your Dockerfile and instead pass that in the command line, e.g. docker build --platform linux/amd64. When a platform argument isn't specified Docker will use the host architecture.

This is a really nice approach because consumers can use the same Docker image without worrying about architecture, and you only have to worry about one Dockerfile.

Got it. Thanks for the feedback @jonyoder @shepherdjerred

I briefly read some of the documentation on BuildKit features but I wasn't sure if using BuildKit was considered portable or not (since it looked pretty new). So good to know others are also looking at this and thinking about it!

BuildKit is now about 3 years old: https://docs.docker.com/engine/release-notes/18.09/

As far as I know, it's the best way to make cross-platform images. You are right that it's not completely mature, this issue shows the gaps: moby/moby#40379

I believe that BuildKit is the default builder now, so that at least shows that Docker considers it to be the way forward: docker/cli#3314

mcbex · 2022-05-03T20:03:13Z

I'm not too familiar with VS code or developing in containers but I had to update how the dev container was installing Go modules to be the same as the other Dockerfile to get the image to build. I installed VS code and was able to connect to the container afterwards.

Happy to discuss the overall approach more as well.

bdeitte · 2022-05-03T21:18:58Z

I'm not sure if this will help here, but I saw it come up in a ticket from @shepherdjerred for RSPM. https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

.devcontainer/Dockerfile

shepherdjerred · 2022-05-03T22:13:25Z

I'm not sure if this will help here, but I saw it come up in a ticket from @shepherdjerred for RSPM. https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

Thanks for the mention Brian! The solution here looks similar to what I'm doing for RSPM.

mcbex · 2022-05-11T18:21:16Z

I was able to remove the logic for downloading the Go tarball by using the TARGETARCH env var which simplified the Dockerfile quite a bit but I couldn't think of a better way to set the correct checksum. Other suggestions welcome.

jonyoder

One remaining question! Looks great!

.devcontainer/Dockerfile

shepherdjerred · 2022-05-11T19:00:23Z

.devcontainer/Dockerfile

+      then echo "7ac7b396a691e588c5fb57687759e6c4db84a2a3bbebb0765f4b38e5b1c5b00e"; \
+      else echo "e85278e98f57cdb150fe8409e6e5df5343ecb13cebf03a5d5ff12bd55a80264f"; fi) \


From what I've seen this is (unfortunately) the idiomatic way to do platform-related logic in Dockerfiles. One note though; explicitly check for amd64, don't assume that arm64 and amd64 are the only possible architectures.

shepherdjerred · 2022-05-11T19:01:48Z

.devcontainer/Dockerfile

 RUN PATH="$PATH:/usr/local/go/bin" go install github.com/go-delve/delve/cmd/dlv@latest
-RUN GO111MODULE=on PATH="$PATH:/usr/local/go/bin" GOPATH=/usr/local/gotools go get -v \
+RUN GO111MODULE=on PATH="$PATH:/usr/local/go/bin" GOPATH=/usr/local/gotools go install \


Should GO111MODULE still be here? I thought it was removed in 1.18.

jonyoder

Marking this as approved so I don't hold things up. If you're able to look into the GO111MODULE question and refactor the unames, this looks great to me!

jonyoder · 2022-07-20T15:58:44Z

@mcbex should we merge this?

mcbex · 2022-07-20T16:02:29Z

Ugh my apologies I never saw these comments. Going to go over it again and make those changes!

mcbex · 2022-07-21T15:03:03Z

.devcontainer/Dockerfile

+
+RUN if [ "${TARGETARCH}" != "amd64" ] && [ "${TARGETARCH}" != "arm64" ]; then \
+    echo "${TARGETARCH} is not a supported architecture. Please use linux/amd64 or linux/arm64"; \
+    exit 1; fi


I don't love this but realized while testing that waiting until the later build stages to discover a particular platform is unsupported is annoying. I couldn't find a built in mechanism to allowlist architectures but happy to hear better ideas.

Hm I've never thought about restricting architectures like this, but it does make sense.

mcbex · 2022-07-22T18:10:58Z

@jonyoder Let me know if you see any other changes I should make before we merge

mcbex added 5 commits May 2, 2022 14:36

support building arm64 images

a42eac1

pass in docker build args

c56a12d

add just target for cleaning up docker images

90cf858

add documentation

b924c5d

fix typo

a6b4203

jonyoder reviewed May 3, 2022

View reviewed changes

mcbex added 2 commits May 3, 2022 10:21

Merge branch 'main' into docker-platform-images

344dd4d

update dev container and support ARM

445c7ab

shepherdjerred reviewed May 3, 2022

View reviewed changes

.devcontainer/Dockerfile Show resolved Hide resolved

use TARGETARCH env var

b4982d7

mcbex requested a review from jonyoder May 11, 2022 18:21

jonyoder reviewed May 11, 2022

View reviewed changes

.devcontainer/Dockerfile Show resolved Hide resolved

shepherdjerred approved these changes May 11, 2022

View reviewed changes

jonyoder approved these changes May 11, 2022

View reviewed changes

address PR feedback

ac6f856

mcbex commented Jul 21, 2022

View reviewed changes

remove debug comment

a397ce7

jonyoder approved these changes Jul 22, 2022

View reviewed changes

mcbex merged commit 0635ea2 into main Jul 22, 2022

jonyoder deleted the docker-platform-images branch November 16, 2022 03:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker images with arm64 support #86

Docker images with arm64 support #86

mcbex commented May 2, 2022

mcbex commented May 2, 2022

mcbex commented May 2, 2022

jonyoder May 3, 2022

mcbex May 3, 2022

jonyoder May 3, 2022

shepherdjerred May 3, 2022 •

edited

Loading

mcbex May 4, 2022

shepherdjerred May 4, 2022

mcbex May 4, 2022

mcbex commented May 3, 2022

bdeitte commented May 3, 2022

shepherdjerred commented May 3, 2022

mcbex commented May 11, 2022

jonyoder left a comment

shepherdjerred May 11, 2022

shepherdjerred May 11, 2022

jonyoder left a comment

jonyoder commented Jul 20, 2022

mcbex commented Jul 20, 2022

mcbex Jul 21, 2022

shepherdjerred Jul 21, 2022

mcbex commented Jul 22, 2022

		then echo "7ac7b396a691e588c5fb57687759e6c4db84a2a3bbebb0765f4b38e5b1c5b00e"; \
		else echo "e85278e98f57cdb150fe8409e6e5df5343ecb13cebf03a5d5ff12bd55a80264f"; fi) \

Docker images with arm64 support #86

Docker images with arm64 support #86

Conversation

mcbex commented May 2, 2022

mcbex commented May 2, 2022

mcbex commented May 2, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shepherdjerred May 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcbex commented May 3, 2022

bdeitte commented May 3, 2022

shepherdjerred commented May 3, 2022

mcbex commented May 11, 2022

jonyoder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonyoder left a comment

Choose a reason for hiding this comment

jonyoder commented Jul 20, 2022

mcbex commented Jul 20, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcbex commented Jul 22, 2022

shepherdjerred May 3, 2022 •

edited

Loading