Make test-images multiarch

[nitkon]

The "make test-cri" command fails on non-x86 architecture because of the images created from image-test and image-user directory are not multi-arch. If these images are made multi-arch when built and pushed, then it would be easy to test on other non-x86 architectures as well

[Random-Liu]

@nitkon Can we do something like this? https://github.com/kubernetes/kubernetes/blob/master/test/utils/image/manifest.go#L103

[nitkon]

@Random-Liu : The following changes would make the test images used in "make test-cri" run on x86 as well as ppc64le arch.

1. Using busybox:1.28 instead of busybox:1.26 PR

2. Using a multi-arch pause image. Issue

3. Using ADD instruction in the Dockerfile as opposed to touch. Those ${TEST} files would be created and then removed by the makefile. Adding a qemu for cross-building just for touch command would be a heavy weight operation hence this suggestion. PR

4. Make the existing gcr.io/google-containers/nonewprivs image multi-arch(not by cross-building) or Push a gcr.io/google-containers/nonewprivs-ppc64le image there. The gcr.io/kubernetes-e2e-test-images/nonewprivs-ppc64le:1.0 fails to unpack on ppc64le after "crictl pull" as it is a cross-built image for ppc64le. The check here fails it at the time of unpacking the image. Thus we cannot use cross-built images but images built on that very platform, then pushed and used as standalone or use in fat manifest file.

[nitkon]

@feiskyer @Random-Liu : I think my analysis for test-image-* mentioned in point 3 above is incorrect as the touch binary depends on the Baseimage. We need to make all these test-image-* multi-arch else they are bound to fail on all non-x86 arch.

Let me know how we would like to proceed further?

These images are currently not multiarch.

./manifest-tool inspect gcr.io/cri-tools/test-image-1
gcr.io/cri-tools/test-image-1: manifest type: application/vnd.docker.distribution.manifest.v2+json
      Digest: sha256:3e5c3f1d52d644c8d426414247b63ccdcd85e8790152f714fd14a1845601c33c
Architecture: amd64
          OS: linux
    # Layers: 2
      layer 1: digest = sha256:65fc92611f38c5c3e31d5c0faec078c60f41103f2ad4a3700fff122c49aab358
      layer 2: digest = sha256:f83b4c12566814a25365d59f5f7841323b4bedc3b423db6282f096408a76dda0

[nitkon]

Point 2 mentioned above is tracked here as well. Issue

[feiskyer]

@Random-Liu Could you help to push the multi-arch images for both pause and busybox?

[nitkon]

@feiskyer : The pause image that is pushed is now multi-arch. See Issue. Also the busybox image now used in the test-cases is multi-arch. See PR.

We need to build the image-test-* images on x86(already done) and ppc64le(I can do it) and any other arch if needed and push these images into gcr.io/cri-tools and then use manifest-tool to create a fat-manifest.

[feiskyer]

@nitkon Yep, you are right. Could you update PR #292 to build multi-arch images? here is a Makefile reference

[nitkon]

@feiskyer: Yes I will. However, I have noticed that cross built images fail to unpack when pulled. Hence it will still fail on ppc64le arch. Here is the reference Issue

Solution:
Option 1: Fix the issue
Option 2: Build the test-images on x86 and on ppc64le(not cross-built, I can provide), push them to a repo, create a fat manifest out of it. So that when "crictl pull" these test-images they don't fail to unpack.

[feiskyer]

Option 1: Fix the issue

@nitkon Seems it is a containerd issue?

[nitkon]

@feiskyer : This PR makes the test images in image-user multi-arch.

Seems it is a containerd issue?

Seems like. Hence we will be using fat manifest/manifest list/multi-arch images for test images.

[nitkon]

@feiskyer @Random-Liu : When are we planning to push all these images?

[feiskyer]

I have no access to the gcr registry. @Random-Liu Could you help to push those images?

[nitkon]

@feiskyer: I realize that you do not have the access to the gcr repo. How do we get this done then before any other changes break my PRs? @Random-Liu could you help?