Make docker-slim arm64 compatible

[kadern0]

Signed-off-by: Pablo Caderno kaderno@gmail.com

Fixes #155

ubuntu@kaderno-test:~/dist_linux_arm64$ file docker-slim
docker-slim: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=WbP5_VoxIUGfc7s61dPC/g9xW7j-Fqi0ieuE2-5C9/28WGfZLcdjMInBoUEk-X/qqJpwONs3u9tG8I95ag3, stripped



./docker-slim build kaderno/flask2
docker-slim[build]: info=http.probe message='using default probe'
docker-slim[build]: state=started
docker-slim[build]: info=params target=kaderno/flask2 continue.mode=probe rt.as.user=true keep.perms=true
docker-slim[build]: state=image.inspection.start
docker-slim[build]: info=image id=sha256:d74d6390c57a168015518b7531f1455ac366fb920a6a1e1fa412f2dfc2307929 size.bytes=356483459 size.human=356 MB
docker-slim[build]: info=image.stack index=0 name='arm64v8/ubuntu:latest' id='sha256:1cb4829d62431ae6571f5047f176b5c6a0b7edb3b9b3b76d516a8e11ca118f18'
docker-slim[build]: info=image.stack index=1 name='kaderno/flask2:latest' id='sha256:d74d6390c57a168015518b7531f1455ac366fb920a6a1e1fa412f2dfc2307929'
docker-slim[build]: info=image.exposed_ports list='5000'
docker-slim[build]: state=image.inspection.done
docker-slim[build]: state=container.inspection.start
docker-slim[build]: info=container status=created name=dockerslimk_36817_20200707053619 id=def43b1a0a635d64c69c4ec26ef14809993a202f5c29f7c45ba967c99ef87e31
docker-slim[build]: info=cmd.startmonitor status=sent
docker-slim[build]: info=event.startmonitor.done status=received
docker-slim[build]: info=container name=dockerslimk_36817_20200707053619 id=def43b1a0a635d64c69c4ec26ef14809993a202f5c29f7c45ba967c99ef87e31 target.port.list=[32783] target.port.info=[5000/tcp => 0.0.0.0:32783] message='YOU CAN USE THESE PORTS TO INTERACT WITH THE CONTAINER'
docker-slim[build]: state=http.probe.starting message='WAIT FOR HTTP PROBE TO FINISH'
docker-slim[build]: info=continue.after mode=probe message='no input required, execution will resume when HTTP probing is completed'
docker-slim[build]: info=prompt message='waiting for the HTTP probe to finish'
docker-slim[build]: state=http.probe.running
docker-slim[build]: info=http.probe.ports count=1 targets='32783'
docker-slim[build]: info=http.probe.commands count=1 commands='GET /'
docker-slim[build]: info=http.probe.call status=200 method=GET target=http://127.0.0.1:32783/ attempt=1  time=2020-07-07T05:36:32Z
docker-slim[build]: info=http.probe.summary total=1 failures=0 successful=1
docker-slim[build]: state=http.probe.done 
docker-slim[build]: info=probe.crawler page=0 url=http://127.0.0.1:32783/
docker-slim[build]: info=probe.crawler.done addr=http://127.0.0.1:32783/
docker-slim[build]: info=event message='HTTP probe is done'
docker-slim[build]: state=container.inspection.finishing
docker-slim[build]: state=container.inspection.artifact.processing
docker-slim[build]: state=container.inspection.done
docker-slim[build]: state=building message='building optimized image'
docker-slim[build]: state=completed
docker-slim[build]: info=results status='MINIFIED BY 17.69X [356483459 (356 MB) => 20149506 (20 MB)]'
docker-slim[build]: info=results  image.name=kaderno/flask2.slim image.size='20 MB' data=true
docker-slim[build]: info=results  artifacts.location='/home/ubuntu/dist_linux_arm64/.docker-slim-state/images/d74d6390c57a168015518b7531f1455ac366fb920a6a1e1fa412f2dfc2307929/artifacts'
docker-slim[build]: info=results  artifacts.report=creport.json
docker-slim[build]: info=results  artifacts.dockerfile.original=Dockerfile.fat
docker-slim[build]: info=results  artifacts.dockerfile.new=Dockerfile
docker-slim[build]: info=results  artifacts.seccomp=kaderno-flask2-seccomp.json
docker-slim[build]: info=results  artifacts.apparmor=kaderno-flask2-apparmor-profile
docker-slim[build]: state=done
docker-slim[build]: info=version status=OUTDATED local=1.29.0-82-g3c30311 current=1.29.0
docker-slim[build]: info=message message='Your version of DockerSlim is out of date! Use the "update" command or download the new version from https://dockersl.im/downloads.html'
docker-slim[build]: info=report file='slim.report.json'

ubuntu@kaderno-test:~/dist_linux_arm64$ docker images
REPOSITORY                  TAG                 IMAGE ID            CREATED              SIZE
kaderno/flask2.slim         latest              9d06f39bb9fd        About a minute ago   20.1MB
kaderno/flask2              latest              d74d6390c57a        4 minutes ago        356MB
kaderno/flask.slim          latest              5e11b0cdabd0        16 minutes ago       19.3MB
kaderno/flask               latest              4fbad4f22789        18 minutes ago       354MB

ubuntu@kaderno-test:~/dist_linux_arm64$ docker run -d -p5000:5000 kaderno/flask2.slim
3aefc9aed46895360bb55e7aeb2127f92114e743394ff2272169463690d653a9
ubuntu@kaderno-test:~/dist_linux_arm64$ curl localhost:5000
{
    "hello": "world"
}

[kadern0]

@kcq this is as far as I could get, I'm stuck with this error and I have no idea what it could be:

./docker-slim --log-level debug --verbose build arm64v8/ubuntu
docker-slim[build]: info=http.probe message='using default probe'
docker-slim[build]: state=started
docker-slim[build]: info=params target=arm64v8/ubuntu continue.mode=probe rt.as.user=true keep.perms=true
time="2020-07-02T05:42:40Z" level=info msg="image=arm64v8/ubuntu http-probe=true remove-file-artifacts=false image-overrides=map[] entrypoint=[] (false) cmd=[] (false) workdir='' env=[] expose=map[]" app=docker-slim command=build
docker-slim[build]: state=image.inspection.start
time="2020-07-02T05:42:41Z" level=info msg="inspecting 'fat' image metadata..." app=docker-slim command=build
docker-slim[build]: info=image id=sha256:454b2a641b9d159a48e67e1436003babda46ac9858a8b094081e32b39bb18abb size.bytes=66678538 size.human=67 MB
time="2020-07-02T05:42:42Z" level=info msg="processing 'fat' image info..." app=docker-slim command=build
docker-slim[build]: info=image.stack index=0 name='arm64v8/ubuntu:latest' id='sha256:454b2a641b9d159a48e67e1436003babda46ac9858a8b094081e32b39bb18abb'
docker-slim[build]: state=image.inspection.done
docker-slim[build]: state=container.inspection.start
time="2020-07-02T05:42:42Z" level=info msg="starting instrumented 'fat' container..." app=docker-slim command=build
docker-slim[build]: info=container status=created name=dockerslimk_1707_20200702054242 id=ec50999d9a713093b3a60330a150447bc7a752486a3908bee14754a11dff34f5
time="2020-07-02T05:43:31Z" level=error msg="channel.Client.Read: read error (read tcp 127.0.0.1:37698->127.0.0.1:32781: read: connection reset by peer), exiting..."
time="2020-07-02T05:43:31Z" level=error msg="channel.NewCommandClient: channel verify error = read tcp 127.0.0.1:37698->127.0.0.1:32781: read: connection reset by peer"
docker-slim[build]: info=cmd.startmonitor status=sent
time="2020-07-02T05:43:37Z" level=error msg="ipc.Client.GetEvent(): event channel error = EOF\n"
time="2020-07-02T05:43:37Z" level=fatal msg="docker-slim: failure" error=EOF stack="goroutine 1 [running]:\nruntime/debug.Stack(0x0, 0x400048f680, 0x40004202a0)\n\truntime/debug/stack.go:24 +0x88\ngithub.com/docker-slim/docker-slim/pkg/util/errutil.FailOn(0x9982a0, 0x4000064060)\n\tgithub.com/docker-slim/docker-slim@/pkg/util/errutil/errutil.go:14 +0x40\ngithub.com/docker-slim/docker-slim/internal/app/master/commands.OnBuild(0x40002ac410, 0xfffffc29ce48, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, ...)\n\tgithub.com/docker-slim/docker-slim@/internal/app/master/commands/build.go:270 +0x106c\ngithub.com/docker-slim/docker-slim/internal/app/master.init.0.func10(0x40000b3ce0, 0x7eb600, 0x0)\n\tgithub.com/docker-slim/docker-slim@/internal/app/master/cli.go:2051 +0x1234\ngithub.com/urfave/cli.HandleAction(0x71cea0, 0x88c508, 0x40000b3ce0, 0x40000b3ce0, 0x0)\n\tgithub.com/urfave/cli@v1.22.4/app.go:526 +0xdc\ngithub.com/urfave/cli.Command.Run(0x8500ba, 0x5, 0x0, 0x0, 0x40002a4400, 0x1, 0x1, 0x87df81, 0x44, 0x0, ...)\n\tgithub.com/urfave/cli@v1.22.4/command.go:174 +0x3e8\ngithub.com/urfave/cli.(*App).Run(0x40002601c0, 0x40000301e0, 0x6, 0x6, 0x0, 0x0)\n\tgithub.com/urfave/cli@v1.22.4/app.go:279 +0x534\ngithub.com/docker-slim/docker-slim/internal/app/master.runCli()\n\tgithub.com/docker-slim/docker-slim@/internal/app/master/cli.go:2504 +0x60\ngithub.com/docker-slim/docker-slim/internal/app/master.Run()\n\tgithub.com/docker-slim/docker-slim@/internal/app/master/app.go:6 +0x24\nmain.main()\n\tgithub.com/docker-slim/docker-slim@/cmd/docker-slim/main.go:8 +0x20\n" version="linux|Transformer|1.29.0-75-g4872d42|4872d42cce64571eb7f385e5ee2392f75b62d724|2020-07-02_05:21:37AM"

Any ideas?

[kcq]

We'll figure this out :) This error seems to indicate that the sensor isn't up and running, which is why the IPC fails to initialize. To know what's going on you'll need to look at the logs in the temporary container docker-slim creates. You should find it by doing docker ps -a and then doing docker logs THE_CONTAINER_ID.

[kadern0]

Found the error the way you suggested:

time="2020-07-02T08:18:25Z" level=info msg="sensor: args => []string{\"/opt/dockerslim/bin/docker-slim-sensor\"}"
time="2020-07-02T08:18:25Z" level=info msg="sensor: waiting for commands..."
time="2020-07-02T08:18:26Z" level=info msg="sensor: monitor starting..."
time="2020-07-02T08:18:26Z" level=info msg="fanmon: Run"
time="2020-07-02T08:18:26Z" level=info msg="ptmon: Run"
time="2020-07-02T08:18:26Z" level=fatal msg="ptmon: collector - PtraceGetRegs(call): input/output error"

It seems the error happens when this function is called syscall.PtraceGetRegs() -> https://github.com/docker-slim/docker-slim/blob/7c6f15c18e3757ac10f1ea97a4a5784a1ae2d19f/internal/app/sensor/monitors/ptrace/monitor.go#L148

But I can't see anything obvious going wrong with that.

[kcq]

syscall.PtraceGetRegs uses PTRACE_GETREGS and it looks like it is not defined on arm64. Need to create a replacement for the PtraceGetRegs call using PTRACE_GETREGSET, which should be available.

[kadern0]

I've been investigating a bit more so I've created a monitor_arm64.go file, in such file I've been trying to use this function 'unix.PtraceGetRegsArm64' as there is no such on the syscall package (I had to update the file system_linux_arm64.go to work with that). Unfortunately I'm still getting the same error. As I wasn't 100% whether the right code was being run, I've updated the log string (on monitor.go and monitor_arm64.go) so I could identify if my new code was being used but I'm getting the same original error:

level=fatal msg="ptmon: collector - PtraceGetRegs(call): input/output error"

and I don't have this string anywhere in my code (because I've modified both files). Any idea where does this string come from?

Thank you for your patience!

[kadern0]

@kcq after some research, I've opted to try using a different function to get the registers for arm64 ->

PtraceGetRegSetArm64(pid, addr int, regsout *PtraceRegsArm64) error {

and I'm calling it with:

if err := unix.PtraceGetRegSetArm64(targetPid, 0, &regs); err != nil {

And this is the error I get (argument types are correct):

time="2020-07-06T10:19:07Z" level=fatal msg="ptmon: collector - PtraceGetRegsArm64(call): invalid argument"

Also I've requested a VM to this guys https://www.linaro.cloud/ for testing this and they seem to be interested on this project (and also this port to work).

[kadern0]

@kcq I think it's working now!

[kcq]

@kadern0 awesome!

also didn't know about https://www.linaro.cloud ... how easy is it to get a vm? how much does it cost?

[kadern0]

I've requested a small VM and they gave me 4VCPUs 8G RAM and 80G of disc. I think it took me 3 days to have access (but I think it was due to the weekend and then the hour difference). It is free (for developers) to use their cloud. This is what they told me about this project:
"Linaro also has interests to make Docker-slim works on Arm64 and also offer more instances for CI resources on Arm64. So I think we can have more talk about upstream collaboration if you want."

t seems they are happy to collaborate :)

[kcq]

Cool! Are you happy with the PR? if so, i'll be happy to merge...

[kadern0]

I think it's good to go, seems to be working. Also, once this is merged, the WISHLIST file could be updated to reflect ARM64 support :)