Cadvisor for ARM

[mboussaa]

I guess the current version of Cadvisor doest not support RPI docker images. I am not sure about that since I did not test it yet.
Is it possible to provide support for ARM images?

[vishh]

I wonder if cross compilation for arm just works.. cc @luxas

[luxas]

I would just add GOARCH=arm to go build command in build/build.sh.
It should just work.

However, if we want to cross-build a docker image, we probably have to base it on debian for other arches than amd64. See the hyperkube image how to do this: https://github.com/kubernetes/kubernetes/blob/master/cluster/images/hyperkube/Makefile

If we decide on a naming convention for non-amd64 cAdvisor and think we should do it, I may send a PR that builds cAdvisor images for arm, arm64, ppc64le.

[vishh]

It might help to automate the docker image release process in cadvisor to
ensure that arm images are built for every release.

On Thu, Apr 21, 2016 at 10:48 AM, Lucas Käldström notifications@github.com
wrote:

I would just add GOARCH=arm to go build command in build/build.sh.
It should just work.

However, if we want to cross-build a docker image, we probably have to
base it on debian for other arches than amd64. See the hyperkube image
how to do this:
https://github.com/kubernetes/kubernetes/blob/master/cluster/images/hyperkube/Makefile

If we decide on a naming convention for non-amd64 cAdvisor and think we
should do it, I may send a PR that builds cAdvisor images for arm, arm64,
ppc64le.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1236 (comment)

[luxas]

I may do it if you want it

[vishh]

@luxas: That will be awesome!!

On Thu, Apr 21, 2016 at 10:55 AM, Lucas Käldström notifications@github.com
wrote:

I may do it if you want it

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1236 (comment)

[mboussaa]

So for instance cAdvisor image does not handle ARM image?

[luxas]

@mboussaa I'm not really sure what you mean.
On an ARM host, cAdvisor built for ARM may monitor ARM containers without problems.

The current cadvisor image on docker hub is built for linux/amd64 (like the most other things), so it won't run on an ARM host at all

[solidnerd]

@luxas
I tried to cross compile with Docker 4 Mac and a golang:1.6.3 container.

1. docker run -it --rm golang:1.6.3
2. apt-get update && apt-get install vim
3. go get -d github.com/google/cadvisor
4. cd $GOPATH/src/github.com/google/cadvisor
5. vim build/build.sh and add env GOARCH=arm GOBIN=$PWD go "$GO_CMD" ${GO_FLAGS} -ldflags "${ldflags}" "${repo_path}"
6. make build

I get the following error:

vendor/github.com/opencontainers/runc/libcontainer/cgroups/systemd/apply_systemd.go:17:2: no buildable Go source files in /go/src/github.com/google/cadvisor/vendor/github.com/coreos/go-systemd/util
Makefile:35: recipe for target 'build' failed

Do you know what I'm doing wrong ?

[fgardt]

I compiled cadvisor and managed to edit the dockerfile to work on my raspberry pi.
I uploaded the result here.
I decided to compile it my self because any other versions weren't up to date :D

[rjlee]

@Brain-Gamer could you provide the dockerfile/instructions so others can do the same ?

[fgardt]

@rjlee sure! It takes maybe a few days but I will do it ;)

[fgardt]

@rjlee I forked this repo and added the information you asked for in the readme. If you need anything else just ask ;)

[rjlee]

@Brain-Gamer many thanks for sharing.

[vaibhavsood]

Hi, want to check if the cadvisor docker image can be considered for being extended to be multi-arch on docker hub? (as per above comments right now its for intel/amd64 arch)

The process for doing so would be along the lines as per here: https://github.com/docker-library/official-images#multiple-architectures

My interest here is to ultimately see a cadvisor docker image under https://hub.docker.com/u/ppc64le/

However since cadvisor is not a official Docker image, the first step i guess would be to make it official https://docs.docker.com/docker-hub/official_repos/#how-do-i-create-a-new-official-repository and then move to making the dockerfiles multi-arch (this process is as per Docker)

Want to check if this makes sense/any comments?

[whatever4711]

Hi,

currently we tried to port cAdvisor to a multiarchitecture Docker Image. The source repository is unibaktr/docker-cadvisor.

Therefore, we restructured the Dockerfile to use a slim Debian based image, since all binaries, cAdvisor uses, are rarely available for Alpine based multiarchitecture images.

Feel free to try it out
docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --volume=/dev/disk/:/dev/disk:ro --publish=8080:8080 --detach=true --name=cadvisor unibaktr/cadvisor

Perhaps we can find a way to integrate our work it into the official cAdvisor Repo.

Best regards

[ZCube]

The images released above are outdated. I created a cadvisor v0.34.0 docker images for my Raspberry Pi. I have used docker buildx which is still experimental. It supports multi architecture(kubernetes/kubernetes#73365) (arm, arm64, 386, x86_64). I would like to have a look if someone is interested.

https://github.com/ZCube/cadvisor-docker
https://hub.docker.com/r/zcube/cadvisor

[rhuss]

FYI, I managed to build and push an ARM image with this multistage Docker build & buildkit:

FROM golang:1.13-buster AS builder
ARG VERSION

RUN apt-get update \
 && apt-get install make git bash gcc \
 && mkdir -p $GOPATH/src/github.com/google \
 && git clone https://github.com/google/cadvisor.git $GOPATH/src/github.com/google/cadvisor

WORKDIR $GOPATH/src/github.com/google/cadvisor
RUN git fetch --tags \
 && git checkout $VERSION \
 && make build \
 && cp ./cadvisor /

# ------------------------------------------
# Copied over from deploy/Dockerfile except that the "zfs" dependency has been removed 
# a its not available fro Alpine on ARM
FROM alpine:3.10
MAINTAINER dengnan@google.com vmarmol@google.com vishnuk@google.com jimmidyson@gmail.com stclair@google.com

RUN apk --no-cache add libc6-compat device-mapper findutils && \
    apk --no-cache add thin-provisioning-tools --repository http://dl-3.alpinelinux.org/alpine/edge/main/ && \
    echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
    rm -rf /var/cache/apk/*

# Grab cadvisor from the staging directory.
COPY --from=builder /cadvisor /usr/bin/cadvisor

EXPOSE 8080

HEALTHCHECK --interval=30s --timeout=3s \
  CMD wget --quiet --tries=1 --spider http://localhost:8080/healthz || exit 1

ENTRYPOINT ["/usr/bin/cadvisor", "-logtostderr"]

and

# Prepare
docker buildx create --name xbuilder
docker buildx use xbuilder

# Cross build
VERSION=v0.35.0
docker buildx build --build-arg VERSION=${VERSION} --platform linux/arm/v7,linux/amd64 --push -t rhuss/cadvisor:${VERSION} .

[rhuss]

Ah, just saw its more or less the same as in https://github.com/ZCube/cadvisor-docker

(not sure why apk add zfs didn't work for me, though)

[rhuss]

(not sure why apk add zfs didn't work for me, though)

well, it doesn't work over there for arm, too :)

[whatever4711]

The images released above are outdated. I created a cadvisor v0.34.0 docker images for my Raspberry Pi. I have used docker buildx which is still experimental. It supports multi architecture(kubernetes/kubernetes#73365) (arm, arm64, 386, x86_64). I would like to have a look if someone is interested.

https://github.com/ZCube/cadvisor-docker
https://hub.docker.com/r/zcube/cadvisor

I updated our images to use buildx with CircleCI, so feel free to use unibaktr/cadvisor again ;)

[muzammil360]

I think it would be great if we can have (non-docker) binaries for arm64 in addition to amd in assets. Right now, I am cloning the repo and building it using make build. But it doesn't seem like a clean approach. Definitely would prefer something that I can just download using wget and unzip. Prometheus and all Prometheus related monitoring applications do that.

[alex-voigt]

still no ARM support? :-/

[sabretus]

ARM is not only RPI but also for example m6g on Amazon. Why it's so hard to build for ARM too?

[thaJeztah]

Perhaps the release script could be updated to build a multi-arch image using buildx;

cadvisor/build/release.sh

Lines 43 to 46 in 6fd4cd9

	# Build the docker image
	echo ">> building cadvisor docker image"
	gcr_tag="gcr.io/cadvisor/cadvisor:$VERSION"
	docker build -t $gcr_tag -f deploy/Dockerfile .

[iwankgb]

We could probably use goreleaser to build downloadable binaries and container images for any platform that we want. I don't think that any official ARM images make sense before #2744 is merged: it seems that majority of reported issues is related to RPi.
@sabretus I will make sure to test these changes against Graviton 2 and any other server ARM platform I will be able to get hold of :)

[muzammil360]

As a workaround i have been building cadvisor from src for arm64. But of course this is problematic since it needs to be run again every time cicd job runs on gitlab.

…

On Mon, Jan 4, 2021, 11:19 PM iwankgb ***@***.***> wrote: We could probably use goreleaser <https://github.com/goreleaser/goreleaser> to build downloadable binaries and container images for any platform that we want. I don't think that any official ARM images make sense before #2744 <#2744> is merged: it seems that majority of reported issues is related to RPi. @sabretus <https://github.com/sabretus> I will make sure to test these changes against Graviton 2 and any other server ARM platform I will be able to get hold of :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1236 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3S5IJHVBP2YG6S3EGMLS3SYIBB5ANCNFSM4CBSE6VA> .

[bahamut657]

@muzammil360 can you share the Dockerfile for arm64? How much time and resources are needed to build?

[muzammil360]

@bahamut657 Building arm64 is trivial. I can share the basic code

git clone https://github.com/google/cadvisor.git 
cd cadvisor
bash ./build/assets.sh 
bash ./build/build.sh

Regarding the dockerfile, I don't have it right now. But putting above code and making one should be again trivial. Just use some golang image as base.

Shoot me for more questions.

[muzammil360]

@bahamut657 just as I was looking, I found a dockerfile online. May be this will work faster for you.

https://github.com/Budry/cadvisor-arm

[muzammil360]

@bobbypage @haircommander I am open to discussing adding arm64 release for cadvisor. But would need some guidance for that.

[iwankgb]

It's something that I want to address in #2835.

[logopk]

I used the buildx approach of @rhuss.

That works fine until v0.39.0. Starting from there I get exec errors. Like this for v0.40.0

cadvisor        | panic: unaligned 64-bit atomic operation
cadvisor        |
cadvisor        | goroutine 72 [running]:
cadvisor        | runtime/internal/atomic.panicUnaligned()
cadvisor        |       /usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x24
cadvisor        | runtime/internal/atomic.Load64(0x31f336c, 0x72355c80, 0xc0331fbf)
cadvisor        |       /usr/local/go/src/runtime/internal/atomic/asm_arm.s:263 +0x14
cadvisor        | github.com/google/cadvisor/manager.(*containerData).updateStats(0x31f3200, 0x71d9d10f, 0xc0331fbf)
cadvisor        |       /src/cadvisor/manager/container.go:675 +0x84
cadvisor        | github.com/google/cadvisor/manager.(*containerData).housekeepingTick(0x31f3200, 0x332a040, 0x5f5e100, 0x0, 0x34fd800)
cadvisor        |       /src/cadvisor/manager/container.go:586 +0xe0
cadvisor        | github.com/google/cadvisor/manager.(*containerData).housekeeping(0x31f3200)
cadvisor        |       /src/cadvisor/manager/container.go:534 +0x200
cadvisor        | created by github.com/google/cadvisor/manager.(*containerData).Start
cadvisor        |       /src/cadvisor/manager/container.go:122 +0x2c

[logopk]

cadvisor | panic: unaligned 64-bit atomic operation

https://twitter.com/fatih/status/1371391996003295234?lang=de

golang/go#36606

introduced by #1837 Expose OOM event count to prometheus
fa07332

stats.OOMEvents = atomic.LoadUint64(&cd.oomEvents)

[deathdefyer]

+1 looking to build this on apple silicon

[logopk]

I created my own Dockerfile to build.
after the clone of the repo I remove the failing stats.OOMEvents-Statement and the import like this:

RUN sed -i 's#"sync/atomic"##g'  /src/cadvisor/manager/container.go
RUN sed -i "s/ atomic.LoadUint64(&cd.oomEvents)/0/g"  /src/cadvisor/manager/container.go

After this it's running fine in v0.40 - v0.43

[martadinata666]

it already fixed in git master, just no release version yet. #3029 , me too still waiting release v0.4x with this fix

[logopk]

I don't understand the fix 🤯. could anyone explain why it matters that the uint64 comes first in the struct? or am I missing sth. obvious???

[daveshow]

Maybe an old issue but this might help if you want to build the image yourself. Thanks, @rhuss for the initial dockerfile. Made some modifications to the build process but if you have your own registry this should work fine.

https://gist.github.com/daveshow/e990656bba8451d85673640cac605e2c

[thaJeztah]

I don't understand the fix 🤯. could anyone explain why it matters that the uint64 comes first in the struct? or am I missing sth. obvious???

Yes, these can be confusing; search for "type alignment"; e.g., see https://go.dev/ref/spec#Size_and_alignment_guarantees and https://go101.org/article/memory-layout.html#alignment-guarantee

[Logicer16]

Pretty sure this was fixed in #3141. Is there a reason this is still open?

[bobbypage]

Yes, it is done, I think we can close this.