Feature: docker build --profile report for step duration and layer size

[jadametz]

📃 Description:

I'd like to see docker build --profile exist to generate a report at the end of a build that contains each step, the duration that step took to execute, and the layer size added to the image during that stage.

☑️ Steps to produce the desired outcome:

docker build -t repo/imagename:tag --profile .

🎉 Describe the results you expect:

➭ docker build -t repo/imagename:tag --profile .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM busybox
 ---> f6e427c148a7
Step 2/3 : RUN sleep 10
 ---> Running in f4d1ad8b4b75
Removing intermediate container f4d1ad8b4b75
 ---> d3cd6a8b9b14
Step 3/3 : RUN sleep 5 &&  echo "Profiling in docker build would be d0pe!"
 ---> Running in 86cf9904a391
Profiling in docker build would be d0pe!
Removing intermediate container 86cf9904a391
 ---> ca2771a06dea
Successfully built ca2771a06dea
Successfully tagged repo/imagename:tag

STEP         DURATION          LAYER SIZE
1/3          1 second          1MB
2/3          10 seconds        100MB
3/3          5 seconds         9001MB

🙏 Guidance:

I'd love to work on this as my first contribution if the maintainers think it's a good idea. I'm thinking it'll at least require a PR here and in docker/cli. I'm confident I'll need guidance on where to make these changes (assuming they're possible) 😄.

Additionally, if this PR is better suited to be opened elsewhere, just let me know!

[thaJeztah]

There's an open PR (but still WIP) to report elapsed time per build-step #35549

However, plans are to integrate BuildKit, which supports open tracing; https://github.com/moby/buildkit#opentracing-support (see moby/buildkit#255), which would provide much more detailed information on build; example screenshot from that PR:

/cc @tonistiigi

[arm64b]

Interesting topic, recently I am seeking for a tool to evaluate the build profile result like @jadametz posted... 😄

[maurorappa]

@arm64b, my PR, as mentioned before, shows the time elapsed for every step:
#35549

[jadametz]

@thaJeztah thanks for responding and pointing out the other issue. @maurorappa good idea and thanks for getting it started!

Are either of you able to speak to if the work is still in progress, as it seems to have been stagnant for a few months.

Is there any chance of getting the timing data summarized at the bottom of the build? While I actually like the suggestion to include it inline, I fear abstracting it from after each step would be rather difficult, especially for large or noisy builds.

Additionally, I think layer size would be useful. Should I carry that suggestion to #35549 or should this issue turn into that feature request?

[thaJeztah]

Are either of you able to speak to if the work is still in progress, as it seems to have been stagnant for a few months.

I just left a comment on that PR, but if I recall correctly, tests were failing, because they need an update to match the new output format.

Is there any chance of getting the timing data summarized at the bottom of the build? While I actually like the suggestion to include it inline, I fear abstracting it from after each step would be rather difficult, especially for large or noisy builds.

While I see a use for this, I'm a bit cautious with adding the new output as a feature if integration with BuildKit would make it redundant.

Note that if should not be too difficult to get the timing from the output (given the ---> completed in: prefix). Size information should be possible to obtain from docker image history for the built image

Let me /cc @tonistiigi on that as well

[keeler]

In case anyone finds this while searching for a way to profile docker builds, the solution given in moby/buildkit#255 requires buildkitd which only works on Linux as of right now.

So, I wrapped buildkit and Jaeger into a docker-in-docker image which should "just work" even outside of Linux.
https://hub.docker.com/r/keelerrussell/docker-build-profiler

It works great on my Linux machine but I am unable to confirm if it works on another OS. If you give it a try let me know!