[bug]: argo wf_dockerfile_build builds workflows retrying due to 143 exit

[uptownhr]

Prior Search

• I have already searched this project's issues to determine if a similar question has already been asked.

What is your question?

I am noticing an increase of retries due to 143 exit code for dockerfile builds using the wf_dockerfile_build workflow module.

What can be the cause of the 143 exit code and solutions to preventing / lessening the occurrence?

Not critical, but this is causing builds and deployments to take 3x the time longer than they should. Sometimes I see it occur 3 - 4 times in a row before succeeding.

Some data: around 5-10% of our builds experience this.

What primary components of the stack does this relate to?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[uptownhr]

Could incresaing the CPU request help? What are some ways to determine the cause?

[fullykubed]

There is no wf_tf_dockerfile module so unclear what is being referenced.

[uptownhr]

• Updated to wf_dockerfile_build
• changed to bug

I believe the minimum resources given to the dockerfile containers was too low. We have currently upped the resource request to cpu: 500 and memory: 2500. This may be excessive given buildkit performs most of the work but these were the values we had prior to the upgrade from edge.24-11-13.

[fullykubed]

If that amount of CPU and memory is being used, something has gone terribly wrong.

Almost certainly that means the user has not optimized their build context so they are inadvertently trying to compress and transfer gigabytes of information to the BuildKit builder. Maybe that is required, but it is definitely not standard and would require custom tuning.

We provide a guide here.