Images don't pushes, when used minikube and useCli Docker's option

[saruman9]

What happened?

Images is not pushed to registry when used minikube k8s context and useCli (args, useBuildKit too) options for Docker builder engine. When DevSpace deployed the container, Pod/Deployment/etc resource can't be created, because the image was not pushed to registry.

The option preferMinikube don't help with the issue.

What did you expect to happen instead?

Images should be pushed to registry when used a local Docker builder (with or without BuildKit) with local cluster.

How can we reproduce the bug?

1. Set useBuildKit option.
2. Use a local cluster (for example minikube) for deploying.

Local Environment:

• DevSpace Version: 5.10.0
• Operating System: linux
• Deployment method: helm | kubectl apply

Kubernetes Cluster:

• Kubernetes Version:

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"archive", BuildDate:"2021-03-25T19:02:48Z", GoVersion:"go1.16.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:20:00Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

/kind bug

[kinzleb]

I believe I see the same issue with my local minikube development -- I'm trying to enable BuildKit for improving build times while also enabling preferMinikube so that images do not need to be pushed to a registry because Kubernetes can simply use the images available in the Docker daemon belonging to the kubelet of the local cluster:

build:
  docker:
    preferMinikube: true
    useBuildKit: true

But these seem to be incompatible because when {useBuildKit: true} then it seems it forces {useCli: true} and uses my local docker to build/store the images rather than the docker daemon in minikube, and all the pods in minikube will fail with image pull errors since the images stored locally aren't available to minikube's docker daemon. I even tried forcing {useCli: false} along with the above build config but it did not seem to respect it since useBuildKit is true.

[FabianKramm]

@saruman9 thanks for opening this issue! Yes, internally we just call the docker cli with DOCKER_BUILDKIT=1 which targets always the local docker. I guess we can change that by calling minikube docker-env beforehand to change the target docker daemon to the minikube one.