Skip to content

Commit

Permalink
Add Dockerfile to build context if passed from outside the build cont…
Browse files Browse the repository at this point in the history
…ext (#542)

Breaking Change: 
If a user provides a Dockerfile, the path specified must be absolute, or relative to the _Pulumi_ context, not the Docker build context.
The `dockerfile` field remains optional, and if left blank, we assume traditional location of Dockerfile within the Docker build context.

Specifics:
- If a user does not specify a Dockerfile the provider defaults to "Dockerfile", relative to Docker build context. The tarball already contains all necessary build files.
- If a user declares a Dockerfile, it must be an absolute path or a path relative to the Pulumi.yaml's location in the file system. The provider does its best to detect if a Dockerfile path was declared in error, and sends a helpful error message to the user during `Check()`, i.e. during a `pulumi preview`. 
- When a Dockerfile is passed from outside the Docker build context, we create a separate Dockerfile context and add it to
the build tarball. Note that any COPY statements are now relative to the Docker context, and not the Pulumi program's filesystem location.


* Add Dockerfile to build context using the clibuild method

Testing this against context: app and dockerfile: Dockerfile, we see
 error hashing dockerfile "Dockerfile": could not open file Dockerfile: open app/Dockerfile: no such file or directory
Unsure if I installed the correct provider version. Retrying.

* First attempt at getting a relative Dockerfile so we can hash it properly

* Pass relDockerfile to hashContext

Currently fails wilt dockerfile frontend error.
relDockerfile needs to be passed to the build context tarball.

* This now errors with

Diagnostics:
  docker:index:Image (my-image):
    error: failed to compute cache key: "/app" not found: not found

But that just means the Dockerfile is passing the wrong folder - this is
an expected error.

* Remove debugs and cleanup

* fix linter; disable dotnet registry tests

* A blank Dockerfile gets tests to pass but reverses the logic for outside-context Dockerfiles again. We need to create different logic for hashing and not use the cli build library.

* Use separator as way to determine whether this dockerfile is relative to the build context or not

* Use filepath separator detection to determine if a Dockerfile context is
needed.

* unskip dotnet tests

* Refactor relative path getter function for testing purposes

* Add unit test for getting the relative path to demonstrate and cement assumptions

* lint: handle error

* Remove default values for Dockerfile and Context. Refine logic for
detecting external Dockerfile.

Return dockerfile and context as outputs in schema.
Return additional outputs from Create: build context and its relative Dockerfile
Pass the replaced Dockerfile to the image build options when necessary

Co-authored-by: Aaron Friel <friel@pulumi.com>

* Verify existence of Dockerfile in Check function

This allows for us to fail early, during preview, and send an error
message to a user who is providing an erroneous Dockerfile.

Co-authored-by: Aaron Friel <friel@pulumi.com>

* Add integration tests for new dockerfile logic

Co-authored-by: Aaron Friel <friel@pulumi.com>

* Re-generate schema and SDKs

* Add a test for a build left blank

* Pacify linter

---------

Co-authored-by: Aaron Friel <friel@pulumi.com>
  • Loading branch information
guineveresaenger and Aaron Friel committed Mar 17, 2023
1 parent df363db commit 993e8a3
Show file tree
Hide file tree
Showing 32 changed files with 505 additions and 137 deletions.
2 changes: 1 addition & 1 deletion examples/docker-container-registry/ts/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const registryInfo = {
// Build and publish the container image.
const image = new docker.Image("my-image", {
build: {
context: "app"
context: "app",
},
imageName: imageName,
registry: registryInfo,
Expand Down
77 changes: 77 additions & 0 deletions examples/dockerfile_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
// Copyright 2016-2018, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build go || all
// +build go all

package examples

import (
"github.com/pulumi/pulumi/pkg/v3/testing/integration"
"github.com/stretchr/testify/assert"
"os"
"path"
"testing"
)

func TestDockerfileDefaultYAML(t *testing.T) {
cwd, err := os.Getwd()
if !assert.NoError(t, err) {
t.FailNow()
}

integration.ProgramTest(t, &integration.ProgramTestOptions{
Dir: path.Join(cwd, "test-dockerfile", "dockerfile-default"),
Quick: true,
SkipRefresh: true,
})
}

func TestDockerfileDefaultContextYAML(t *testing.T) {
cwd, err := os.Getwd()
if !assert.NoError(t, err) {
t.FailNow()
}

integration.ProgramTest(t, &integration.ProgramTestOptions{
Dir: path.Join(cwd, "test-dockerfile", "dockerfile-default-context"),
Quick: true,
SkipRefresh: true,
})
}

func TestDockerfileExternalYAML(t *testing.T) {
cwd, err := os.Getwd()
if !assert.NoError(t, err) {
t.FailNow()
}

integration.ProgramTest(t, &integration.ProgramTestOptions{
Dir: path.Join(cwd, "test-dockerfile", "dockerfile-external"),
Quick: true,
SkipRefresh: true,
})
}

func TestDockerfileInContextYAML(t *testing.T) {
cwd, err := os.Getwd()
if !assert.NoError(t, err) {
t.FailNow()
}

integration.ProgramTest(t, &integration.ProgramTestOptions{
Dir: path.Join(cwd, "test-dockerfile", "dockerfile-in-context"),
Quick: true,
SkipRefresh: true,
})
}
2 changes: 1 addition & 1 deletion examples/examples_nodejs_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ func TestGcpContainerRegistry(t *testing.T) {
integration.ProgramTest(t, &test)
}

func TestDockerContainerRegistry(t *testing.T) {
func TestDockerContainerRegistryNode(t *testing.T) {
username := "pulumibot"
password := os.Getenv("DOCKER_HUB_PASSWORD")
test := getJsOptions(t).
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM scratch

COPY . /
13 changes: 13 additions & 0 deletions examples/test-dockerfile/dockerfile-default-context/Pulumi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: dockerfile-default-context
runtime: yaml
resources:
demo-image:
type: docker:Image
properties:
imageName: pulumibot/test-image:tag
skipPush: true
options:
version: v4.0.0
outputs:
imageName: ${demo-image.imageName}
out-dockerfile: ${demo-image.dockerfile}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hi! I am a test file!
15 changes: 15 additions & 0 deletions examples/test-dockerfile/dockerfile-default/Pulumi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: dockerfile-default
runtime: yaml
resources:
demo-image:
type: docker:Image
properties:
imageName: pulumibot/test-image:tag2
skipPush: true
build:
context: ./app
options:
version: v4.0.0
outputs:
imageName: ${demo-image.imageName}
out-dockerfile: ${demo-image.dockerfile}
3 changes: 3 additions & 0 deletions examples/test-dockerfile/dockerfile-default/app/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM scratch

COPY . /
1 change: 1 addition & 0 deletions examples/test-dockerfile/dockerfile-default/app/app.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hi! I am a test file!
3 changes: 3 additions & 0 deletions examples/test-dockerfile/dockerfile-external/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM scratch

COPY .. /
16 changes: 16 additions & 0 deletions examples/test-dockerfile/dockerfile-external/Pulumi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: dockerfile-external
runtime: yaml
resources:
demo-image:
type: docker:Image
properties:
imageName: pulumibot/test-image:tag3
skipPush: true
build:
context: ./app
dockerfile: ./Dockerfile
options:
version: v4.0.0
outputs:
imageName: ${demo-image.imageName}
out-dockerfile: ${demo-image.dockerfile}
1 change: 1 addition & 0 deletions examples/test-dockerfile/dockerfile-external/app/app.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hi! I am a test file!
15 changes: 15 additions & 0 deletions examples/test-dockerfile/dockerfile-in-context/Pulumi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: dockerfile-in-context
runtime: yaml
resources:
demo-image:
type: docker:Image
properties:
imageName: pulumibot/test-image:tag1
skipPush: true
build:
context: ./app
dockerfile: ./app/Dockerfile
options:
version: v4.0.0
outputs:
imageName: ${demo-image.imageName}
3 changes: 3 additions & 0 deletions examples/test-dockerfile/dockerfile-in-context/app/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM scratch

COPY . /
1 change: 1 addition & 0 deletions examples/test-dockerfile/dockerfile-in-context/app/app.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hi! I am a test file!
14 changes: 10 additions & 4 deletions provider/cmd/pulumi-resource-docker/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -3065,13 +3065,11 @@
},
"context": {
"type": "string",
"description": "The path to the build context to use.",
"default": "."
"description": "The path to the build context to use."
},
"dockerfile": {
"type": "string",
"description": "The path to the Dockerfile to use.",
"default": "Dockerfile"
"description": "The path to the Dockerfile to use."
},
"platform": {
"type": "string",
Expand Down Expand Up @@ -4422,6 +4420,14 @@
"type": "string",
"description": "The fully qualified image name that was pushed to the registry."
},
"context": {
"type": "string",
"description": "The path to the build context to use."
},
"dockerfile": {
"type": "string",
"description": "The location of the Dockerfile relative to the docker build context."
},
"imageName": {
"type": "string",
"description": "The fully qualified image name"
Expand Down
55 changes: 28 additions & 27 deletions provider/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ require (
github.com/docker/distribution v2.8.1+incompatible
github.com/docker/docker v23.0.1+incompatible
github.com/golang/protobuf v1.5.2
github.com/moby/buildkit v0.11.3
github.com/moby/buildkit v0.11.4
github.com/moby/moby v23.0.1+incompatible
github.com/moby/patternmatcher v0.5.0
github.com/pkg/errors v0.9.1
Expand All @@ -29,14 +29,14 @@ replace (
)

require (
cloud.google.com/go v0.107.0 // indirect
cloud.google.com/go/compute v1.15.1 // indirect
cloud.google.com/go v0.110.0 // indirect
cloud.google.com/go/compute v1.18.0 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v0.8.0 // indirect
cloud.google.com/go/kms v1.7.0 // indirect
cloud.google.com/go/logging v1.6.1 // indirect
cloud.google.com/go/longrunning v0.3.0 // indirect
cloud.google.com/go/storage v1.28.0 // indirect
cloud.google.com/go/iam v0.12.0 // indirect
cloud.google.com/go/kms v1.9.0 // indirect
cloud.google.com/go/logging v1.7.0 // indirect
cloud.google.com/go/longrunning v0.4.1 // indirect
cloud.google.com/go/storage v1.28.1 // indirect
github.com/AlecAivazis/survey/v2 v2.0.5 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.2.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.0 // indirect
Expand Down Expand Up @@ -90,9 +90,10 @@ require (
github.com/cheggaaa/pb v1.0.29 // indirect
github.com/cloudflare/circl v1.1.0 // indirect
github.com/containerd/console v1.0.3 // indirect
github.com/containerd/containerd v1.6.16 // indirect
github.com/containerd/containerd v1.7.0 // indirect
github.com/containerd/continuity v0.3.1-0.20230206214859-2a963a2f56e8 // indirect
github.com/containerd/typeurl v1.0.2 // indirect
github.com/containerd/typeurl/v2 v2.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/djherbis/times v1.5.0 // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
Expand Down Expand Up @@ -123,7 +124,7 @@ require (
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/google/wire v0.5.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.1 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
github.com/googleapis/gax-go/v2 v2.7.0 // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
Expand Down Expand Up @@ -173,11 +174,11 @@ require (
github.com/json-iterator/go v1.1.12 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.15.12 // indirect
github.com/klauspost/compress v1.16.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/mattn/go-runewidth v0.0.14 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect
github.com/mitchellh/cli v1.1.5 // indirect
Expand All @@ -192,16 +193,16 @@ require (
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/sys/signal v0.7.0 // indirect
github.com/moby/sys/symlink v0.2.0 // indirect
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
github.com/moby/term v0.0.0-20221205130635-1aeaba878587 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/natefinch/atomic v1.0.1 // indirect
github.com/oklog/run v1.1.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20220303224323-02efb9a75ee1 // indirect
github.com/opencontainers/runc v1.1.3 // indirect
github.com/opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b // indirect
github.com/opencontainers/runc v1.1.4 // indirect
github.com/opentracing/basictracer-go v1.1.0 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pgavlin/goldmark v1.1.33-0.20200616210433-b5eb04559386 // indirect
Expand Down Expand Up @@ -247,37 +248,37 @@ require (
github.com/vmihailenco/msgpack/v4 v4.3.12 // indirect
github.com/vmihailenco/tagparser v0.1.1 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
github.com/zclconf/go-cty v1.12.1 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.29.0 // indirect
go.opentelemetry.io/otel v1.11.1 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.11.1 // indirect
go.opentelemetry.io/otel/sdk v1.11.1 // indirect
go.opentelemetry.io/otel/trace v1.11.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.40.0 // indirect
go.opentelemetry.io/otel v1.14.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.14.0 // indirect
go.opentelemetry.io/otel/metric v0.37.0 // indirect
go.opentelemetry.io/otel/sdk v1.14.0 // indirect
go.opentelemetry.io/otel/trace v1.14.0 // indirect
go.opentelemetry.io/proto/otlp v0.19.0 // indirect
go.uber.org/atomic v1.10.0 // indirect
gocloud.dev v0.28.0 // indirect
gocloud.dev/secrets/hashivault v0.28.0 // indirect
golang.org/x/crypto v0.6.0 // indirect
golang.org/x/mod v0.9.0 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/oauth2 v0.4.0 // indirect
golang.org/x/oauth2 v0.5.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/term v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/term v0.6.0 // indirect
golang.org/x/text v0.8.0 // indirect
golang.org/x/time v0.2.0 // indirect
golang.org/x/tools v0.6.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/api v0.106.0 // indirect
google.golang.org/api v0.110.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f // indirect
google.golang.org/genproto v0.0.0-20230306155012-7f2fa6fef1f4 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gotest.tools/v3 v3.4.0 // indirect
lukechampine.com/frand v1.4.2 // indirect
sourcegraph.com/sourcegraph/appdash v0.0.0-20211028080628-e2786a622600 // indirect
)
Loading

0 comments on commit 993e8a3

Please sign in to comment.