Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow overriding Repository / Tag on Get #337

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

adnankobir
Copy link

@adnankobir adnankobir commented Mar 9, 2023

This is a very simplistic approach to a specific use-case:

We have various docker base images that we would like to pull in to build from.

We currently use the registry-image-resource to pull this base image in oci format to satisfy the FROM requirement in our Dockerfiles from a private repository using ARG:

Dockerfile

ARG base_image=base
FROM ${base_image}

...

Pipeline

resources:
- base
   type: registry-image
   source:
     aws_access_key_id: ((id))
     aws_secret_access_key: ((key)
     aws_region: ((region))
     repository: base-dotnet
     tag: 1.0.0

jobs:
  ...
  # we trigger the pipeline some other way

  - get: base
     params:
       format: oci

  - task: build
     privileged: true
     config:
       image_resource:
         type: registry-image
         source:
            repository: concourse/oci-build-task
       inputs:
          - name: base
       ...

We now would like to parallelize our builds of multiple docker images - we produce a json that contains metadata about each of docker images we want to build - including the base we want to use - this is what requires us to override the source configuration for repository and tag:

metadata.json

[
  { 
    "service": "foo-api",
    "base_image": "foo-base",
    "base_tag": "1.0.0"
  },
  { 
    "service": "bar-api",
    "base_image": "bar-base",
    "base_tag": "2.0.0"
  }
]

Pipeline

jobs:
  ...
  # we trigger the pipeline some other way

  - load_var: metadata
    file: metadata.json

  - do:
    - get: base
       params:
         format: oci
         repository: ((.:service.base_image))
         tag: ((.:service.base_tag))

    - task: build
       privileged: true
       config:
         image_resource:
           type: registry-image
           source:
              repository: concourse/oci-build-task
         inputs:
            - name: base
       ...

     across:
       - var: service
          max_in_fligh: all
          values: ((.:metadata))

Now clearly this will have implications if we wanted to use the base/registry-image-resource to trigger.

Any thoughts around a better way to achieve what we want?
cc @taylorsilva

Signed-off-by: Adnan Kobir <adnan.kobir@gmail.com>
Signed-off-by: Adnan Kobir <adnan.kobir@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant