Add metadata from get steps to the local build var

[PentaHelix]

Changes proposed by this PR

Metadata from get steps are added to the local build var ((.:)) under the name of the get step. E.g.:

- get: repo

Will then have all it's metadata available under the var ((.:repo.<metadata-field-name)).

Example from CI:

Will then have the following vars available in the build for later steps to use:

• ((.:repo.commit))
• ((.:repo.author))
• ((.:repo.author_date))
• etc.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: PentaHelix / name: Jakob Kuen (1d1224e)

[PentaHelix]

There is probably a better place for this function

[taylorsilva]

I'd add it onto a new type of []MetadataField here:

concourse/atc/resource_types.go

Lines 10 to 13 in e81401c

	type MetadataField struct {
	Name string `json:"name"`
	Value string `json:"value"`
	}

Something like:

type Metadata []MetadataField

func (m Metadata) AsMap() map[string]any {
	result := make(map[string]any, len(m))
	for _, v := range m {
		result[v.Name] = v.Value
	}
	return result
}

Then VersionResult can also use that type:

concourse/atc/resource/resource.go

Lines 19 to 22 in e81401c

	type VersionResult struct {
	Version atc.Version `json:"version"`
	Metadata []atc.MetadataField `json:"metadata,omitempty"`
	}

Then AsMap(versionResult.Metadata) would become versionResult.Metadata.AsMap()!

[taylorsilva]

@PentaHelix Thanks for the PR! I have added you the all team in the concourse github org, which will allow you to see the build logs for the PR checks. You'll probably get an email to accept the invite.

[taylorsilva]

Thanks for the PR! Will review after I get 8.0 out.

[taylorsilva]

LGTM! I suggested how you can refactor AsMap() if you want to. As-is though, this works as expected.

Given this pipeline:

resources:
  - name: metadata
    type: mock
    source:
      metadata:
        - name: hello
          value: from metadata
        - name: hope
          value: it works!

jobs:
  - name: job
    plan:
      - get: metadata
      - task: echo
        config:
          platform: linux
          image_resource:
            type: mock
            source:
              mirror_self: true
          run:
            path: bash
            args:
              - -c
              - |
                echo ((.:metadata.hello))
                echo ((.:metadata.hope))

It worked!

If you don't have time to refactor I'll merge it as-is and refactor it later. Let me know. Thanks again for the PR!

[taylorsilva]

I'd add it onto a new type of []MetadataField here:

concourse/atc/resource_types.go

Lines 10 to 13 in e81401c

	type MetadataField struct {
	Name string `json:"name"`
	Value string `json:"value"`
	}

Something like:

type Metadata []MetadataField

func (m Metadata) AsMap() map[string]any {
	result := make(map[string]any, len(m))
	for _, v := range m {
		result[v.Name] = v.Value
	}
	return result
}

Then VersionResult can also use that type:

concourse/atc/resource/resource.go

Lines 19 to 22 in e81401c

	type VersionResult struct {
	Version atc.Version `json:"version"`
	Metadata []atc.MetadataField `json:"metadata,omitempty"`
	}

Then AsMap(versionResult.Metadata) would become versionResult.Metadata.AsMap()!

[kcbimonte]

To prevent possible collisions with Local Vars, wonder if it'd be beneficial to use the resource name instead of a . to denote that it's coming from a resource instead of a local var.

Like if I have a local var in the job of metadata and the mock resource providing metadata, which one "wins"?

[taylorsilva]

@PentaHelix asked about that too. My response: https://github.com/orgs/concourse/discussions/9417#discussioncomment-15479153

After this PR, if someone currently has a pipeline that has a get step with the same name as load_var step, nothing will functionally change for them. The load_var will overwrite the metadata.

The part before the colon is also meant to be used to refer to var_sources, so this would introduce a new risk of step names colliding with var_sources which I think is a worse situation to put users in.