FetchArtifact fetches older version of an artifact and not the latest

[ProcessMaster]

Issue Type

• Bug Report

Summary

I have 9 pipelines that produce an artifact each that is a file with just one string of about 200 char. I have another pipeline that fetches all those artifacts before doing something. I noticed today that the artifact fetched from one of the pipelines is not the latest one that was created today but is the previous version created about a month ago.

Environment

{
  "Timestamp": "2019-05-07T15:29:02-07:00",
  "Go Server Information": {
    "Version": "19.1.0 (8469-3885582184c6f7c4bbbeb94239e5dba6f5f772f4)"
  },

Basic environment details

Go Server Version: | 19.1.0 (8469-3885582184c6f7c4bbbeb94239e5dba6f5f772f4)
JVM version: | 1.8.0_144
OS Information: | Linux 3.13.0-162-generic
Usable space in artifacts repository: | 18.5 GB
Database schema version: | 1901001

Additional Environment Details

Steps to Reproduce

I am not able to reproduce it.

Expected Results

Was expecting it use the artifact from the latest version of the pipeline. In the case below 3-1134bf0 and not the 2-1134bf0

Actual Results

It used the older artifact.

Possible Fix

Log snippets

From upstream pipeline - look at this line for the version it is using -

[go] Fetching artifact [image] from [annotations-django-stg-build/2-1134bf0/build-tag-push/1/defaultJob]

[go] **Job Started: 2019-05-07 10:39:10 PDT**
[go] Start to prepare annotations-django-stg-deploy/9-7e2e66f/defaultStage/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
[go] Start to build annotations-django-stg-deploy/9-7e2e66f/defaultStage/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
[go] Task: fetch artifact [image] => [ecs/annotations/django-server/] from [annotations-django-stg-build/build-tag-push/defaultJob]took: 0.79s
[go] Fetching artifact [image] from [annotations-django-stg-build/2-1134bf0/build-tag-push/1/defaultJob]
[go] Saved artifact to [pipelines/annotations-django-stg-deploy/ecs/annotations/django-server/image] after verifying the integrity of its contents.
[go] Task status: passed, took: 0.79s

From downstrem pipeline - see this line for the version -
[go] Start to upload annotations-django-stg-build/3-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]

[go] Job Started: 2019-05-07 10:20:26 PDT
[go] Start to prepare annotations-django-stg-build/3-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
[go] Start to build annotations-django-stg-build/3-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
....
....
[go] Start to create properties annotations-django-stg-build/3-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
[go] Start to upload annotations-django-stg-build/3-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
[go] Uploading artifacts from /home/go/work/pipelines/annotations-django-stg-build/django_server/build/image to [defaultRoot]
[go] Job completed annotations-django-stg-build/3-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-02.aws.counsyl.com [/home/go/work]
...
COMPLETED ON: 07 May 2019 at 10:24:51 Local Time

And here is the older version that it is using -
[go] Job Started: 2019-04-26 19:44:25 PDT
[go] Start to prepare annotations-django-stg-build/2-1134bf0/build-tag-push/1/defaultJob on go-agent-misc-01.aws.counsyl.com [/home/go/work]

Code snippets/Screenshots

Any other info

[arvindsv]

@ProcessMaster Do the 9 upstream pipelines share any (repo) materials?

[arvindsv]

If possible, post a screenshot of the VSM please.

[ProcessMaster]

The first 9 pipelines use the same git repo but they are independent. Not sure what did you mean by sharing.

[ProcessMaster]

[ProcessMaster]

Any update on this? Thanks.

[arvindsv]

No. I haven't had a chance to look. I think this has something to do with fan-in, but it's probably easiest to find out if I set this up and reproduce it.

[ankitsri11]

@ProcessMaster I tried the setup and wasn't able to replicate on both latest and 19.1.0 version of GoCD.

I tried below:

1. Created 9 pipelines pointing to the same git-1 material
2. Created a Downstream pipeline with all the 9 pipelines as pipeline material (upstream pipeline ) and a git-2 material.
3. Triggered all the upstream pipelines with git-1 material change and later downstream pipeline was triggered due to dependency
4. Tried force triggering one or two upstream pipelines, downstream pipeline triggered and fetched the correct artifact.

I've pushed all the pipelines configuration to a config-repo here: https://github.com/ankitsri11/testing-6262

My current suspicion is that the behavior you are seeing could happen if the upstream pipeline was manually triggered in between the downstream pipeline was running. In that case, the downstream pipeline will fetch the older artifact and not the latest.

[arvindsv]

No response. Stale. Closing.