Disabling scheduling for Pipeline Material still schedules pipeline (sometimes) #9870
Labels
Comments
|
Are you able to clarify how you are confident that the reason for the trigger is due to
Or the equivalent (subtle, undocumented) "information" icon on the stage details for one of these ones you cancelled:
Also, side note - which GoCD version is this? It looks older. |
Pretty certain. It's part of my procedure to check and swat pipelines after I commit to the Configuration repo. The
Ahh yes, had some trouble going through the upgrade process. |
|
Yes, that I'm not aware of a specific bug that might have been fixed, but I am curious about the use of the It's also detecting a change in that same repo. Would that change have normally passed the whitelist? (noting that you have tried to disable auto_updates on that repo/material) I think to figure out what is going on we might need to look at server logs when the |
|
I wasn't able to reproduce this. Not sure I have the setup right.
The above trigger happened properly on a commit to repo.2. A commit to repo.1 did not trigger the downstream pipeline. Might need a bit of your help to come up with a simplified case which reproduces your scenario. This is the config I used: pipelines:
upstream-pipeline-1:
group: test
materials:
mygit:
git: /tmp/repo.1
stages:
- stage1:
jobs:
job1:
tasks:
- exec:
command: echo
downstream-pipeline-1:
group: test
materials:
mygit:
git: /tmp/repo.2
upstream:
ignore_for_scheduling: true
pipeline: upstream-pipeline-1
stage: stage1
stages:
- stage1a:
jobs:
job1a:
tasks:
- exec:
command: echo
arguments:
- "hello"I see Chad noted the fact that you use |
|
Taking notes as I try this. 1. Everything is stable:
test.gocd.yaml in the config repo looks like this:pipelines:
upstream-pipeline-1:
group: test
materials:
mygit:
git: /tmp/repo.1
stages:
- stage1:
jobs:
job1:
tasks:
- exec:
command: echo
downstream-pipeline-1:
group: test
materials:
models:
type: configrepo
shallow_clone: true
auto_update: false
whitelist:
- downstream/down.gocd.yaml
upstream:
ignore_for_scheduling: true
pipeline: upstream-pipeline-1
stage: stage1
stages:
- stage1a:
jobs:
job1a:
tasks:
- exec:
command: echo
arguments:
- "hello"downstream/down.gocd.yaml in the config repo looks like this:pipelines:
test:
group: test
materials:
mygit:
git: /tmp/repo.2
stages:
- stage1:
jobs:
job1:
tasks:
- exec:
command: echo2. A commit to repo.2I expect only the
That went as expected:
Everything stable again. 3. A commit to repo.1I expect only
Yes, that's fine. Everything stable again. 4. A commit to the config repo itself - but not to an allow-listed fileI expected it to trigger nothing, due to the allow-list.
But it triggered the downstream pipeline. Is that the bug you're seeing? 5. A commit to the config repo itself - but to an allow-listed fileI expect it to trigger the
Yes, that's correct.
6. Another commit to a non-allow-listed file (like in step 4)I thought that the allow-list might now take effect. And it did. Nothing was triggered.
So, if step 4 is a reproduction of the bug, then the fact that the upstream pipeline has a "waiting" run which did not trigger a downstream pipeline line might somehow be causing it to trigger the pipeline, when a related material has an allow-list which should have prevented it from triggering. Meaning: In step 4, the downstream pipeline should be checked against two materials (git commit with a non-allow-listed file changed, and a pipeline with an upstream run available but not allowed to be used due to the |
|
If that is really what is happening ... I'm not sure I know enough about that part of the code to fix it for sure. But, is adding a second stage to
In my tests, once I got a green "Stage 2" (just the one time), the bug shown in Step 4 earlier did not happen. The allowlist worked properly. test.gocd.yaml in the config repo looks like this:pipelines:
upstream-pipeline-1:
group: test
materials:
mygit:
git: /tmp/repo.1
stages:
- stage1:
jobs:
job1:
tasks:
- exec:
command: echo
- stage2:
approval:
type: manual
jobs:
job1:
tasks:
- exec:
command: echo
downstream-pipeline-1:
group: test
materials:
models:
type: configrepo
shallow_clone: true
auto_update: false
whitelist:
- downstream/down.gocd.yaml
upstream:
pipeline: upstream-pipeline-1
stage: stage2
stages:
- stage1a:
jobs:
job1a:
tasks:
- exec:
command: echo
arguments:
- "hello" |
|
arvindsv I think the steps you are missing occur after 6. That is, you must commit to the configrepo with non-allow-listed file changes, and then you are in a situation where upstream pipeline runs will trigger the downstream. I have reproduced the issue with the latest gocd server, and simplified the pipelines so there is no config repos or yaml pipelines. Server Version: See the following triggered pipeline: Full server config added to bottom of post, and the git repos are public so feel free to commit. Setup looks like:
Test pipeline materials: (note the whitelist targeting txt files in the <materials>
<git url="https://github.com/badbort/gocd-bug1.git" invertFilter="true">
<filter>
<ignore pattern="Proj1/**/*.txt" />
</filter>
</git>
<pipeline pipelineName="Config" stageName="Config" ignoreForScheduling="true" materialName="Config" />
</materials>Repos involved: Now do the following:
See below for pipeline run that was triggered by above steps. I have had another confusing issue which the reverse happened: made a commit to Server XMLServer Log |
|
Continuing from above post You can continuously repeat the the process.
No changes to the Proj1 dir have been made at all. I think you can repeat these steps endlessly. I've been giving gocd time in between commits just to make sure the server polls and is aware of commits. Manually running
|
WorkaroundSeeing as I can reliably trigger the Setup: Once again, push changes to
Pipeline: <pipeline name="Test2GitMaterials">
<materials>
<git url="https://github.com/badbort/gocd-bug1.git" invertFilter="true" dest="Project">
<filter>
<ignore pattern="Proj1/**/*.txt" />
</filter>
</git>
<git url="https://github.com/badbort/gocd-bug1-shared.git" branch="main" dest="Shared">
<filter>
<ignore pattern="**/*.*" />
</filter>
</git>
</materials>
<stage name="Test">
<jobs>
<job name="Proj1">
<tasks>
<exec command="echo">
<arg>123</arg>
</exec>
</tasks>
</job>
</jobs>
</stage>
</pipeline> |
|
Can you confirm you can reproduce the issue in the server xml define in this post? #9870 (comment) I've put together the simplest scenario I can above |
|
@badbort Hello! I haven't tried that. But, I was able to reproduce it in Step 4 (in one of my previous comments). Is that wrong? Either way, as I mentioned, I agree that it's a bug. Tricky to reproduce but possible. I'm not sure exactly why it happens (in code, I mean). Hopefully the workaround you mentioned, or the one I mentioned helps in this situation. |
|
@arvindsv I think its similar scenario but slightly different. Root cause may be the same.
The server xml config I've linked earlier reproduces the issue consistently and simplifies the configuration, removing yaml and config pipelines from the equation
And following post discovered you can continue to reproduce this doing the following:
Note: 8 can be replaced by manually running the upstream pipeline For now I'm converting pipelines to use the workaround mentioned, so this is a low priority issue for me. |
|
This issue has been automatically marked as stale because it has not had activity in the last 90 days. |
|
Hi, I'm encountering this bug with version 22.2.0. Has there been any work done on this? The workaround mentioned isn't an option, as the pipeline concerned needs to depend on both an upstream pipeline and a git repo. I'm not familiar with the GoCD codebase, but if there was some previous work done to build on, or even if someone could point me to the right place in the code to look at, I could spend some time trying a fix |
|
I think it's probably safe to say that no work has been done to figure out what is happening here, or what the unintended consequences might be of trying to fix it 😅 Personally i haven't grokked the specific scenario enough to guess at where the issue might be, unfortunately. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Report
Summary
I'm encountering issues where a pipeline runs in response to a commit as expected, but triggers dependencies despite those defining this pipeline as "do not schedule".
This pipeline contains common configuration and settings files and is I do not wish for it to trigger anything except the 'artifacting' pipeline.
In the above image, the offending pipeline here is
Configuration. This pipeline merely artifacts some settings files from a git repo.As you can see in the image, I have disabled the scheduling for the next pipeline. We'll call that pipeline 'Project-A'.
Project-Ahas a git material calledClient1.The bug I'm reporting is
Configurationstill triggersProject-A, but the behaviour is not consistent. There are many pipelines just likeProject-Abut for different projects and clients - just different content files. Some of these pipelines aren't triggered.Other things to note about the pipeline setup:
Client1repo, and there is a pipeline for each. e.g.Project-B, ect..Client1.The following image is a real world example of
Configurationit triggering many pipelines and in all cases it has had scheduling disabled. I regularly have to play whack-a-mole:The workaround I will be attempting for now was mentioned in another thread; remove the pipeline material and reference the git repo directly, with a blacklist that matches all files: */
What I think is happening is after
Configurationcompletes it checks all dependent pipelines. This check onProject-Adoes not appear to obey the whitelist forClient1. Commits toClient1do get filtered through the whitelist, butConfigurationdoes not appear to.I may abandon this intermediate artifacting pipeline if a direct git reference approach doesn't exhibit this triggering behaviour - so we will see.
The text was updated successfully, but these errors were encountered: