Defining the build matrix only with matrix.include leads to an additional, unwanted job #4681

bilderbuchi · 2015-08-20T07:44:37Z

Simple reproduction of the problem: A .travis.yml, makes two builds, one without environment variables defined. One build only, with TARGET=linux is desired:

language: c++
compiler: gcc
matrix:
  include:
    - os: linux
      env: TARGET="linux"

install: true

script: true

leads to

    15.1 Compiler: gcc  no environment variables set
    15.2 Compiler: gcc  TARGET="linux"

https://travis-ci.org/bilderbuchi/travis_production_test/builds/76336816

Related issue: #1228

The text was updated successfully, but these errors were encountered:

BanzaiMan · 2015-08-24T15:21:00Z

For now, you can exclude the default job:

language: c++
compiler: gcc
matrix:
  include:
    - os: linux
      env: TARGET="linux"
  exclude:
    - compiler: gcc

bilderbuchi · 2015-08-24T21:25:25Z

Ah, nice workaround, thanks!

o11c · 2015-09-30T20:04:07Z

I think this works if you omit the top-level compiler: gcc line.

ldionne · 2015-11-07T17:43:59Z

The proposed workaround is very annoying, because some of us might actually need the global compiler: gcc (or in my case compiler: clang). The other option is to end up with compiler: gcc globally and then to redefine compiler: clang for each job inside matrix.include, but when you have 20+ jobs, that makes a lot of repetition. I beg you for a clean fix.

- Improve the logical separation between jobs in .travis.yml. - Put all the jobs that are allowed to fail at the end in order to know the result of important jobs first. - Workaround the empty job bug [1] and regroup documentation jobs [1]: travis-ci/travis-ci#4681

wavexx · 2016-02-05T17:34:16Z

Is there a proper fix planned for this issue?

I use language: python with matrix expansion to test for optional dependencies and I'm tired to see the build fail for the extraneous entry.

BanzaiMan · 2016-02-20T20:21:46Z

@wavexx Is the workaround shown above not working for you?

wavexx · 2016-02-20T20:33:37Z

I have to work-around basically all projects where I'm using travis.

See details here: travis-ci/travis-ci#4681

…nt variables. Workaround travis-ci/travis-ci#4681

Details: * We define the job matrix explicitly with include statements, so that we have granular control over which combinations will run. Also, an explicit job matrix is required for OS-X support on the public Travis. When defining the job matrix explicitly, there are Travis environments that produce an additional default job. See these Travis issues: travis-ci/travis-ci#1228 travis-ci/travis-ci#4681 travis-ci/travis-ci#9843 The public Travis does not seem to have this issue anymore, but Travis@IBM does have this issue (as of 9/2018). The workaround for this issue is to define variables globally and to exclude this same variable value in the matrix definition. Experiments have shown that not all variable combinations work. Using a combination of 'language' and 'os' set to the default values works. * TODO: OS-X is not supported on Travis@IBM, so the inclusion of OS-X into the job matrix has been made conditional. Signed-off-by: Andreas Maier <maiera@de.ibm.com>

…absent Details: * We define the job matrix explicitly with include statements, so that we have granular control over which combinations will run. Also, an explicit job matrix is required for OS-X support on the public Travis. When defining the job matrix explicitly, there are Travis environments that produce an additional default job. See these Travis issues: travis-ci/travis-ci#1228 travis-ci/travis-ci#4681 travis-ci/travis-ci#9843 The public Travis does not seem to have this issue anymore, but Travis@IBM does have this issue (as of 9/2018). The workaround for this issue is to define variables globally and to exclude this same variable value in the matrix definition. Experiments have shown that not all variable combinations work. Using a combination of 'language' and 'os' set to the default values works. * OS-X is not supported on Travis@IBM, so this change adds a statement that exits the job when os=osx runs on Linux. Signed-off-by: Andreas Maier <maiera@de.ibm.com>

Details: * We define the job matrix explicitly with include statements, so that we have granular control over which combinations will run. Also, an explicit job matrix is required for OS-X support on the public Travis. When defining the job matrix explicitly, there are Travis environments that produce an additional default job. See these Travis issues: travis-ci/travis-ci#1228 travis-ci/travis-ci#4681 travis-ci/travis-ci#9843 The public Travis does not seem to have this issue anymore, but Travis@IBM does have this issue (as of 9/2018). The workaround for this issue is to define variables globally and to exclude this same variable value in the matrix definition. Experiments have shown that not all variable combinations work. Using a combination of 'language' and 'os' set to the default values (language=ruby, os=linux) works. As an extra safety net against running with language=ruby, there is a statement that exits the job when the language is ruby. * OS-X is not supported on Travis@IBM, so this change adds a statement that exits the job when os=osx runs on Linux. Signed-off-by: Andreas Maier <maiera@de.ibm.com>

…nd the mayhem triggered from this change So we adding the compiler, to see if it will get travis to build the right matrix. then linter broke on addons for some weird reason, so remove addons->apt->update too

@here

* netdata/packaging/ci: [WIP] Redesign packaging release * netdata/packaging/ci: Add explicitly os definition, also remove sourcing of slack, its already done and rename before_install -> install * netdata/packaging/ci: fix stray stage name * netdata/packaging/ci: style * netdata/packaging/ci: fix syntax for yaml * netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now) * Revert "netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)" This reverts commit ad5556a. wow travis - this is weird * netdata/packaging/ci: a few more changes * netdata/packaging/ci: This is totally experimental, will adjust after i test it * netdata/packaging/ci: Reinstate original design - the problem i am trying to solve is more travis-related and fundamental on the way travis works * netdata/packaging/ci: i wonder.. * Revert "netdata/packaging/ci: i wonder.." This reverts commit a160df4. * netdata/packaging/ci: Fix broken items * netdata/packaging/ci: docker images are built and published independently. As long as tagging has taken place earlier, during packaging * netdata/packaging/ci: Following hints from travis-ci/travis-ci#4681 and the mayhem triggered from this change So we adding the compiler, to see if it will get travis to build the right matrix. then linter broke on addons for some weird reason, so remove addons->apt->update too * netdata/packaging/ci: Now do the rest of the changes, since with addition of compiler, things got worse * netdata/packaging/ci: Ok, as expected the workaround from them did not work. Lets get creative though and assume they exclude what they added, so lets do that for os * netdata/packaging/ci: test slack * netdata/packaging/ci: Bring the modified slack script on this branch * netdata/packaging/ci: Add a few more slack messages * Fix empty stage occurrence * netdata/packaging/ci: Another set of restructure changes, explained below 1) Publish release -> Publish for release (minor rename for clarity) 2) Add some more work on the make dist testing (temp folder for root and for now just validate the existence of the tarball) 3) Add some more slack notifications (that is still beta, will have to properly refine once we are done with changes of the flow) 4) Restructure the release process, finally a) relase.sh becomes draft_release.sh and changelog_and_tag_release.sh, basically splitting up the steps into meaningful independent blocks b) Adjust the pipeline, now we run 'changelog_and_tag_release.sh' to get the latest changelog and tag the release, then we run artifacts submission and draft release generation on github still more things to re-check on this step, it will become clearer once i get it up to speed to run on my local repos end to end * netdata/packaging/ci: Fix wrong path usage * netdata/packaging/ci: Update webhook URL * netdata/packaging/ci: Update stage config as proposed by the fix from travis team * netdata/packaging/ci: Another round of restructuring, after 1-1 comparison with old version Briefly: 1) Publish for release stage will only run when there is a TAG defined (To be tested though, but thats the idea). The code previously had the releaser.sh script that did exactly that, if tagger script failed, it would stop process. Now we have distributed the processing in clear chunks and we pick up the process based on shared info that travis or the environment provides 2) Rename <<Create release (only on special commit msg)>> to a more appropriate name, we are not really creating release there its more complicated 3) Move the labeler execution to the packaging step, since publishing now not happening all the time. Previously we hacked this by exit 0 with the tagger and allow the labeler (which was the second part of the execution) to run. 4) before_install in the publishing stage is wrong. We only need this as before_deploy before deploy execution. That is where we use it, really. 5) Rename changelog_and_tag_release.sh to generate_changelog_and_tag_release.sh, just to be clearer. No problem with slightly longer file names 6) Do not let generate_changelog_and_tag_release.sh execute if GIT_TAG is not defined, just exit quietly with a message and a success code 7) Remove the tagger from the generate_changelog_and_tag_release.sh, its confusing to shove it in there. Pull it out and let it run on a clear step from the pipeline but within the same machine to make sure it shares environment with generate_changelog_and_tag_release.sh (So it has to be within the same job) This is it mostly, keep in mind that the nightly and release parts are not tested yet. After this commit, there will be more code that will allow me to execute all the flows on my pipeline and fully test and adjust the workflows as needed * netdata/packaging/ci: Add more notifications, adjust the existing ones too * netdata/packaging/ci: Add safeguards for beta, do a rename too 1) rename generate_changelog.sh -> generate_changelog_for_nightlies.sh, just to provide clarity on the usage. Note: we will simplify this later, this was a workaround that raised a bug on release after refactoring nightlies. We will have to clean this up properly once we merge the new pipeline flow 2) Add checks based on TRAVIS_REPO_SLUG to identify when we running on personal repo and abort mission critical actions like deploying or generating changelogs Note: The deployment step for now will simply fail, due to the lack of the necessary encryption keys on the environment. At a later iteration, we should have beta buckets on GCS and also beta repositories on Docker to properly test our publishing capability on our beta stack * netdata/packaging/ci: Refactor README.md -- still half way there, will resume when i am done with testing * netdata/packaging/ci: At this stage make the scripts harder to run, add checks to bypass tested execution. * netdata/packaging/ci: Seems that adding the install tag within the stage, kind of overriding the outer install pull out install items from packaging to the main install, no point to isolate two packages, just install all required all the time * netdata/packaging/ci: tag obviously cant work at that step, moving it down to the jobs and i need to recheck if the syntax is applicable for travis too * netdata/packaging/ci: So the jobs in a list dont share the environment, lets see if we do it otherwise. also remove the tag check, not work as expected * netdata/packaging/ci: Add another way to check the RC condition * netdata/packaging/ci: experimental - check conditions on message * netdata/packaging/ci: Introduce an alternate master branch for the workflow testing 1. I created pipeline-redesign-master that will be the master branch for my use cases 2. Modified travis to trigger all production workflows on that branch, so that i can experiment freely with test commits to trigger process and verify execution flow * remove spaces * netdata/packaging/ci: comment out * netdata/packaging/ci: wording nit * netdata/packaging/ci: adjust the condition properly * netdata/packaging/ci: test condition for triggering on specific keywords * netdata/packaging/ci: try different syntax * netdata/packaging/ci: remove depth, causes problems as we dont get all required objects from repo (adds --depth 50 in clone) * Missing equal sign in regex? * netdata/packaging/ci: Update tagger script 1) More verbose messages 2) Update comments copyrights etc 3) Rename release_candidate to set_tag_release_candidate 4) Add curlies on the variables, also change the way we check for path * netdata/packaging/ci: reinstate git depth * netdata/packaging/ci: Remove git depth from packaging too * netdata/packaging/ci: RTFM - was using wrong syntax for slack @here approach * netdata/packaging/ci: Using install tag at the stage layer, overrides the global one for some reason. fix that * netdata/packaging/ci: Dont use the pattern matching from travis, do it outside with a script * netdata/packaging/ci: Do it the other way, script execution doesnt work there * netdata/packaging/ci: Attempt to fix conditional on release, remove the faulty assumptions as provided by travis team * netdata/packaging/ci: reintroduce the tag checking * netdata/packaging/ci: you need to bring tag checking at the stage, as it is calculated during packaging step * netdata/packaging/ci: you cant escape that one for the moment, merge them and we will rework it later again * netdata/packaging/ci: dont forget git depth * netdata/packaging/ci: [WIP] Redesign packaging release * netdata/packaging/ci: Add explicitly os definition, also remove sourcing of slack, its already done and rename before_install -> install * netdata/packaging/ci: fix stray stage name * netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now) * Revert "netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)" * netdata/packaging/ci: Reinstate original design - the problem i am trying to solve is more travis-related and fundamental on the way travis works * netdata/packaging/ci: Fix broken items * netdata/packaging/ci: docker images are built and published independently. As long as tagging has taken place earlier, during packaging * netdata/packaging/ci: Following hints from travis-ci/travis-ci#4681 and the mayhem triggered from this change So we adding the compiler, to see if it will get travis to build the right matrix. then linter broke on addons for some weird reason, so remove addons->apt->update too * netdata/packaging/ci: Now do the rest of the changes, since with addition of compiler, things got worse * netdata/packaging/ci: Ok, as expected the workaround from them did not work. Lets get creative though and assume they exclude what they added, so lets do that for os * netdata/packaging/ci: test slack * netdata/packaging/ci: Bring the modified slack script on this branch * netdata/packaging/ci: Add a few more slack messages * Fix empty stage occurrence * netdata/packaging/ci: Another set of restructure changes, explained below 1) Publish release -> Publish for release (minor rename for clarity) 2) Add some more work on the make dist testing (temp folder for root and for now just validate the existence of the tarball) 3) Add some more slack notifications (that is still beta, will have to properly refine once we are done with changes of the flow) 4) Restructure the release process, finally a) relase.sh becomes draft_release.sh and changelog_and_tag_release.sh, basically splitting up the steps into meaningful independent blocks b) Adjust the pipeline, now we run 'changelog_and_tag_release.sh' to get the latest changelog and tag the release, then we run artifacts submission and draft release generation on github still more things to re-check on this step, it will become clearer once i get it up to speed to run on my local repos end to end * netdata/packaging/ci: Fix wrong path usage * netdata/packaging/ci: Update webhook URL * netdata/packaging/ci: Update stage config as proposed by the fix from travis team * netdata/packaging/ci: Another round of restructuring, after 1-1 comparison with old version Briefly: 1) Publish for release stage will only run when there is a TAG defined (To be tested though, but thats the idea). The code previously had the releaser.sh script that did exactly that, if tagger script failed, it would stop process. Now we have distributed the processing in clear chunks and we pick up the process based on shared info that travis or the environment provides 2) Rename <<Create release (only on special commit msg)>> to a more appropriate name, we are not really creating release there its more complicated 3) Move the labeler execution to the packaging step, since publishing now not happening all the time. Previously we hacked this by exit 0 with the tagger and allow the labeler (which was the second part of the execution) to run. 4) before_install in the publishing stage is wrong. We only need this as before_deploy before deploy execution. That is where we use it, really. 5) Rename changelog_and_tag_release.sh to generate_changelog_and_tag_release.sh, just to be clearer. No problem with slightly longer file names 6) Do not let generate_changelog_and_tag_release.sh execute if GIT_TAG is not defined, just exit quietly with a message and a success code 7) Remove the tagger from the generate_changelog_and_tag_release.sh, its confusing to shove it in there. Pull it out and let it run on a clear step from the pipeline but within the same machine to make sure it shares environment with generate_changelog_and_tag_release.sh (So it has to be within the same job) This is it mostly, keep in mind that the nightly and release parts are not tested yet. After this commit, there will be more code that will allow me to execute all the flows on my pipeline and fully test and adjust the workflows as needed * netdata/packaging/ci: Add more notifications, adjust the existing ones too * netdata/packaging/ci: Add safeguards for beta, do a rename too 1) rename generate_changelog.sh -> generate_changelog_for_nightlies.sh, just to provide clarity on the usage. Note: we will simplify this later, this was a workaround that raised a bug on release after refactoring nightlies. We will have to clean this up properly once we merge the new pipeline flow 2) Add checks based on TRAVIS_REPO_SLUG to identify when we running on personal repo and abort mission critical actions like deploying or generating changelogs Note: The deployment step for now will simply fail, due to the lack of the necessary encryption keys on the environment. At a later iteration, we should have beta buckets on GCS and also beta repositories on Docker to properly test our publishing capability on our beta stack * netdata/packaging/ci: Refactor README.md -- still half way there, will resume when i am done with testing * netdata/packaging/ci: At this stage make the scripts harder to run, add checks to bypass tested execution. * netdata/packaging/ci: Seems that adding the install tag within the stage, kind of overriding the outer install pull out install items from packaging to the main install, no point to isolate two packages, just install all required all the time * netdata/packaging/ci: tag obviously cant work at that step, moving it down to the jobs and i need to recheck if the syntax is applicable for travis too * netdata/packaging/ci: So the jobs in a list dont share the environment, lets see if we do it otherwise. also remove the tag check, not work as expected * netdata/packaging/ci: Add another way to check the RC condition * netdata/packaging/ci: experimental - check conditions on message * netdata/packaging/ci: Introduce an alternate master branch for the workflow testing 1. I created pipeline-redesign-master that will be the master branch for my use cases 2. Modified travis to trigger all production workflows on that branch, so that i can experiment freely with test commits to trigger process and verify execution flow * remove spaces * netdata/packaging/ci: comment out * netdata/packaging/ci: wording nit * netdata/packaging/ci: adjust the condition properly * netdata/packaging/ci: test condition for triggering on specific keywords * netdata/packaging/ci: try different syntax * netdata/packaging/ci: remove depth, causes problems as we dont get all required objects from repo (adds --depth 50 in clone) * Missing equal sign in regex? * netdata/packaging/ci: Update tagger script 1) More verbose messages 2) Update comments copyrights etc 3) Rename release_candidate to set_tag_release_candidate 4) Add curlies on the variables, also change the way we check for path * netdata/packaging/ci: reinstate git depth * netdata/packaging/ci: Remove git depth from packaging too * netdata/packaging/ci: RTFM - was using wrong syntax for slack @here approach * netdata/packaging/ci: Using install tag at the stage layer, overrides the global one for some reason. fix that * netdata/packaging/ci: Dont use the pattern matching from travis, do it outside with a script * netdata/packaging/ci: Do it the other way, script execution doesnt work there * netdata/packaging/ci: Attempt to fix conditional on release, remove the faulty assumptions as provided by travis team * netdata/packaging/ci: reintroduce the tag checking * netdata/packaging/ci: you need to bring tag checking at the stage, as it is calculated during packaging step * netdata/packaging/ci: you cant escape that one for the moment, merge them and we will rework it later again * netdata/packaging/ci: dont forget git depth * netdata/packaging/ci: bring back the dist checking * netdata/packaging/ci: fix syntax * netdata/packaging/ci: no depth * netdata/packaging/ci: Introduce make dist validation tasks: Install from dist file and verify service is starting up * netdata/packaging/ci: Add a couple more information for troubleshooting * netdata/packaging/ci: Add extra information prior to executing crucial operations * netdata/packaging/ci: Make rm error soft here * netdata/packaging/ci: This is not needed here * netdata/packaging/ci: Prepare for beta stage deployment process - introduce a beta provider - adjust key decryption process, different keys for beta, different for prod - Enter conditions to deploy beta only on beta process and production only on production process - Use travis variables, to allow multiple users have beta stages for deployment, if they want * netdata/packaging/ci: add one more message * netdata/packaging/ci: Bring back naming to the actual master branch * netdata/packaging/ci: rename beta to dev * netdata/packaging/ci: Make permanent restrictions for some of the stricts, so that we don't mess things up * netdata/packaging/ci: Update README that was missed and also make sure git depth false is at the places needed * netdata/packaging/ci: fix codacy complaints

@here

* netdata/packaging/ci: [WIP] Redesign packaging release * netdata/packaging/ci: Add explicitly os definition, also remove sourcing of slack, its already done and rename before_install -> install * netdata/packaging/ci: fix stray stage name * netdata/packaging/ci: style * netdata/packaging/ci: fix syntax for yaml * netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now) * Revert "netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)" This reverts commit ad5556a. wow travis - this is weird * netdata/packaging/ci: a few more changes * netdata/packaging/ci: This is totally experimental, will adjust after i test it * netdata/packaging/ci: Reinstate original design - the problem i am trying to solve is more travis-related and fundamental on the way travis works * netdata/packaging/ci: i wonder.. * Revert "netdata/packaging/ci: i wonder.." This reverts commit a160df4. * netdata/packaging/ci: Fix broken items * netdata/packaging/ci: docker images are built and published independently. As long as tagging has taken place earlier, during packaging * netdata/packaging/ci: Following hints from travis-ci/travis-ci#4681 and the mayhem triggered from this change So we adding the compiler, to see if it will get travis to build the right matrix. then linter broke on addons for some weird reason, so remove addons->apt->update too * netdata/packaging/ci: Now do the rest of the changes, since with addition of compiler, things got worse * netdata/packaging/ci: Ok, as expected the workaround from them did not work. Lets get creative though and assume they exclude what they added, so lets do that for os * netdata/packaging/ci: test slack * netdata/packaging/ci: Bring the modified slack script on this branch * netdata/packaging/ci: Add a few more slack messages * Fix empty stage occurrence * netdata/packaging/ci: Another set of restructure changes, explained below 1) Publish release -> Publish for release (minor rename for clarity) 2) Add some more work on the make dist testing (temp folder for root and for now just validate the existence of the tarball) 3) Add some more slack notifications (that is still beta, will have to properly refine once we are done with changes of the flow) 4) Restructure the release process, finally a) relase.sh becomes draft_release.sh and changelog_and_tag_release.sh, basically splitting up the steps into meaningful independent blocks b) Adjust the pipeline, now we run 'changelog_and_tag_release.sh' to get the latest changelog and tag the release, then we run artifacts submission and draft release generation on github still more things to re-check on this step, it will become clearer once i get it up to speed to run on my local repos end to end * netdata/packaging/ci: Fix wrong path usage * netdata/packaging/ci: Update webhook URL * netdata/packaging/ci: Update stage config as proposed by the fix from travis team * netdata/packaging/ci: Another round of restructuring, after 1-1 comparison with old version Briefly: 1) Publish for release stage will only run when there is a TAG defined (To be tested though, but thats the idea). The code previously had the releaser.sh script that did exactly that, if tagger script failed, it would stop process. Now we have distributed the processing in clear chunks and we pick up the process based on shared info that travis or the environment provides 2) Rename <<Create release (only on special commit msg)>> to a more appropriate name, we are not really creating release there its more complicated 3) Move the labeler execution to the packaging step, since publishing now not happening all the time. Previously we hacked this by exit 0 with the tagger and allow the labeler (which was the second part of the execution) to run. 4) before_install in the publishing stage is wrong. We only need this as before_deploy before deploy execution. That is where we use it, really. 5) Rename changelog_and_tag_release.sh to generate_changelog_and_tag_release.sh, just to be clearer. No problem with slightly longer file names 6) Do not let generate_changelog_and_tag_release.sh execute if GIT_TAG is not defined, just exit quietly with a message and a success code 7) Remove the tagger from the generate_changelog_and_tag_release.sh, its confusing to shove it in there. Pull it out and let it run on a clear step from the pipeline but within the same machine to make sure it shares environment with generate_changelog_and_tag_release.sh (So it has to be within the same job) This is it mostly, keep in mind that the nightly and release parts are not tested yet. After this commit, there will be more code that will allow me to execute all the flows on my pipeline and fully test and adjust the workflows as needed * netdata/packaging/ci: Add more notifications, adjust the existing ones too * netdata/packaging/ci: Add safeguards for beta, do a rename too 1) rename generate_changelog.sh -> generate_changelog_for_nightlies.sh, just to provide clarity on the usage. Note: we will simplify this later, this was a workaround that raised a bug on release after refactoring nightlies. We will have to clean this up properly once we merge the new pipeline flow 2) Add checks based on TRAVIS_REPO_SLUG to identify when we running on personal repo and abort mission critical actions like deploying or generating changelogs Note: The deployment step for now will simply fail, due to the lack of the necessary encryption keys on the environment. At a later iteration, we should have beta buckets on GCS and also beta repositories on Docker to properly test our publishing capability on our beta stack * netdata/packaging/ci: Refactor README.md -- still half way there, will resume when i am done with testing * netdata/packaging/ci: At this stage make the scripts harder to run, add checks to bypass tested execution. * netdata/packaging/ci: Seems that adding the install tag within the stage, kind of overriding the outer install pull out install items from packaging to the main install, no point to isolate two packages, just install all required all the time * netdata/packaging/ci: tag obviously cant work at that step, moving it down to the jobs and i need to recheck if the syntax is applicable for travis too * netdata/packaging/ci: So the jobs in a list dont share the environment, lets see if we do it otherwise. also remove the tag check, not work as expected * netdata/packaging/ci: Add another way to check the RC condition * netdata/packaging/ci: experimental - check conditions on message * netdata/packaging/ci: Introduce an alternate master branch for the workflow testing 1. I created pipeline-redesign-master that will be the master branch for my use cases 2. Modified travis to trigger all production workflows on that branch, so that i can experiment freely with test commits to trigger process and verify execution flow * remove spaces * netdata/packaging/ci: comment out * netdata/packaging/ci: wording nit * netdata/packaging/ci: adjust the condition properly * netdata/packaging/ci: test condition for triggering on specific keywords * netdata/packaging/ci: try different syntax * netdata/packaging/ci: remove depth, causes problems as we dont get all required objects from repo (adds --depth 50 in clone) * Missing equal sign in regex? * netdata/packaging/ci: Update tagger script 1) More verbose messages 2) Update comments copyrights etc 3) Rename release_candidate to set_tag_release_candidate 4) Add curlies on the variables, also change the way we check for path * netdata/packaging/ci: reinstate git depth * netdata/packaging/ci: Remove git depth from packaging too * netdata/packaging/ci: RTFM - was using wrong syntax for slack @here approach * netdata/packaging/ci: Using install tag at the stage layer, overrides the global one for some reason. fix that * netdata/packaging/ci: Dont use the pattern matching from travis, do it outside with a script * netdata/packaging/ci: Do it the other way, script execution doesnt work there * netdata/packaging/ci: Attempt to fix conditional on release, remove the faulty assumptions as provided by travis team * netdata/packaging/ci: reintroduce the tag checking * netdata/packaging/ci: you need to bring tag checking at the stage, as it is calculated during packaging step * netdata/packaging/ci: you cant escape that one for the moment, merge them and we will rework it later again * netdata/packaging/ci: dont forget git depth * netdata/packaging/ci: [WIP] Redesign packaging release * netdata/packaging/ci: Add explicitly os definition, also remove sourcing of slack, its already done and rename before_install -> install * netdata/packaging/ci: fix stray stage name * netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now) * Revert "netdata/packaging/ci: Try removing this, doesnt seem to make much sense to be there (for now)" * netdata/packaging/ci: Reinstate original design - the problem i am trying to solve is more travis-related and fundamental on the way travis works * netdata/packaging/ci: Fix broken items * netdata/packaging/ci: docker images are built and published independently. As long as tagging has taken place earlier, during packaging * netdata/packaging/ci: Following hints from travis-ci/travis-ci#4681 and the mayhem triggered from this change So we adding the compiler, to see if it will get travis to build the right matrix. then linter broke on addons for some weird reason, so remove addons->apt->update too * netdata/packaging/ci: Now do the rest of the changes, since with addition of compiler, things got worse * netdata/packaging/ci: Ok, as expected the workaround from them did not work. Lets get creative though and assume they exclude what they added, so lets do that for os * netdata/packaging/ci: test slack * netdata/packaging/ci: Bring the modified slack script on this branch * netdata/packaging/ci: Add a few more slack messages * Fix empty stage occurrence * netdata/packaging/ci: Another set of restructure changes, explained below 1) Publish release -> Publish for release (minor rename for clarity) 2) Add some more work on the make dist testing (temp folder for root and for now just validate the existence of the tarball) 3) Add some more slack notifications (that is still beta, will have to properly refine once we are done with changes of the flow) 4) Restructure the release process, finally a) relase.sh becomes draft_release.sh and changelog_and_tag_release.sh, basically splitting up the steps into meaningful independent blocks b) Adjust the pipeline, now we run 'changelog_and_tag_release.sh' to get the latest changelog and tag the release, then we run artifacts submission and draft release generation on github still more things to re-check on this step, it will become clearer once i get it up to speed to run on my local repos end to end * netdata/packaging/ci: Fix wrong path usage * netdata/packaging/ci: Update webhook URL * netdata/packaging/ci: Update stage config as proposed by the fix from travis team * netdata/packaging/ci: Another round of restructuring, after 1-1 comparison with old version Briefly: 1) Publish for release stage will only run when there is a TAG defined (To be tested though, but thats the idea). The code previously had the releaser.sh script that did exactly that, if tagger script failed, it would stop process. Now we have distributed the processing in clear chunks and we pick up the process based on shared info that travis or the environment provides 2) Rename <<Create release (only on special commit msg)>> to a more appropriate name, we are not really creating release there its more complicated 3) Move the labeler execution to the packaging step, since publishing now not happening all the time. Previously we hacked this by exit 0 with the tagger and allow the labeler (which was the second part of the execution) to run. 4) before_install in the publishing stage is wrong. We only need this as before_deploy before deploy execution. That is where we use it, really. 5) Rename changelog_and_tag_release.sh to generate_changelog_and_tag_release.sh, just to be clearer. No problem with slightly longer file names 6) Do not let generate_changelog_and_tag_release.sh execute if GIT_TAG is not defined, just exit quietly with a message and a success code 7) Remove the tagger from the generate_changelog_and_tag_release.sh, its confusing to shove it in there. Pull it out and let it run on a clear step from the pipeline but within the same machine to make sure it shares environment with generate_changelog_and_tag_release.sh (So it has to be within the same job) This is it mostly, keep in mind that the nightly and release parts are not tested yet. After this commit, there will be more code that will allow me to execute all the flows on my pipeline and fully test and adjust the workflows as needed * netdata/packaging/ci: Add more notifications, adjust the existing ones too * netdata/packaging/ci: Add safeguards for beta, do a rename too 1) rename generate_changelog.sh -> generate_changelog_for_nightlies.sh, just to provide clarity on the usage. Note: we will simplify this later, this was a workaround that raised a bug on release after refactoring nightlies. We will have to clean this up properly once we merge the new pipeline flow 2) Add checks based on TRAVIS_REPO_SLUG to identify when we running on personal repo and abort mission critical actions like deploying or generating changelogs Note: The deployment step for now will simply fail, due to the lack of the necessary encryption keys on the environment. At a later iteration, we should have beta buckets on GCS and also beta repositories on Docker to properly test our publishing capability on our beta stack * netdata/packaging/ci: Refactor README.md -- still half way there, will resume when i am done with testing * netdata/packaging/ci: At this stage make the scripts harder to run, add checks to bypass tested execution. * netdata/packaging/ci: Seems that adding the install tag within the stage, kind of overriding the outer install pull out install items from packaging to the main install, no point to isolate two packages, just install all required all the time * netdata/packaging/ci: tag obviously cant work at that step, moving it down to the jobs and i need to recheck if the syntax is applicable for travis too * netdata/packaging/ci: So the jobs in a list dont share the environment, lets see if we do it otherwise. also remove the tag check, not work as expected * netdata/packaging/ci: Add another way to check the RC condition * netdata/packaging/ci: experimental - check conditions on message * netdata/packaging/ci: Introduce an alternate master branch for the workflow testing 1. I created pipeline-redesign-master that will be the master branch for my use cases 2. Modified travis to trigger all production workflows on that branch, so that i can experiment freely with test commits to trigger process and verify execution flow * remove spaces * netdata/packaging/ci: comment out * netdata/packaging/ci: wording nit * netdata/packaging/ci: adjust the condition properly * netdata/packaging/ci: test condition for triggering on specific keywords * netdata/packaging/ci: try different syntax * netdata/packaging/ci: remove depth, causes problems as we dont get all required objects from repo (adds --depth 50 in clone) * Missing equal sign in regex? * netdata/packaging/ci: Update tagger script 1) More verbose messages 2) Update comments copyrights etc 3) Rename release_candidate to set_tag_release_candidate 4) Add curlies on the variables, also change the way we check for path * netdata/packaging/ci: reinstate git depth * netdata/packaging/ci: Remove git depth from packaging too * netdata/packaging/ci: RTFM - was using wrong syntax for slack @here approach * netdata/packaging/ci: Using install tag at the stage layer, overrides the global one for some reason. fix that * netdata/packaging/ci: Dont use the pattern matching from travis, do it outside with a script * netdata/packaging/ci: Do it the other way, script execution doesnt work there * netdata/packaging/ci: Attempt to fix conditional on release, remove the faulty assumptions as provided by travis team * netdata/packaging/ci: reintroduce the tag checking * netdata/packaging/ci: you need to bring tag checking at the stage, as it is calculated during packaging step * netdata/packaging/ci: you cant escape that one for the moment, merge them and we will rework it later again * netdata/packaging/ci: dont forget git depth * netdata/packaging/ci: bring back the dist checking * netdata/packaging/ci: fix syntax * netdata/packaging/ci: no depth * netdata/packaging/ci: Introduce make dist validation tasks: Install from dist file and verify service is starting up * netdata/packaging/ci: Add a couple more information for troubleshooting * netdata/packaging/ci: Add extra information prior to executing crucial operations * netdata/packaging/ci: Make rm error soft here * netdata/packaging/ci: This is not needed here * netdata/packaging/ci: Prepare for beta stage deployment process - introduce a beta provider - adjust key decryption process, different keys for beta, different for prod - Enter conditions to deploy beta only on beta process and production only on production process - Use travis variables, to allow multiple users have beta stages for deployment, if they want * netdata/packaging/ci: add one more message * netdata/packaging/ci: Bring back naming to the actual master branch * netdata/packaging/ci: rename beta to dev * netdata/packaging/ci: Make permanent restrictions for some of the stricts, so that we don't mess things up * netdata/packaging/ci: Update README that was missed and also make sure git depth false is at the places needed * netdata/packaging/ci: fix codacy complaints

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

bilderbuchi mentioned this issue Aug 20, 2015

Build jobs running that aren't specified in the matrix #1228

Closed

jonnangle mentioned this issue Aug 23, 2015

Defining the build matrix only with matrix.include leads to an additional unwanted build grosser/wwtd#28

Closed

BanzaiMan added the matrix expansion label Sep 1, 2015

theuni mentioned this issue Sep 2, 2015

travis: fix for travis generating an extra build bitcoin/bitcoin#6617

Merged

dexX7 mentioned this issue Sep 2, 2015

Fix/remove empty build target of Travis CI OmniLayer/omnicore#218

Merged

patricklodder mentioned this issue Sep 5, 2015

fix for travis generating an extra build dogecoin/dogecoin#1277

Merged

BanzaiMan self-assigned this Sep 30, 2015

erenon added a commit to erenon/double_ended that referenced this issue Dec 13, 2015

travis: workaround travis bug travis-ci/travis-ci#4681

0f7c649

BanzaiMan mentioned this issue Dec 19, 2015

Multi Os build with trusty enables precise build #5337

Closed

jpsim added a commit to jpsim/realm-cocoa that referenced this issue Jan 6, 2016

try a different workaround for travis-ci/travis-ci#4681

df8d418

BanzaiMan mentioned this issue Feb 20, 2016

Testing on both Precise and Trusty #5666

Closed

BanzaiMan changed the title ~~Defining the build matrix only with matrix.include leads to an additional, unwanted build~~ Defining the build matrix only with matrix.include leads to an additional, unwanted job Feb 20, 2016

brunocodutra added a commit to brunocodutra/metal that referenced this issue Feb 25, 2016

[ci] workaround travis-ci/travis-ci#4681

f9c9748

jfirebaugh mentioned this issue Apr 6, 2016

Bug in triggered build travis matrix mapbox/mason#147

Closed

brunocodutra added a commit to brunocodutra/metal that referenced this issue Apr 12, 2016

[ci] workaround travis-ci/travis-ci#4681

ba8304b

nebgnahz added a commit to damellis/ESP that referenced this issue May 31, 2016

Needs to explicitly set excludes.

4d8afe9

See details here: travis-ci/travis-ci#4681

BanzaiMan mentioned this issue Jun 23, 2016

Redundant "no language set" jobs #6213

Closed

othiym23 added a commit to npm/npm that referenced this issue Jun 24, 2016

ci: travis-ci/travis-ci#4681

2155613

robertknight mentioned this issue Jul 13, 2016

Add ESLint configuration and use it to replace JSCS / JSHint hypothesis/client#27

Merged

BanzaiMan mentioned this issue Aug 3, 2016

OS X: One more job than matrix include row · no custom environment variables #6404

Closed

jdhealy added a commit to jdhealy/PrettyColors that referenced this issue Aug 3, 2016

Tests: Travis: Exclude default job which lacks our included environme…

bb2c5c2

…nt variables. Workaround travis-ci/travis-ci#4681

jdhealy added a commit to jdhealy/PrettyColors that referenced this issue Aug 3, 2016

Tests: Travis: Exclude default job which lacks our included environme…

04de021

…nt variables. Workaround travis-ci/travis-ci#4681

elmodaddyb mentioned this issue Dec 20, 2017

Fix #205 - Add Gerbera UI tests to Travis CI gerbera/gerbera#208

Merged

BanzaiMan mentioned this issue Jan 29, 2018

Build stages result is not correct when combining build stages with matrix expansion that expands to one job only #9157

Closed

BanzaiMan mentioned this issue May 14, 2018

Conditional Job overwrites default script #9614

Closed

zoechi mentioned this issue Jul 5, 2018

matrix.include leads to additional unwanted job #9843

Closed

cclauss mentioned this issue Oct 18, 2018

Travis CI: Simplify the build matrix Netflix/security_monkey#1125

Closed

vgvassilev added a commit to vgvassilev/clad that referenced this issue Mar 25, 2020

Remove the exclude clause

80231c0

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

vgvassilev added a commit to vgvassilev/clad that referenced this issue Mar 25, 2020

Remove the exclude clause

2e12a55

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

vgvassilev added a commit to vgvassilev/root that referenced this issue Apr 1, 2020

[travis][skip-ci]Remove the exclude clause

6bc9912

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

vgvassilev mentioned this issue Apr 1, 2020

[travis][skip-ci]Remove the exclude clause root-project/root#5277

Merged

vgvassilev added a commit to vgvassilev/root that referenced this issue Apr 2, 2020

[travis]Remove the exclude clause

f5781dc

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

vgvassilev added a commit to root-project/root that referenced this issue Apr 2, 2020

[travis]Remove the exclude clause

1e4a782

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

szepeviktor mentioned this issue Sep 22, 2020

Separate CI jobs GraphQLAPI/graphql-api-for-wp#55

Merged

osschar pushed a commit to osschar/root that referenced this issue Dec 21, 2020

[travis]Remove the exclude clause

1236604

The exclude clause was a workaround travis-ci/travis-ci#4681 but it causes travis-ci/travis-ci#8536 in recent travis.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Defining the build matrix only with matrix.include leads to an additional, unwanted job #4681

Defining the build matrix only with matrix.include leads to an additional, unwanted job #4681

bilderbuchi commented Aug 20, 2015

BanzaiMan commented Aug 24, 2015

bilderbuchi commented Aug 24, 2015

o11c commented Sep 30, 2015

ldionne commented Nov 7, 2015

wavexx commented Feb 5, 2016

BanzaiMan commented Feb 20, 2016

wavexx commented Feb 20, 2016 via email

Defining the build matrix only with matrix.include leads to an additional, unwanted job #4681

Defining the build matrix only with matrix.include leads to an additional, unwanted job #4681

Comments

bilderbuchi commented Aug 20, 2015

BanzaiMan commented Aug 24, 2015

bilderbuchi commented Aug 24, 2015

o11c commented Sep 30, 2015

ldionne commented Nov 7, 2015

wavexx commented Feb 5, 2016

BanzaiMan commented Feb 20, 2016

wavexx commented Feb 20, 2016 via email