New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add job label/display name #5898

Closed
Snaipe opened this Issue Apr 13, 2016 · 30 comments

Comments

Projects
None yet
@Snaipe

Snaipe commented Apr 13, 2016

The build summary screen can become a hassle to read for projects with a complex build matrix.

It would be great if we could add names/labels to specific jobs in the matrix.

The visual result could be something like this:

travis-job-label-request

And the .travis.yml config, like so:

matrix:
  include:
  - name: Build with foo as bar

Having no display name would simply blank out the relevant section.

This, if it were to be implemented:

  • Would allow developers to see at a glance which builds failed an succeeded -- currently, I have to count the index of the job or read its log to see that a job cross compiles for architecture X or Y.
  • Would allow developers to sum up the contents of the environment field to the right, which can become easily cluttered when you have 3+ variables defined in your matrix.
  • Would allow developers to make a distinction between otherwise visually identical builds (e.g. builds that only differ by a secure variable, like builds testing on two specialized remotes machines with credentials, or builds that have the same environment variables but use different apt packages).

A possible workaround could be putting a _ environment variable -- with the name as a value -- at the start of the environment list (not tested), but this is hardly convenient and is visually inconsistent.

@jimdebeer

This comment has been minimized.

jimdebeer commented Jul 24, 2016

yes please!

@BanzaiMan

This comment has been minimized.

Member

BanzaiMan commented Jul 25, 2016

This attribute, if implemented, is meaningful only for those jobs that you explicitly include in matrix.include (because others are created as a cartesian product of matrix dimensions). In such cases, though, you can explicitly define just as easily an environment variable which will show up in the current display.

matrix:
  include:
    - env: NAME="Build with foo as far" VAR1=VAL1 VAR2=VAL2
@jimdebeer

This comment has been minimized.

jimdebeer commented Jul 26, 2016

Its only a cosmetic feature --- since env vars don't really look nice

see the difference
image
looks pretty!

image
looks ugly!

@jayvdb

This comment has been minimized.

jayvdb commented Jul 26, 2016

While this may help individual teams, I think it detracts from the travis community, as it means there is no longer a common logic for understanding a build matrix screen. If it is done, I hope it is togglable, so someone can switch back to raw envvars.

Arch is already a separate column shown with an icon. Maybe add a variant icon for 32 vs 64 is needed?

IMO debug vs release should be a builtin travis feature (are there any supported languages where tbis doesnt make sense?) and shown in another column using an icon. Not sure about deploy.

@Snaipe

This comment has been minimized.

Snaipe commented Jul 26, 2016

@jayvdb

While this may help individual teams, I think it detracts from the travis community, as it means there is no longer a common logic for understanding a build matrix screen. If it is done, I hope it is togglable, so someone can switch back to raw envvars.

It should of course be completely optional. This is only relevant for non-trivial, hand-crafted build matrices (typically on multi-arch & cross-platforms projects w/ assembly)

Arch is already a separate column shown with an icon. Maybe add a variant icon for 32 vs 64 is needed?

Arch does not suffice by itself because of cross-compilation.

Here's my build matrix: https://travis-ci.org/Snaipe/Mimick/builds/141036082

Can you tell which builds are i386, x86_64, AArch64, ARM? Not being able to name jobs for what they simply are is a hassle. I am reluctant to add more architectures to the CI config precisely because of this.

IMO debug vs release should be a builtin travis feature (are there any supported languages where tbis doesnt make sense?) and shown in another column using an icon. Not sure about deploy.

I don't see working in practice, as this is highly dependent on both the language and the build system used. How would you make that work for projects using autotools for instance, where the debug mode is a user-defined flag?

@jayvdb

This comment has been minimized.

jayvdb commented Jul 26, 2016

As you are cross compiling, that should be in the Compiler column of Travis build matrix. Currently it says "gcc", but that isnt really true. The same is roughly true of multi-arch platforms.

Wrt your config specifically, wouldnt it be enough if Travis build env column said "CONFIGURATION=Debug, *i386-env", in which case it is a minor change to the UI to better show complex yaml. Then you could defined x86_64-linux-debug and it would appear as "*x86_64-linux-debug" in the env column -- at least then it isnt an arbitrary name; it is a defined name elsewhere in the yaml.

As for debug, on most languages, it is quite easily set with an env var.
For C/C++, it is CFLAGS and CXXFLAGS respectively, and the usual approach is to use -g or -O2. autoconf understands those. With direct clang invocation, I think llvm-config needs to be used, but I am rusty on that. The build script could still fiddle with those defaults env vars for project specific preferences wrt debug/release, but it isnt hard to have sane defaults which tease out bugs. (and then it would be possibly to add support for fancy config, such as building only debug versions by default for branches (so it is quicker), and enable release builds on master.

@swarren

This comment has been minimized.

swarren commented Oct 25, 2016

We're hitting this issue in U-Boot too, and the proposed feature is exactly what I was searching for. It seems like a good generic fix for the issue to me.

Naming jobs as an implicit side-effect of the environment variables that are set seems like a hack to me, especially if I wanted to just run different scripts instead rather than setting environment variables to influence the operation of a different script.

For reference, the WAR I applied to U-Boot's Travis config:
https://patchwork.ozlabs.org/patch/686225/
[U-Boot,1/2] travis-ci: set env vars to name jobs

@mdeguzis

This comment has been minimized.

mdeguzis commented Oct 31, 2016

What I really am looking for, is the ability to have labels for each env outputted as a clean label. Instead of just "Build stats: passing" I want sub labels for each job/env that runs. This would be ideal for folks who run multiple passes. My use case here is multi-distro builds via Docker images.

@joshk

This comment has been minimized.

Member

joshk commented Oct 31, 2016

I like the idea of supporting this, and would be interested in adding it when explicitly listing jobs and their configurations.

I'll be adding this to our Roadmap for Q1 of next year.

@JohannesBeranek

This comment has been minimized.

JohannesBeranek commented May 31, 2017

@joshk did anything happen on this?

Another reason an env var doesn't always work: As soon as env vars are different, the cache can't be shared anymore (which IMHO is a different issue - would be good to ignore certain things for cache dependency or define the exact cache dependency)

Especially with the new Travis Stages feature this has become more relevant again.

@simonvanderveldt

This comment has been minimized.

simonvanderveldt commented Jul 14, 2017

Another reason an env var doesn't always work: As soon as env vars are different, the cache can't be shared anymore (which IMHO is a different issue - would be good to ignore certain things for cache dependency or define the exact cache dependency)

This has just tripped me up. Eventually I noticed the mention of it in the cache docs, but I totally didn't expect env vars to have an impact on which cache is shared.

This means I'll have to remove those env vars and I only used them to identify what each job in the matrix was doing, so now I'm back to a list that doesn't really tell me anything about what each job does :(

@joshk Are there any updates on this? We're past Q1 2017 already ;)

simonvanderveldt added a commit to gentoo-audio/audio-overlay that referenced this issue Jul 14, 2017

Travis: Remove marker env vars
Because they cause separate caches, which means the cache is useless for our use cases.
See travis-ci/travis-ci#5898 for related issue

simonvanderveldt added a commit to gentoo-audio/audio-overlay that referenced this issue Jul 14, 2017

Travis: Remove marker env vars (#66)
Because they cause separate caches, which means the cache is useless for our use cases.
See travis-ci/travis-ci#5898 for related issue

sormuras added a commit to junit-team/junit5-samples that referenced this issue Dec 3, 2017

@gnzlbg

This comment has been minimized.

gnzlbg commented Jan 18, 2018

It's 2018. Any updates on this? @BanzaiMan ?

@shs96c

This comment has been minimized.

shs96c commented Jan 30, 2018

Adding to this: without job labels, this build provides very little actionable information unless someone delves into the outputs themselves:

https://travis-ci.org/SeleniumHQ/selenium/builds/331260746

We don't want to use environment variables since the cache warming stage can be expensive and is slow, but without them it's impossible to tell which of these are the JS tests, and which ones are the Java tests (or which browser is being used)

@gr2m

This comment has been minimized.

gr2m commented Feb 2, 2018

As a workaround, we do this

script: npm run $COMMAND
jobs:
  include:
    - stage: test
      env: COMMAND=test:unit
    - env: COMMAND=test:integration

instead of this

jobs:
  include:
    - stage: test
      script: npm run test:unit
    - script: npm run test:integration

Example: https://travis-ci.org/octokit/fixtures/builds/336419815

@azu

This comment has been minimized.

azu commented Feb 19, 2018

I have similar problem on Travis CI API.
https://developer.travis-ci.com/resource/job#Job
API does not includes env variable.
I want to get custom name or env value from job id.

@pauldraper

This comment has been minimized.

pauldraper commented May 5, 2018

@gr2m the problem with that as @JohannesBeranek, @simonvanderveldt, and @shs96c have stated, the env variable "workaround" affects caching behavior.

In short, there is no possible way to mark a job as "test", "lint", or "integration" without structurally altering how Travis treats the build.

@wking

This comment has been minimized.

wking commented May 13, 2018

The Travis GitHub App Checks support includes a table with columns for matrix variables. For an example with a Rust-version column, see here. I don't know if changing script will create a new column or not.

@plata

This comment has been minimized.

plata commented Jun 24, 2018

I'm currently using build stages to mitigate this issue, i.e. I put single jobs into their own build stage to name them. Of course, this is not really a solution because it means that the jobs cannot be executed in parallel anymore. However, the alternative is that people do not know what's wrong with their pull requests, so I'm willing to accept that issue for now.

@jakul

This comment has been minimized.

jakul commented Jul 30, 2018

Travis now supports naming jobs within a build matrix - https://docs.travis-ci.com/user/customizing-the-build/#naming-jobs-within-matrices

@ojab

This comment has been minimized.

ojab commented Jul 30, 2018

Unfortunately job names is not displayed on github checks page, so it's not possible to see what exactly fails without going to travis example

Is there plans to add it to notes?

@plata

This comment has been minimized.

plata commented Jul 30, 2018

+1 for @ojab. Just noted the same.

nomeata added a commit to antalsz/hs-to-coq that referenced this issue Jul 30, 2018

@swarren

This comment has been minimized.

swarren commented Jul 30, 2018

Another issue: The list of builds includes the job name, which is great:
https://travis-ci.org/swarren/u-boot/builds/409925753
However, as soon as you click a link to an individual build which has a name:
https://travis-ci.org/swarren/u-boot/jobs/409925755
... the job name isn't shown anywhere, so you have to remember which link you clicked. The job name probably should be displayed prominently in the header, e.g. along with commit/compare/branch details (just below the ToT commit description text), but certainly somewhere.

@ianha

This comment has been minimized.

ianha commented Aug 14, 2018

Would really love to see this supported in a robust way. It would help save our team time debugging and make us more productive. Supporting this would really help all the teams that use Travis.

thbar added a commit to thbar/truffleruby that referenced this issue Sep 5, 2018

Show how to display "truffleruby" instead of "system" on TravisCI
This leverages the [new job naming feature](travis-ci/travis-ci#5898 (comment)) available in TravisCI.
@joshk

This comment has been minimized.

Member

joshk commented Sep 6, 2018

@swarren we are working on this right now, and hope to have something ready in the next week or two.

In general, since Job naming is supported, can we close this issue?

@swarren

This comment has been minimized.

swarren commented Sep 6, 2018

Yes, I'm fine with closing this if you open another issue to cover displaying the job name within the job page itself. Note that I'm not the original issue requester here though.

havenwood added a commit to havenwood/truffleruby that referenced this issue Sep 7, 2018

Show how to display "truffleruby" instead of "system" on TravisCI
This leverages the [new job naming feature](travis-ci/travis-ci#5898 (comment)) available in TravisCI.
@Snaipe

This comment has been minimized.

Snaipe commented Sep 7, 2018

I'm fine with this being closed. Thank you for addressing this. :)

@Snaipe Snaipe closed this Sep 7, 2018

@joshk

This comment has been minimized.

Member

joshk commented Sep 7, 2018

@swarren we should have this deployed in the next week or two max

@joshk

This comment has been minimized.

Member

joshk commented Sep 7, 2018

@Snaipe thanks for closing the issue :)

@drdavella

This comment has been minimized.

drdavella commented Oct 26, 2018

This is awesome, thanks for the work! Is there any chance you'd be able to continue displaying the environment variables in addition to the name? It seems like in most cases there's plenty of room.

@giampaolo

This comment has been minimized.

giampaolo commented Oct 26, 2018

How do we use this? What's the directive name? "name"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment