Skip to content

[JENKINS-41124] Can't get a human readable job name anymore #637

Closed

Description

@jenkins-infra-bot

Since the SCM 2.0 release, I can't find a way, in a pipeline, to get a human-readable job & branch name.

For example, if I had a BitBucket team called 'Foo', a repository called 'Bar' and a branch 'feature/KEY-Baz', it used to be possible to get a nearly human readable branch - env.BRANCH_NAME would be 'Foo/Bar%2FKEY-Baz', which was easy enough to escape to a human readable form.

Now, however, it's a shortened form, with numbers in it - something like 'Foo/Bar.qxyz123/feature-KEY.abcd12345ghij-Baz'. And there appears to be no way to get the human readable version. This has broken all of my build notification templates.

Here's my 'build started' notification, for example:

hipchatSend color: 'YELLOW', credentialId: 'HipChat', message: "'${env.BUILD_URL}'>${env.JOB_NAME.replace('%2F', '/')} ${env.BUILD_NUMBER} Started", notify: true, room: 'Build Server Messages', sendAs: 'Jenkins', server: '', v2enabled: false

Originally reported by twasink, imported from: Can't get a human readable job name anymore
  • assignee: stephenconnolly
  • status: Closed
  • priority: Critical
  • component(s): branch-api-plugin
  • resolution: Fixed
  • resolved: 2017-01-23T21:09:59+00:00
  • votes: 0
  • watchers: 7
  • imported: 20251211-141027
Raw content of original issue

Since the SCM 2.0 release, I can't find a way, in a pipeline, to get a human-readable job & branch name.

For example, if I had a BitBucket team called 'Foo', a repository called 'Bar' and a branch 'feature/KEY-Baz', it used to be possible to get a nearly human readable branch - env.BRANCH_NAME would be 'Foo/Bar%2FKEY-Baz', which was easy enough to escape to a human readable form.

Now, however, it's a shortened form, with numbers in it - something like 'Foo/Bar.qxyz123/feature-KEY.abcd12345ghij-Baz'. And there appears to be no way to get the human readable version. This has broken all of my build notification templates.

Here's my 'build started' notification, for example:

hipchatSend color: 'YELLOW', credentialId: 'HipChat', message: "<a href='${env.BUILD_URL}'>${env.JOB_NAME.replace('%2F', '/')} ${env.BUILD_NUMBER}</a> Started", notify: true, room: 'Build Server Messages', sendAs: 'Jenkins', server: '', v2enabled: false
  • environment: Jenkins 2.32.1, Bitbucket Branch Source Plugin 2.0.0

Activity

  1. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
  2. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:

    Probably similar to JENKINS-41104">JENKINS-41104

  3. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:
    • Original comment link
    • Raw content of original comment:

      This is also breaking my use of the S3BucketPublisher - It used to publish to S3 with a path prefix of (using the above) 'Foo/Bar/feature%2FBaz'.

      It now publishes with 'Foo/Bar.qxyz123/feature-KEY.abcd12345ghij-Baz'. I could probably live with the suffix on the project name (though that sucks, it is at least consistent), but the mangling of the branch name is horrible, as it's non-deterministic.

      At this point, I'm about to rollback to my backup and stick with the v1.0 version; notification templates were painful, but this is a problem that completely breaks my ops infrastructure.

    This is also breaking my use of the S3BucketPublisher - It used to publish to S3 with a path prefix of (using the above) 'Foo/Bar/feature%2FBaz'.

    It now publishes with 'Foo/Bar.qxyz123/feature-KEY.abcd12345ghij-Baz'. I could probably live with the suffix on the project name (though that sucks, it is at least consistent), but the mangling of the branch name is horrible, as it's non-deterministic.

    At this point, I'm about to rollback to my backup and stick with the v1.0 version; notification templates were painful, but this is a problem that completely breaks my ops infrastructure.

  4. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:
    • Original comment link
    • Raw content of original comment:

      I know the mangling of the paths is there to ensure uniqueness - but it's causing problems for me like crazy. An option to disable it would help a lot.

    I know the mangling of the paths is there to ensure uniqueness - but it's causing problems for me like crazy. An option to disable it would help a lot.

  5. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    stephenconnolly:
    • Original comment link
    • Raw content of original comment:

      env.BRANCH_NAME should now be the plain unmangled name

    env.BRANCH_NAME should now be the plain unmangled name

  6. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    jamesdumay:
    • Original comment link
    • Raw content of original comment:

      This should be BLOCKER. We can't go about breaking peoples URLs.

    This should be BLOCKER. We can't go about breaking peoples URLs.

  7. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:
    • Original comment link
    • Raw content of original comment:

      What about the job name? Is there a way to get an unmangled form of that?

    What about the job name? Is there a way to get an unmangled form of that?

  8. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:
    • Original comment link
    • Raw content of original comment:

      The S3BucketPublisher, BTW, uses run.getParent().getFullName(); to determine the prefix. If I can get an unmangled job name, I can do my own S3 uploads, but if I want to keep using this plugin, I need a way to make the parent's full name to be unmangled.

    The S3BucketPublisher, BTW, uses run.getParent().getFullName(); to determine the prefix. If I can get an unmangled job name, I can do my own S3 uploads, but if I want to keep using this plugin, I need a way to make the parent's full name to be unmangled.

  9. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    recampbell:

    The impact of this issue is that previously users expected to be able to construct the expected URL on their own, but now it is generated using a non-transparent algorithm.

    Mic says he's had 10 people in the last 24 hours mentioned that they are surprised by the new URL scheme.

    For example, here is a new URL:

    https://ci.blueocean.io/blue/organizations/jenkins/blueocean/detail/feature-J.hunjes6l7b7p.om-github/5/pipeline/

    Which is not love-able.

    There is also external tooling which depends on this structure.

  10. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    stephenconnolly:
    • Original comment link
    • Raw content of original comment:

      the job display name should be the unmangled name (unless the user has customized, but you cannot customize display name for multibranch)

      We can expose the display name as an environment variable too

      The Job name cannot stay as is currently.

      % is not a valid character on some file systems.

      .. is a valid branch name on some source control systems.

      Even seemingly "safe" filenames like aux can cause issues for users.

      There is a very limited set of characters that are safe for filesystems: A-Za-z0-9_.- is all there is to play with

      Furthermore, the name length causes lots of issues. So we need to ensure that the branch name is not "too long"

      The only way out of that with minimal risk of collisions is a deterministic hashing algorithm... which becomes ugly... or part name part hash with some characters used to disambiguate to prevent the smaller hash from causing issues.

      The current algorithm was chosen to reduce the risk of collision to less than 1 in 33 million, which is about as low as we can go before things get ugly.

      If somebody else has a wonderful magical scheme for mapping names from the entire unicode space in a predictable pattern into the character set A-Za-z0-9_.- that also makes care to ensure silly branch names like .. are not given the name .. and dangerous branch names like AUX, CON, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9, LPT1, LPT2, LPT3, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, LPT9, PRN, NULL or their lower case equivalents are mapped into something sensible, and doesn't have risk of collisions (e.g. when JENKINS-36240 gets merged and then we can potentially start building "trusted" fork branches such that the branch name is now controlled by the fork)... oh and can we not actually change the job name from the branch name... and can we keep the job names to something short... well I'm all ears... but I think you will find it an impossible and thankless task.

      What we need to do is expose the human readable bits correctly and ensure that plugins do correct things rather than blinding assuming things.

      We probably also need to apply name mangling in core to all jobs... but that is a different story.

      Blue Ocean could use getItemByBranch() to resolve branches if blue ocean wants to handle the case of branch names like .. other than that, jenkins core will still resolve the old urls

    the job display name should be the unmangled name (unless the user has customized, but you cannot customize display name for multibranch)

    We can expose the display name as an environment variable too

    The Job name cannot stay as is currently.

    % is not a valid character on some file systems.

    .. is a valid branch name on some source control systems.

    Even seemingly "safe" filenames like aux can cause issues for users.

    There is a very limited set of characters that are safe for filesystems: A-Za-z0-9_.- is all there is to play with

    Furthermore, the name length causes lots of issues. So we need to ensure that the branch name is not "too long"

    The only way out of that with minimal risk of collisions is a deterministic hashing algorithm... which becomes ugly... or part name part hash with some characters used to disambiguate to prevent the smaller hash from causing issues.

    The current algorithm was chosen to reduce the risk of collision to less than 1 in 33 million, which is about as low as we can go before things get ugly.

    If somebody else has a wonderful magical scheme for mapping names from the entire unicode space in a predictable pattern into the character set A-Za-z0-9_.- that also makes care to ensure silly branch names like .. are not given the name .. and dangerous branch names like AUX, CON, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9, LPT1, LPT2, LPT3, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, LPT9, PRN, NULL or their lower case equivalents are mapped into something sensible, and doesn't have risk of collisions (e.g. when JENKINS-36240">JENKINS-36240 gets merged and then we can potentially start building "trusted" fork branches such that the branch name is now controlled by the fork)... oh and can we not actually change the job name from the branch name... and can we keep the job names to something short... well I'm all ears... but I think you will find it an impossible and thankless task.

    What we need to do is expose the human readable bits correctly and ensure that plugins do correct things rather than blinding assuming things.

    We probably also need to apply name mangling in core to all jobs... but that is a different story.

    Blue Ocean could use getItemByBranch() to resolve branches if blue ocean wants to handle the case of branch names like .. other than that, jenkins core will still resolve the old urls

  11. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    stephenconnolly:
    • Original comment link
    • Raw content of original comment:

      if people want we can increase the length of the hash but reduce the bits encoded by making those hashes "pronounceable"... but that limits the bit space that we have to play with. Base32 is what we use currently.

    if people want we can increase the length of the hash but reduce the bits encoded by making those hashes "pronounceable"... but that limits the bit space that we have to play with. Base32 is what we use currently.

  12. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:
    • Original comment link
    • Raw content of original comment:

      Is there a reason we can't have an option (disabled by default) to leave the job & branch name the way the user specified it? Because a lot of those problems aren't problems for a lot of users, a lot of the time, and can be worked around by selection of repository & branch name. Whilst mangling the job name can not be.

      I understand that in environments like CloudBees that this will be a problem - but in a standard self-hosted development environment, we would be extremely unlikely to encounter these issues.

      [Also, I believe you're unnecessarily conflating the Job name with the workspace name - the Job name isn't part of any filespace other than the web URL (and HTML escaping works fine for that)]

      I've rolled back now, so I can't easily check, but from my recollection the env.JOB_DISPLAY_NAME didn't have the unmangled version - I did a dump of all of the environment properties, and not one was unmangled except for the BRANCH_NAME property. It's possible that the JOB_DISPLAY_NAME wasn't set yet, as I did the dump at the start of the job; I'll double-check that tonight.

    Is there a reason we can't have an option (disabled by default) to leave the job & branch name the way the user specified it? Because a lot of those problems aren't problems for a lot of users, a lot of the time, and can be worked around by selection of repository & branch name. Whilst mangling the job name can not be.

    I understand that in environments like CloudBees that this will be a problem - but in a standard self-hosted development environment, we would be extremely unlikely to encounter these issues.

    [Also, I believe you're unnecessarily conflating the Job name with the workspace name - the Job name isn't part of any filespace other than the web URL (and HTML escaping works fine for that)]

    I've rolled back now, so I can't easily check, but from my recollection the env.JOB_DISPLAY_NAME didn't have the unmangled version - I did a dump of all of the environment properties, and not one was unmangled except for the BRANCH_NAME property. It's possible that the JOB_DISPLAY_NAME wasn't set yet, as I did the dump at the start of the job; I'll double-check that tonight.

  13. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    michaelneale:
    • Original comment link
    • Raw content of original comment:

      twasink it looks like this is going to be temporarily moved out of the main update center until this (and one other issue) are addressed and tested. For now it seems you are in an ok place after you restored?

    twasink it looks like this is going to be temporarily moved out of the main update center until this (and one other issue) are addressed and tested. For now it seems you are in an ok place after you restored?

  14. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    stephenconnolly:
    • Original comment link
    • Raw content of original comment:

      Check your JENKINS HOME:

      JENKINS_HOME/jobs/${orgFolderName}/jobs/${mangledMultibranchProjectName}/branches/${mangledBranchName}

      It's not the workspace name as that is already mangled (with an unnecessarily long hash that causes issues for windows builds but that is a different story)

      In the above path the only user controlled names is orgFolderName

      The other two names come from the SCMNaviagtor and the SCMSource respectively and we have no control over what names they give us back.

      We already saw one issue with names containing Korean and Chinese characters on unix filesystems

    Check your JENKINS HOME:

    JENKINS_HOME/jobs/${orgFolderName}/jobs/${mangledMultibranchProjectName}/branches/${mangledBranchName}

    It's not the workspace name as that is already mangled (with an unnecessarily long hash that causes issues for windows builds but that is a different story)

    In the above path the only user controlled names is orgFolderName

    The other two names come from the SCMNaviagtor and the SCMSource respectively and we have no control over what names they give us back.

    We already saw one issue with names containing Korean and Chinese characters on unix filesystems

  15. jenkins-infra-bot commented on Jan 17, 2017

    @jenkins-infra-bot
    Author
    twasink:
    • Original comment link
    • Raw content of original comment:

      michaelneale Yes, after restoring from the backup (which I took as advised in the upgrade notes), I'm working fine again.

    michaelneale Yes, after restoring from the backup (which I took as advised in the upgrade notes), I'm working fine again.

  16. 33 remaining items

  17. jenkins-infra-bot commented on Jan 22, 2017

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Stephen Connolly
    Path:
    pom.xml
    src/main/java/jenkins/branch/MultiBranchProject.java
    src/main/java/jenkins/branch/MultiBranchProjectDescriptor.java
    src/main/java/jenkins/branch/OrganizationFolder.java
    http://jenkins-ci.org/commit/branch-api-plugin/ba727fac801d6d47743277928447ca3c347a1185
    Log:
    JENKINS-41124">JENKINS-41124 Integrates the fixed from cloudbees-folder

  18. jenkins-infra-bot commented on Jan 22, 2017

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Stephen Connolly
    Path:
    src/main/java/jenkins/branch/MultiBranchProjectFactory.java
    src/main/java/jenkins/branch/NameEncoder.java
    src/test/java/integration/BrandingTest.java
    src/test/java/integration/MigrationTest.java
    src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java
    http://jenkins-ci.org/commit/branch-api-plugin/32c5aba1102a2a8928a1decc17c1170bec0e2845
    Log:
    JENKINS-41124">JENKINS-41124 Address code review comments

  19. jenkins-infra-bot commented on Jan 22, 2017

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Stephen Connolly
    Path:
    src/main/java/jenkins/branch/MultiBranchProjectFactory.java
    http://jenkins-ci.org/commit/branch-api-plugin/091997f77adfa37704f83584f153776a2a239cd7
    Log:
    JENKINS-41124">JENKINS-41124 Typo in javadoc

  20. jenkins-infra-bot commented on Jan 22, 2017

    @jenkins-infra-bot
    Author
    scm_issue_link:
    • Original comment link
    • Raw content of original comment:

      Code changed in jenkins User: Stephen Connolly Path: pom.xml src/main/java/jenkins/branch/Branch.java src/main/java/jenkins/branch/MultiBranchProject.java src/main/java/jenkins/branch/MultiBranchProjectDescriptor.java src/main/java/jenkins/branch/MultiBranchProjectFactory.java src/main/java/jenkins/branch/NameEncoder.java src/main/java/jenkins/branch/OrganizationFolder.java src/test/java/integration/EventsTest.java src/test/java/integration/MigrationTest.java src/test/java/integration/harness/BasicMultiBranchProjectFactory.java src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java src/test/resources/integration/MigrationTest/nameMangling_2.zip src/test/resources/integration/MigrationTest/nameMangling_2_folder_reload.zip src/test/resources/integration/MigrationTest/nameMangling_2_full_reload.zip src/test/resources/integration/MigrationTest/nameMangling_folder_reload.zip src/test/resources/integration/MigrationTest/nameMangling_full_reload.zip http://jenkins-ci.org/commit/branch-api-plugin/05030c1c0a5a0811c562785d209698c22aee5a86 Log: Merge pull request #72 from stephenc/jenkins-41124

      JENKINS-41124 Move name mangling support responsibility to AbstractFolder

      Compare: 228cd1a...05030c1

    Code changed in jenkins
    User: Stephen Connolly
    Path:
    pom.xml
    src/main/java/jenkins/branch/Branch.java
    src/main/java/jenkins/branch/MultiBranchProject.java
    src/main/java/jenkins/branch/MultiBranchProjectDescriptor.java
    src/main/java/jenkins/branch/MultiBranchProjectFactory.java
    src/main/java/jenkins/branch/NameEncoder.java
    src/main/java/jenkins/branch/OrganizationFolder.java
    src/test/java/integration/EventsTest.java
    src/test/java/integration/MigrationTest.java
    src/test/java/integration/harness/BasicMultiBranchProjectFactory.java
    src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java
    src/test/resources/integration/MigrationTest/nameMangling_2.zip
    src/test/resources/integration/MigrationTest/nameMangling_2_folder_reload.zip
    src/test/resources/integration/MigrationTest/nameMangling_2_full_reload.zip
    src/test/resources/integration/MigrationTest/nameMangling_folder_reload.zip
    src/test/resources/integration/MigrationTest/nameMangling_full_reload.zip
    http://jenkins-ci.org/commit/branch-api-plugin/05030c1c0a5a0811c562785d209698c22aee5a86
    Log:
    Merge pull request #72 from stephenc/jenkins-41124

    JENKINS-41124">JENKINS-41124 Move name mangling support responsibility to AbstractFolder

    Compare: 228cd1a...05030c1

  21. jenkins-infra-bot commented on Jan 22, 2017

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Stephen Connolly
    Path:
    pom.xml
    src/main/java/jenkins/branch/MultiBranchProject.java
    http://jenkins-ci.org/commit/branch-api-plugin/93e7fa149036ef63842924b5a3514209747efdc7
    Log:
    JENKINS-41124">JENKINS-41124 Post-merge tidy up and pick up cloudbees-folder 5.17-beta-1

  22. jenkins-infra-bot commented on Feb 2, 2017

    @jenkins-infra-bot
    Author
    stephenconnolly:
    • Original comment link
    • Raw content of original comment:

      scm-api-2.0.2 git-2.6.4 or git-3.0.4 branch-api-2.0.2 bitbucket-branch-source-2.0.2 github-branch-source-2.0.1 cloudbees-folder-5.17 workflow-multibranch-2.11

    scm-api-2.0.2
    git-2.6.4 or git-3.0.4
    branch-api-2.0.2
    bitbucket-branch-source-2.0.2
    github-branch-source-2.0.1
    cloudbees-folder-5.17
    workflow-multibranch-2.11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions