Skip to content

[JENKINS-62985] Jenkins leaks a file handle when viewing a build log that contains an improperly encoded ConsoleNote (base64 error) #11610

Closed
Bug
Closed
[JENKINS-62985] Jenkins leaks a file handle when viewing a build log that contains an improperly encoded ConsoleNote (base64 error)#11610
Bug
Labels
component:coreimported-jira-issuepriority:minorresolution:fixed

Description

@jenkins-infra-bot
jenkins-infra-bot
opened on Jul 6, 2020

Repro:
1. Create a pipeline build job
2. Run a build
3. Replace the log on (ex: /var/lib/jenkins/jobs/ThisIsATest/builds/55/log) with the attached bad.log file bad.log
4. Browse to the build in Jenkins, view the log as plain text (https://jenkins/job/ThisIsATest/55/consoleText).
5. Observe the attached error in the jenkins log. error.log
6. Observe that Jenkins has leaked a file descriptor to the log on disk.

For example:

root@​jenkins-test-local:/var/lib/jenkins/jobs/ThisIsATest/builds/55# lsof -p 10007|grep log
java    10007 jenkins  mem       REG8,1  1213761  408229 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-servicecatalog-1.11.791.jar
java    10007 jenkins  mem       REG8,1   563715  408224 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-logs-1.11.791.jar
java    10007 jenkins  mem       REG8,1   138625  408266 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-marketplacecatalog-1.11.791.jar
java    10007 jenkins    1w      REG8,1 24123228  395200 /var/log/tomcat8/catalina.out
java    10007 jenkins    2w      REG8,1 24123228  395200 /var/log/tomcat8/catalina.out
java    10007 jenkins    3w      REG8,1  1047507  392476 /var/lib/jenkins/gc/gc-2020-07-06_16-15-31.log.0.current
java    10007 jenkins   17w      REG8,1 10462062  408102 /var/log/tomcat8/catalina.2020-07-06.log
java    10007 jenkins   18w      REG8,1   348255  394397 /var/log/tomcat8/localhost.2020-07-06.log
java    10007 jenkins   59w      REG8,1  7486004  399682 /var/log/tomcat8/access.log
java    10007 jenkins   66r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   68r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   69r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   70r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   71r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   72r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   73r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   74r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   75r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   76r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   77r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   78r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   79r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   80r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   82r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   91r      REG       0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins  361r      REG8,1   138625  408266 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-marketplacecatalog-1.11.791.jar
java    10007 jenkins  417r      REG8,1   563715  408224 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-logs-1.11.791.jar
java    10007 jenkins  441r      REG8,1  1213761  408229 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-servicecatalog-1.11.791.jar
Originally reported by mikeryan_tan, imported from: Jenkins leaks a file handle when viewing a build log that contains an improperly encoded ConsoleNote (base64 error)
  • assignee: raihaan
  • status: Resolved
  • priority: Minor
  • component(s): core
  • label(s): 2.263.1-fixed
  • resolution: Fixed
  • resolved: 2020-11-03T14:25:37+00:00
  • votes: 0
  • watchers: 3
  • imported: 2025-11-24
Raw content of original issue 
Repro:
1. Create a pipeline build job
2. Run a build
3. Replace the log on (ex: /var/lib/jenkins/jobs/ThisIsATest/builds/55/log) with the attached bad.log file  bad.log 
4. Browse to the build in Jenkins, view the log as plain text (https://jenkins/job/ThisIsATest/55/consoleText).
5. Observe the attached error in the jenkins log.  error.log 
6. Observe that Jenkins has leaked a file descriptor to the log on disk.



For example:




root@jenkins-test-local:/var/lib/jenkins/jobs/ThisIsATest/builds/55# lsof -p 10007|grep log
java    10007 jenkins  mem       REG                8,1  1213761  408229 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-servicecatalog-1.11.791.jar
java    10007 jenkins  mem       REG                8,1   563715  408224 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-logs-1.11.791.jar
java    10007 jenkins  mem       REG                8,1   138625  408266 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-marketplacecatalog-1.11.791.jar
java    10007 jenkins    1w      REG                8,1 24123228  395200 /var/log/tomcat8/catalina.out
java    10007 jenkins    2w      REG                8,1 24123228  395200 /var/log/tomcat8/catalina.out
java    10007 jenkins    3w      REG                8,1  1047507  392476 /var/lib/jenkins/gc/gc-2020-07-06_16-15-31.log.0.current
java    10007 jenkins   17w      REG                8,1 10462062  408102 /var/log/tomcat8/catalina.2020-07-06.log
java    10007 jenkins   18w      REG                8,1   348255  394397 /var/log/tomcat8/localhost.2020-07-06.log
java    10007 jenkins   59w      REG                8,1  7486004  399682 /var/log/tomcat8/access.log
java    10007 jenkins   66r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   68r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   69r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   70r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   71r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   72r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   73r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   74r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   75r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   76r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   77r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   78r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   79r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   80r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   82r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins   91r      REG               0,45      152  269033 /mnt/jenkins/jobs/ThisIsATest/builds/55/log (127.0.0.1:/exports/jenkins)
java    10007 jenkins  361r      REG                8,1   138625  408266 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-marketplacecatalog-1.11.791.jar
java    10007 jenkins  417r      REG                8,1   563715  408224 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-logs-1.11.791.jar
java    10007 jenkins  441r      REG                8,1  1213761  408229 /var/lib/jenkins/plugins/aws-java-sdk/WEB-INF/lib/aws-java-sdk-servicecatalog-1.11.791.jar
environment 
Jenkins 2.222.3, 2.235.1<br/>
java 1.8.0 201, java 1.8.0.251<br/>
tomcat8 8.0.32-1ubuntu1.11
2 attachments

Activity

  1. jenkins-infra-bot commented on Jul 6, 2020

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 6, 2020
    Author

  2. jenkins-infra-bot commented on Jul 6, 2020

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 6, 2020
    Author
    mikeryan_tan:
    • Original comment link
    • Raw content of original comment: 
      The file handle leak is the result of a bug that produces corrupt log lines.

    The file handle leak is the result of a bug that produces corrupt log lines.

  3. jenkins-infra-bot commented on Jul 6, 2020

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 6, 2020
    Author
    mikeryan_tan:

    This file handle leak is especially problematic in my environment, as prevents us from properly rotating logs/discarding builds. Our jenkins build logs sit on an NFS share, and we end up with `.nfs123412341234` files that cannot be deleted ("Device or resource busy" errors). Pretty much: https://support.cloudbees.com/hc/en-us/articles/115001371588-Device-or-resource-busy-error

  4. jenkins-infra-bot commented on Oct 29, 2020

    @jenkins-infra-bot
    jenkins-infra-bot
    on Oct 29, 2020
    Author
    raihaan:
    • Original comment link
    • Raw content of original comment: 
      Confirm I can reproduce this with a pipeline job and changing its logfile

    Confirm I can reproduce this with a pipeline job and changing its logfile

  5. jenkins-infra-bot commented on Oct 29, 2020

    @jenkins-infra-bot
    jenkins-infra-bot
    on Oct 29, 2020
    Author
    raihaan:
    • Original comment link
    • Raw content of original comment: 
      After testing with the linked stapler PR i can no longer reproduce the issue

    After testing with the linked stapler PR i can no longer reproduce the issue

  6. jenkins-infra-bot
    closed this as completedon Nov 3, 2020
  7. jenkins-infra-bot
    added
    bugFor changelog: Minor bug. Will be listed after features
    imported-jira-issue
    component:core
    priority:minor
    resolution:fixed
    and removed
    bugFor changelog: Minor bug. Will be listed after features
    on Nov 24, 2025
  8. jenkins-infra-bot
    added the
    Bug
    issue type on Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:coreimported-jira-issuepriority:minorresolution:fixed

    Type

    Bug

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions