[JENKINS-65829] Workspace Cleanup does not delete @libs directories created by libraries

[jenkins-infra-bot]

In Jenkins Pipelines one can use shared libraries. With them a additional directory is created:

Usual-Workspace: /var/lib/jenkins/workspace/job-id/
Additional SharedLibrary-Folder: /var/lib/jenkins/workspace/job-id@​tmp

Seems as if there the shared library sources are stored in case one want s to rerun the build.

In order to completely wipe out old build data, a option to delete this @​tmp folder could be introduced.

---

Originally reported by xbw1220, imported from: Workspace Cleanup does not delete @libs directories created by libraries

• assignee: allan_burdajewicz
• status: Closed
• priority: Minor
• component(s): clearcase-plugin, core, pipeline-groovy-lib-plugin
• resolution: Fixed
• resolved: 2024-04-16T14:09:33+00:00
• votes: 7
• watchers: 13
• imported: 2025-11-24

Raw content of original issue

In Jenkins Pipelines one can use shared libraries. With them a additional directory is created:

Usual-Workspace: /var/lib/jenkins/workspace/job-id/
 Additional SharedLibrary-Folder: /var/lib/jenkins/workspace/job-id@tmp

Seems as if there the shared library sources are stored in case one want s to rerun the build.

In order to completely wipe out old build data, a option to delete this @tmp folder could be introduced.

environment

2.249.2 lts <br/>
Workspace  Cleanup Plugin-0.38

2 attachments

• job-master-1.png

  

• job1-1.png

  

[jenkins-infra-bot]

• Remote link associated with this issue:

  • core #​9083

[jenkins-infra-bot]

cpottiers:

• Original comment link
• Raw content of original comment:

  same issue here, i have just cleanup 30Go of remaining @libs directories ... 
  

same issue here, i have just cleanup 30Go of remaining @​libs directories ...

[jenkins-infra-bot]

orrimp:

• Original comment link
• Raw content of original comment:

  Same here. Are there any workarounds or other plugins to use here instead? 
  

Same here. Are there any workarounds or other plugins to use here instead? 

[jenkins-infra-bot]

arnabcse28:

• Original comment link
• Raw content of original comment:

  Similar issue is still observed that's creating unnecessary space issue. 
  
  But a couple of differences in my observation:
  
  	
  Shared Library is being cloned to a folder with @lib suffix, NOT @tmp suffix...  e.g. /var/lib/jenkins/workspace/job-id@libs
  	
  Even though the Jenkins controller (master) is never used for running a Build, these folders are getting created on Jenkins Controller's job workspace.
  
  
  
  Is someone taking a look at this issue?
  

Similar issue is still observed that's creating unnecessary space issue.

But a couple of differences in my observation:

• Shared Library is being cloned to a folder with @​lib suffix, NOT @​tmp suffix... e.g. /var/lib/jenkins/workspace/job-id@​libs
• Even though the Jenkins controller (master) is never used for running a Build, these folders are getting created on Jenkins Controller's job workspace.

Is someone taking a look at this issue?

[jenkins-infra-bot]

jglick:

• Original comment link
• Raw content of original comment:

  JENKINS-70869 allows you to never create the …@libs directory to begin with.
  

JENKINS-70869 allows you to never create the …@​libs directory to begin with.

[jenkins-infra-bot]

arnabcse28:

• Original comment link
• Raw content of original comment:

  hi jglick, thanks for sharing the info but the issue you mentioned (https://issues.jenkins.io/browse/JENKINS-70869) talks about an option is marked as resolved..  
  Is this the default behavior of pipeline-groovy-lib plugin now that it doesn't create the @libs folders? 
  If that's the case can you please share the Jenkins version that has the fix (We are using Jenkins 2.332.4 with "Pipeline: Groovy LibrariesVersion 629.vb_5627b_ee2104" and it still creates these on controller nodes)?
  If not, then could you please add some guidance on how to achieve this?
  

hi jglick, thanks for sharing the info but the issue you mentioned (https://issues.jenkins.io/browse/JENKINS-70869) talks about an option is marked as resolved..
Is this the default behavior of pipeline-groovy-lib plugin now that it doesn't create the @​libs folders?
If that's the case can you please share the Jenkins version that has the fix (We are using Jenkins 2.332.4 with "Pipeline: Groovy LibrariesVersion 629.vb_5627b_ee2104" and it still creates these on controller nodes)?
If not, then could you please add some guidance on how to achieve this?

[jenkins-infra-bot]

arnabcse28:

• Original comment link
• Raw content of original comment:

  Okay..  I see the new version released 3 days back has some relevant information..  
  https://plugins.jenkins.io/pipeline-groovy-lib/releases/
  
  Adding this to the other ticket as well for easy access 
  

Okay.. I see the new version released 3 days back has some relevant information..
https://plugins.jenkins.io/pipeline-groovy-lib/releases/

Adding this to the other ticket as well for easy access

[jenkins-infra-bot]

allan_burdajewicz:

• Original comment link
• Raw content of original comment:

  Actually the WorkspaceClleanerThread is supposed to cleanup the suffixed directories since the fix for https://issues.jenkins.io/browse/JENKINS-41805 but this is not happening for @libs suffixed workspaces on the controller for builds that are executed on non-controller executors. cc timja
  
  In those cases, there is only a @libs suffixed workspace but not the original and we don'tpass the condition https://github.com/jenkinsci/jenkins/blob/jenkins-2.447/core/src/main/java/hudson/model/WorkspaceCleanupThread.java#L107.
  
  In such case, the WorkspaceLocator finds out that there is a workspace on the controller FS, but it looks for existence of $WORKSPACE and not $WORKSPACE@libs. 
  
  Here is an example of a job built by test agent test-agent that use a shared library. The library is cloned under /var/jenkins_home/workspace/pipelines/pipelineFromScriptWithLib@libs but there is no /var/jenkins_home/workspace/pipelines/pipelineFromScriptWithLib:
  
  
  Mar. 05, 2024 1:12:03 PM FINE jenkins.branch.WorkspaceLocatorImpl locate
  ignoring non-branch project org.jenkinsci.plugins.workflow.job.WorkflowJob@1744e0b[pipelines/pipelineFromScriptWithLib] on hudson.model.Hudson@5787278d
  Mar. 05, 2024 1:12:03 PM FINE hudson.model.WorkspaceCleanupThread shouldBeDeleted
  Directory /var/jenkins_home/workspace/pipelines/pipelineFromScriptWithLib does not exist
  
  
  Here is another example with a branch jobs, the library is cloned under /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2@libs but there is no /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2:
  
  Mar. 05, 2024 1:12:03 PM FINER jenkins.branch.WorkspaceLocatorImpl locate
  index already lists /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2 for org.jenkinsci.plugins.workflow.job.WorkflowJob@21ed6f70[organizations/organizationGithubScm/ci-catalog-poc-pipelines/withLib] on hudson.model.Hudson@5787278d
  Mar. 05, 2024 1:12:03 PM FINE hudson.model.WorkspaceCleanupThread shouldBeDeleted
  Directory /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2 does not exist
  
  
  To reproduce:
  
  	
  start Jenkins with -Dhudson.model.WorkspaceCleanupThread.retainForDays=0
  	
  Set up a remote agent
  	
  Create a pipeline that clones a library:
  
  
  @Library("testLibs") _
  node ('test') {
      echo "test"
  }
  
  
  
  	
  Build the job
  	
  Run hudson.model.AsyncPeriodicWork.all().get(WorkspaceCleanupThread.class).run() in the script console to execute the thread run
  
  

Actually the WorkspaceClleanerThread is supposed to cleanup the suffixed directories since the fix for https://issues.jenkins.io/browse/JENKINS-41805 but this is not happening for @​libs suffixed workspaces on the controller for builds that are executed on non-controller executors. cc timja

In those cases, there is only a @​libs suffixed workspace but not the original and we don'tpass the condition https://github.com/jenkinsci/jenkins/blob/jenkins-2.447/core/src/main/java/hudson/model/WorkspaceCleanupThread.java#L107.

In such case, the WorkspaceLocator finds out that there is a workspace on the controller FS, but it looks for existence of $WORKSPACE and not $WORKSPACE@​libs.

Here is an example of a job built by test agent test-agent that use a shared library. The library is cloned under /var/jenkins_home/workspace/pipelines/pipelineFromScriptWithLib@​libs but there is no /var/jenkins_home/workspace/pipelines/pipelineFromScriptWithLib:

Mar. 05, 2024 1:12:03 PM FINE jenkins.branch.WorkspaceLocatorImpl locate
ignoring non-branch project org.jenkinsci.plugins.workflow.job.WorkflowJob@​1744e0b[pipelines/pipelineFromScriptWithLib] on hudson.model.Hudson@​5787278d
Mar. 05, 2024 1:12:03 PM FINE hudson.model.WorkspaceCleanupThread shouldBeDeleted
Directory /var/jenkins_home/workspace/pipelines/pipelineFromScriptWithLib does not exist

Here is another example with a branch jobs, the library is cloned under /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2@​libs but there is no /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2:

Mar. 05, 2024 1:12:03 PM FINER jenkins.branch.WorkspaceLocatorImpl locate
index already lists /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2 for org.jenkinsci.plugins.workflow.job.WorkflowJob@​21ed6f70[organizations/organizationGithubScm/ci-catalog-poc-pipelines/withLib] on hudson.model.Hudson@​5787278d
Mar. 05, 2024 1:12:03 PM FINE hudson.model.WorkspaceCleanupThread shouldBeDeleted
Directory /var/jenkins_home/workspace/_catalog-poc-pipelines_withLib_2 does not exist

To reproduce:

• start Jenkins with -Dhudson.model.WorkspaceCleanupThread.retainForDays=0
• Set up a remote agent
• Create a pipeline that clones a library:

@​Library("testLibs") _
node ('test') {
    echo "test"
}

• Build the job
• Run hudson.model.AsyncPeriodicWork.all().get(WorkspaceCleanupThread.class).run() in the script console to execute the thread run