[JENKINS-64347] Bounds exceeds available space exception after successful build-step

[jenkins-infra-bot]

As previously reported in JENKINS-6423, I have an exception after my build successfully completes. It only started happening this weekend, only on the Mac worker, right after I updated to Big Sur... It doesn't happen to every build, just seemingly random across jobs. It also doesn't always happen after the whole job is finished, it can happen after X out of Y steps.

20:53:05.897 FATAL: Bounds exceeds available space : size=1048576, offset=1048577
20:53:05.898 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to worker-mobile-01
20:53:05.898 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
20:53:05.898 		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
20:53:05.898 		at hudson.remoting.Channel.call(Channel.java:1001)
20:53:05.898 		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1147)
20:53:05.898 		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
20:53:05.898 		at hudson.model.Run.execute(Run.java:1894)
20:53:05.898 		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
20:53:05.898 		at hudson.model.ResourceController.execute(ResourceController.java:97)
20:53:05.898 		at hudson.model.Executor.run(Executor.java:428)
20:53:05.898 java.lang.IndexOutOfBoundsException: Bounds exceeds available space : size=1048576, offset=1048577
20:53:05.898 	at com.sun.jna.Memory.boundsCheck(Memory.java:221)
20:53:05.898 	at com.sun.jna.Memory.getByte(Memory.java:443)
20:53:05.898 	at hudson.util.ProcessTree$Darwin$DarwinProcess$1StringArrayMemory.readString(ProcessTree.java:1739)
20:53:05.898 	at hudson.util.ProcessTree$Darwin$DarwinProcess.parse(ProcessTree.java:1811)
20:53:05.898 	at hudson.util.ProcessTree$Darwin$DarwinProcess.getEnvironmentVariables(ProcessTree.java:1688)
20:53:05.898 	at hudson.util.ProcessTree$OSProcess.hasMatchingEnvVars(ProcessTree.java:339)
20:53:05.898 	at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:733)
20:53:05.898 	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1164)
20:53:05.898 	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1155)
20:53:05.898 	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
20:53:05.898 	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
20:53:05.898 	at hudson.remoting.Request$2.run(Request.java:375)
20:53:05.898 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
20:53:05.898 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
20:53:05.898 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
20:53:05.898 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
20:53:05.898 	at java.lang.Thread.run(Thread.java:748)

---

Originally reported by malice00, imported from: Bounds exceeds available space exception after successful build-step

• assignee: kenji
• status: Closed
• priority: Minor
• component(s): core
• label(s): 2.289.2-fixed
• resolution: Fixed
• resolved: 2021-06-09T07:26:53+00:00
• votes: 3
• watchers: 11
• imported: 2025-11-24

Raw content of original issue

As previously reported in JENKINS-6423, I have an exception after my build successfully completes. It only started happening this weekend, only on the Mac worker, right after I updated to Big Sur... It doesn't happen to every build, just seemingly random across jobs. It also doesn't always happen after the whole job is finished, it can happen after X out of Y steps.

20:53:05.897 FATAL: Bounds exceeds available space : size=1048576, offset=1048577
20:53:05.898 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to worker-mobile-01
20:53:05.898 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
20:53:05.898 		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
20:53:05.898 		at hudson.remoting.Channel.call(Channel.java:1001)
20:53:05.898 		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1147)
20:53:05.898 		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
20:53:05.898 		at hudson.model.Run.execute(Run.java:1894)
20:53:05.898 		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
20:53:05.898 		at hudson.model.ResourceController.execute(ResourceController.java:97)
20:53:05.898 		at hudson.model.Executor.run(Executor.java:428)
20:53:05.898 java.lang.IndexOutOfBoundsException: Bounds exceeds available space : size=1048576, offset=1048577
20:53:05.898 	at com.sun.jna.Memory.boundsCheck(Memory.java:221)
20:53:05.898 	at com.sun.jna.Memory.getByte(Memory.java:443)
20:53:05.898 	at hudson.util.ProcessTree$Darwin$DarwinProcess$1StringArrayMemory.readString(ProcessTree.java:1739)
20:53:05.898 	at hudson.util.ProcessTree$Darwin$DarwinProcess.parse(ProcessTree.java:1811)
20:53:05.898 	at hudson.util.ProcessTree$Darwin$DarwinProcess.getEnvironmentVariables(ProcessTree.java:1688)
20:53:05.898 	at hudson.util.ProcessTree$OSProcess.hasMatchingEnvVars(ProcessTree.java:339)
20:53:05.898 	at hudson.util.ProcessTree$Unix.killAll(ProcessTree.java:733)
20:53:05.898 	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1164)
20:53:05.898 	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1155)
20:53:05.898 	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
20:53:05.898 	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
20:53:05.898 	at hudson.remoting.Request$2.run(Request.java:375)
20:53:05.898 	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
20:53:05.898 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
20:53:05.898 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
20:53:05.898 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
20:53:05.898 	at java.lang.Thread.run(Thread.java:748)

environment

Master on RHEL7.6<br/>
Worker on MacOS Big Sur<br/>
Jenkins 2.249.3

1 attachment

• image-2021-05-28-11-10-01-235.png

  

[jenkins-infra-bot]

• Remote link associated with this issue:

  • jenkins#​5548

[jenkins-infra-bot]

timja:

• Original comment link
• Raw content of original comment:

  cc jthompson oleg_nenashev not sure what info you need here.
  

cc jthompson oleg_nenashev not sure what info you need here.

[jenkins-infra-bot]

jthompson:

• Original comment link
• Raw content of original comment:

  I really don't have much to suggest. I'm not terribly familiar with the ProcessTree class. This particular part of the code was last changed 12 years ago.
  
  My first guess with things like this is always that something changed with the environment, the system, network or something. It could be something changed with Big Sur, but I don't know that we can really be confident with that yet.
  
  It looks like it could be a memory allocation or consumption issue, but I'm not familiar enough with the ProcessTree code to know. Maybe memory is used differently on Big Sur than before – there are certainly instances of such behavior on significant OS upgrades.
  
  With the inconsistent nature of this problem, it's going to require good troubleshooting to isolate the problem. I don't have any specific troubleshooting ideas, though. If there's any way to make it reproducible that would help.
  

I really don't have much to suggest. I'm not terribly familiar with the ProcessTree class. This particular part of the code was last changed 12 years ago.

My first guess with things like this is always that something changed with the environment, the system, network or something. It could be something changed with Big Sur, but I don't know that we can really be confident with that yet.

It looks like it could be a memory allocation or consumption issue, but I'm not familiar enough with the ProcessTree code to know. Maybe memory is used differently on Big Sur than before – there are certainly instances of such behavior on significant OS upgrades.

With the inconsistent nature of this problem, it's going to require good troubleshooting to isolate the problem. I don't have any specific troubleshooting ideas, though. If there's any way to make it reproducible that would help.

[jenkins-infra-bot]

malice00:

• Original comment link
• Raw content of original comment:

  I will keep an eye on it and see if I can find a pattern – maybe it's concurrent builds or some other thing I haven't noticed yet...
  
  Anyway, if anybody has some ideas for troubleshooting, let me know and I will try!
  

I will keep an eye on it and see if I can find a pattern – maybe it's concurrent builds or some other thing I haven't noticed yet...

Anyway, if anybody has some ideas for troubleshooting, let me know and I will try!

[jenkins-infra-bot]

raihaan:

• Original comment link
• Raw content of original comment:

  My understanding of the code from a not too deep read is that, a combination of process arguments and environment variables exceed a megabyte in size.
  
  I'm not sure if this behaviour is normal or if my understanding is correct but perhaps this will narrow our search a little.
  

My understanding of the code from a not too deep read is that, a combination of process arguments and environment variables exceed a megabyte in size.

I'm not sure if this behaviour is normal or if my understanding is correct but perhaps this will narrow our search a little.

[jenkins-infra-bot]

malice00:

• Original comment link
• Raw content of original comment:

  a combination of process arguments and environment variables exceed a megabyte in size
  Wouldn't that mean that it would be easy to reproduce? I mean, when a build fails with this error, I can just do a rebuild and most of the time it then finishes correctly – with the same arguments and environment. Or am I missing something here?
  
  Update on the issue itself: I rebooted the machine shortly after my last comment and all was quiet for about 4 days. Now it has started again, unfortunately with the same random behavior... Last night even my smallest job on that machine (runs every night, no parameters, just a simple 'cocoapods update') failed with this error. This is an old-school matrix-build, but I don't think that makes a real difference.
  
  Any suggestions as to how I might figure out the size of the arguments and environment on the jobs/runs?
  

a combination of process arguments and environment variables exceed a megabyte in size

Wouldn't that mean that it would be easy to reproduce? I mean, when a build fails with this error, I can just do a rebuild and most of the time it then finishes correctly – with the same arguments and environment. Or am I missing something here?

Update on the issue itself: I rebooted the machine shortly after my last comment and all was quiet for about 4 days. Now it has started again, unfortunately with the same random behavior... Last night even my smallest job on that machine (runs every night, no parameters, just a simple 'cocoapods update') failed with this error. This is an old-school matrix-build, but I don't think that makes a real difference.

Any suggestions as to how I might figure out the size of the arguments and environment on the jobs/runs?

[jenkins-infra-bot]

raihaan:

• Original comment link
• Raw content of original comment:

  AFAICT you can't change KERN_ARGMAX.
  
  I agree it should be easy to reproduce if that was in fact the case. It could be a more subtle bug.
  

AFAICT you can't change KERN_ARGMAX.

I agree it should be easy to reproduce if that was in fact the case. It could be a more subtle bug.

[jenkins-infra-bot]

malice00:

• Original comment link
• Raw content of original comment:

  This has been annoying us more and more, so I went Googling and found another old issue: JENKINS-9634
  
  I also found this: https://stackoverflow.com/questions/6147815/jenkins-throwing-indexoutofboundsexception-at-end-of-build-on-mac, which suggests disabling the ProcessTreeKiller (which apparently has been renamed since – read that somewhere else, but don't have a link atm). Does anybody know if it would be possible to disable this on only the node running Mac or do I need to do it for the whole instance of Jenkins?
  
  Since I am not sure if disabling the PTK has any (negative) effects, I don't really want to do it on the whole instance...
  

This has been annoying us more and more, so I went Googling and found another old issue: JENKINS-9634

I also found this: https://stackoverflow.com/questions/6147815/jenkins-throwing-indexoutofboundsexception-at-end-of-build-on-mac, which suggests disabling the ProcessTreeKiller (which apparently has been renamed since – read that somewhere else, but don't have a link atm). Does anybody know if it would be possible to disable this on only the node running Mac or do I need to do it for the whole instance of Jenkins?

Since I am not sure if disabling the PTK has any (negative) effects, I don't really want to do it on the whole instance...

[jenkins-infra-bot]

timja:

• Original comment link
• Raw content of original comment:

  It might be worth trying the latest weekly release, we’ve updated JNA to the latest version recently
  

It might be worth trying the latest weekly release, we’ve updated JNA to the latest version recently

[jenkins-infra-bot]

malice00:

• Original comment link
• Raw content of original comment:

  I'll give that a try next week – Do I just need the core or also some plugins?
  

I'll give that a try next week – Do I just need the core or also some plugins?

[jenkins-infra-bot]

timja:

• Original comment link
• Raw content of original comment:

  A number of plugins need updating too
  

A number of plugins need updating too

[jenkins-infra-bot]

nittanymountain:

• Original comment link
• Raw content of original comment:

  I added this when launch jenkins, now only seeing this issue for tasks running on nodes.
      -Dhudson.util.ProcessTreeKiller.disable=true
  
  	
  could reproduce it with a dump task, say, just echo a string in the build script.  even the node is on the same machine (the server)
  	
  and this happens for both jenkins 1 (lte) and jenkins2
  
  
  
  -David
  

I added this when launch jenkins, now only seeing this issue for tasks running on nodes.
   -Dhudson.util.ProcessTreeKiller.disable=true

• could reproduce it with a dump task, say, just echo a string in the build script.  even the node is on the same machine (the server)
• and this happens for both jenkins 1 (lte) and jenkins2

-David

[jenkins-infra-bot]

kenji:

• Original comment link
• Raw content of original comment:

  The
  
  -Dhudson.util.ProcessTreeKiller.disable=true
  
   should be an option of the launch agent on the node side
  No errors since one week since option added to the macOS "LaunchAgent" command in a shell script :
  
  java -Dhudson.util.ProcessTreeKiller.disable=true -jar agent.jar -jnlpUrl https://REDACTED/slave-agent.jnlp -secret REDACTED -noCertificateCheck
  
  

The

-Dhudson.util.ProcessTreeKiller.disable=true

 should be an option of the launch agent on the node side

No errors since one week since option added to the macOS "LaunchAgent" command in a shell script :

java -Dhudson.util.ProcessTreeKiller.disable=true -jar agent.jar -jnlpUrl https://REDACTED/slave-agent.jnlp -secret REDACTED -noCertificateCheck

[jenkins-infra-bot]

nittanymountain:

• Original comment link
• Raw content of original comment:

  thanks kenji,  it works when launch agent with that option
  (actually need to have it for both, launching server and each node...)
  

thanks kenji,  it works when launch agent with that option
(actually need to have it for both, launching server and each node...)

[jenkins-infra-bot]

damon:

• Original comment link
• Raw content of original comment:

  kenji How to add?
  
  

kenji How to add?