Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in provisioning: NullPointerException #33

Closed
dasJ opened this issue Jun 5, 2016 · 9 comments
Closed

Error in provisioning: NullPointerException #33

dasJ opened this issue Jun 5, 2016 · 9 comments
Labels
bug

Comments

@dasJ
Copy link

dasJ commented Jun 5, 2016

I'm trying to run a Docker container with SSH on localhost.

When running a job which matches the labels and Jenkins tries to start the container, I get the following errors in the log:

INFO: Trying to run container for myregistry.com/mycontainer
Jun 05, 2016 3:07:38 PM com.github.kostyasha.yad.DockerCloud lambda$provision$0
SEVERE: Error in provisioning; template='com.github.kostyasha.yad.DockerSlaveTemplate@1243d31a[dockerContainerLifecycle=com.github.kostyasha.yad.DockerContainerLifecycle@f3d2d3fb,id=39f91140-a61b-4a6f-ac97-6db0f8990922,labelString=generic container privileged docker,launcher=com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher@5a03e1bc,remoteFsMapping=,remoteFs=/home/jenkins,maxCapacity=10,mode=EXCLUSIVE,retentionStrategy=com.github.kostyasha.yad.strategy.DockerOnceRetentionStrategy@27f,numExecutors=1,nodeProperties=[]]' for cloud='Localhost'
java.lang.NullPointerException
        at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.getHostAndPort(DockerComputerSSHLauncher.java:122)
        at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.waitUp(DockerComputerSSHLauncher.java:81)
        at com.github.kostyasha.yad.DockerCloud.provisionWithWait(DockerCloud.java:216)
        at com.github.kostyasha.yad.DockerCloud.lambda$provision$0(DockerCloud.java:117)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Jun 05, 2016 3:07:43 PM hudson.slaves.NodeProvisioner$2 run
WARNING: Provisioned agent myregistry.com/mycontainer failed to launch
java.lang.NullPointerException
        at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.getHostAndPort(DockerComputerSSHLauncher.java:122)
        at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.waitUp(DockerComputerSSHLauncher.java:81)
        at com.github.kostyasha.yad.DockerCloud.provisionWithWait(DockerCloud.java:216)
        at com.github.kostyasha.yad.DockerCloud.lambda$provision$0(DockerCloud.java:117)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

No container is started and the job is not run. Instead, yad is trying to launch the container over and over again.

Output of docker daemon -l debug:

DEBU[1019] Calling GET /containers/json?all=0&size=0
DEBU[1019] Calling GET /containers/json?all=0&size=0
DEBU[1019] Calling GET /images/json?all=0
DEBU[1019] Calling POST /containers/create
DEBU[1019] form data: {"AttachStderr":false,"AttachStdin":false,"AttachStdout":false,"BlkioWeight":null,"Cmd":["bash","-c","/usr/sbin/sshd -D -p 22"],"CpuPeriod":null,"CpuShares":0,"Cpuset":null,"CpusetMems":null,"Domainname":"","Entrypoint":null,"Env":null,"ExposedPorts":{},"HostConfig":{"Binds":null,"CapAdd":null,"CapDrop":null,"ContainerIDFile":null,"CpuShares":0,"Devices":null,"Dns":null,"DnsSearch":null,"ExtraHosts":null,"Links":null,"LogConfig":null,"LxcConf":null,"Memory":0,"MemorySwap":0,"NetworkMode":null,"PidMode":null,"PortBindings":{"22/tcp":[{"HostIp":"0.0.0.0","HostPort":""}]},"Privileged":true,"PublishAllPorts":false,"ReadonlyRootfs":false,"RestartPolicy":null,"Ulimits":null,"VolumesFrom":null},"Hostname":"","Image":"myregistry.com/myimage","Labels":{"com.github.kostyasha.yad.DockerCloud":"Localhost","com.github.kostyasha.yad.DockerSlave":"39f91140-a61b-4a6f-ac97-6db0f8990922"},"MacAddress":null,"Memory":0,"MemorySwap":0,"NetworkDisabled":false,"OomKillDisable":null,"OpenStdin":false,"PortSpecs":["22/tcp"],"StdinOnce":false,"Tty":false,"User":"","Volumes":{},"WorkingDir":"","capAdd":null,"capDrop":null,"containerIDFile":null,"name":null,"readonlyRootfs":false,"ulimits":null}
DEBU[1019] Creating dest directory: /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386-init
DEBU[1019] Calling TarUntar(/var/lib/docker/vfs/dir/98e4860d0d1c01b8c25bc6752effd160539f126256b45979a578abbc9c08ab84, /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386-init)
DEBU[1019] TarUntar(/var/lib/docker/vfs/dir/98e4860d0d1c01b8c25bc6752effd160539f126256b45979a578abbc9c08ab84 /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386-init)
DEBU[1022] Creating dest directory: /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386
DEBU[1022] Calling TarUntar(/var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386-init, /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386)
DEBU[1022] TarUntar(/var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386-init /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386)
DEBU[1024] container mounted via layerStore: /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386
DEBU[1024] Calling POST /containers/4bc716406aa2645e165fd7172df78a56cd648bdfc53d3840b2b3632c92460e07/start
DEBU[1024] form data: {}
DEBU[1024] container mounted via layerStore: /var/lib/docker/vfs/dir/1757d5695980ac86044cfd837f0b9fe3d3b9496370965022829b51e78830a386
DEBU[1024] Assigning addresses for endpoint silly_jang's interface on network bridge
DEBU[1024] RequestAddress(LocalDefault/172.18.0.0/16, <nil>, map[])
DEBU[1024] Assigning addresses for endpoint silly_jang's interface on network bridge
DEBU[1024] Programming external connectivity on endpoint silly_jang (52f7bebf0b245dc1d54d895ce9b6357985417446a4943332484214ddbb8c4a1c)
DEBU[1024] sandbox set key processing took 37.238424ms for container 4bc716406aa2645e165fd7172df78a56cd648bdfc53d3840b2b3632c92460e07
DEBU[1024] received containerd event: &types.Event{Type:"start-container", Id:"4bc716406aa2645e165fd7172df78a56cd648bdfc53d3840b2b3632c92460e07", Status:0x0, Pid:"", Timestamp:0x57542640}
DEBU[1024] event unhandled: type:"start-container" id:"4bc716406aa2645e165fd7172df78a56cd648bdfc53d3840b2b3632c92460e07" timestamp:1465132608
DEBU[1024] Calling GET /containers/4bc716406aa2645e165fd7172df78a56cd648bdfc53d3840b2b3632c92460e07/json

All containers started by yad are running in the background with an sshd running.
@KostyaSha
Copy link
Owner

Your image has no EXPOSE or ssh port, that's it can't get exposed ssh port from portbindings. Either rebuild image, either specify port exposure, either use JNLP launcher that doesn't do ssh connection.

@antoineco
Copy link

@KostyaSha: I don't think that's the source of the problem, our image declares EXPOSE 22 and CMD ["/usr/sbin/sshd", "-D"]. YaD creates the containers and SSHD is running in each of them:

0cdfc9b227e7        jenkins-slave      "usr/sbin/s"   4 minutes ago       Up 4 minutes        0.0.0.0:32789->22/tcp                                      elated_bassi
4722089e7e35       jenkins-slave      "/usr/sbin/s"   4 minutes ago       Up 4 minutes        0.0.0.0:32788->22/tcp                                      modest_leavitt
084e0c04720d        jenkins-slave      "/usr/sbin/s"   4 minutes ago       Up 4 minutes        0.0.0.0:32787->22/tcp                                      kickass_mayer
0619621b89e2        jenkins-slave      "/usr/sbin/s"   4 minutes ago       Up 4 minutes        0.0.0.0:32786->22/tcp                                      suspicious_heyrovsky
654b6027f1ba       jenkins-slave      "/usr/sbin/s"   5 minutes ago       Up 5 minutes        0.0.0.0:32785->22/tcp                                      big_lichterman

Yet the same error is thrown:

ug 03, 2016 10:01:45 AM com.github.kostyasha.yad.DockerCloud provisionWithWait
INFO: Trying to run container for jenkins-slave
Aug 03, 2016 10:01:46 AM com.github.kostyasha.yad.DockerCloud lambda$provision$0
SEVERE: Error in provisioning; template='DockerSlaveTemplate[id=3aa18ba6-81d1-4103-9fcf-80172ba6870c,labelString=t-pay,launcher=com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher@787a1a8f,remoteFsMapping=,remoteFs=/home/jenkins,maxCapacity=10,mode=NORMAL,retentionStrategy=com.github.kostyasha.yad.strategy.DockerOnceRetentionStrategy@27f,numExecutors=1,dockerContainerLifecycle=com.github.kostyasha.yad.DockerContainerLifecycle@c7417f8[image=jenkins-slave,pullImage=com.github.kostyasha.yad.commons.DockerPullImage@cfdbc29d,createContainer=DockerCreateContainer[command=,hostname=,dnsHosts=[],volumes=[],volumesFrom=[],environment=[],bindPorts=,bindAllPorts=false,memoryLimit=0,cpuShares=<null>,privileged=false,tty=false,macAddress=<null>,extraHosts=[],networkMode=<null>,devices=[],cpusetCpus=,cpusetMems=],stopContainer=com.github.kostyasha.yad.commons.DockerStopContainer@27f,removeContainer=com.github.kostyasha.yad.commons.DockerRemoveContainer@5b0f],nodeProperties=[]]' for cloud='example.com
java.lang.NullPointerException
    at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.getHostAndPort(DockerComputerSSHLauncher.java:137)
    at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.waitUp(DockerComputerSSHLauncher.java:84)
    at com.github.kostyasha.yad.DockerCloud.provisionWithWait(DockerCloud.java:216)
    at com.github.kostyasha.yad.DockerCloud.lambda$provision$0(DockerCloud.java:117)
    at com.github.kostyasha.yad.DockerCloud$$Lambda$19/1989120500.call(Unknown Source)
    at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

The funny part is that the plugin acknowledges the limit of 10 slaves nevertheless:

Aug 03, 2016 9:58:55 AM com.github.kostyasha.yad.DockerCloud addProvisionedSlave
INFO: Not Provisioning 'jenkins-slave'. Instance limit of '10' reached on server 'example.com'

@antoineco
Copy link

I figured it out, my "Cloud Name" contained dots. Removing them fixed the issue.

Btw, thanks a bunch for maintaining that plugin 👍

@KostyaSha
Copy link
Owner

Looks very strange,

 java.lang.NullPointerException
    at com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher.getHostAndPort(DockerComputerSSHLauncher.java:137)

Please don't close issue, will check what is happening here.

@KostyaSha KostyaSha added the bug label Aug 3, 2016
@KostyaSha KostyaSha reopened this Aug 3, 2016
@KostyaSha KostyaSha mentioned this issue Aug 13, 2016
Merged
@Reonaydo
Copy link

Reonaydo commented Sep 9, 2016

Hello!
I have same issue. But I have no dots anywhere.

Here traceback:

SEVERE: Error in provisioning; template='DockerSlaveTemplate[id=f7e0973e-50b5-42ae-89d8-a33520535d81,labelString=docker centos7amd64docker,launcher=com.github.kostyasha.yad.launcher.DockerComputerSSHLauncher@79aacf92,remoteFs=/root/,maxCapacity=1,mode=EXCLUSIVE,retentionStrategy=com.github.kostyasha.yad.strategy.DockerCloudRetentionStrategy@119c8205,numExecutors=1,dockerContainerLifecycle=com.github.kostyasha.yad.DockerContainerLifecycle@17165cf8[image=centos-7,pullImage=com.github.kostyasha.yad.commons.DockerPullImage@dcad5037,createContainer=DockerCreateContainer[command=,hostname=,dnsHosts=[],volumes=[/root/ccache/centos7:/root/.ccache],volumesFrom=[],environment=[NODE_LABEL=centos7amd64, REPONAME=centos, REPOVER=7/x86_64],bindPorts=,bindAllPorts=true,memoryLimit=0,cpuShares=<null>,privileged=false,tty=false,macAddress=<null>,extraHosts=[],networkMode=<null>,devices=[],cpusetCpus=,cpusetMems=],stopContainer=com.github.kostyasha.yad.commons.DockerStopContainer@27f,removeContainer=com.github.kostyasha.yad.commons.DockerRemoveContainer@5b0f],nodeProperties=[hudson.slaves.EnvironmentVariablesNodeProperty@34c000da]]' for cloud='dockerserver1'
java.lang.NullPointerException
        at java.util.TreeMap.getEntry(TreeMap.java:347)
        at java.util.TreeMap.containsKey(TreeMap.java:232)
        at jenkinsci.plugin.browseraxis.label.BrowserFinder.checkNodes(BrowserFinder.java:44)
        at jenkinsci.plugin.browseraxis.label.BrowserFinder.findLabels(BrowserFinder.java:72)
        at hudson.model.Node.getDynamicLabels(Node.java:309)
        at hudson.model.Node.getAssignedLabels(Node.java:294)
        at hudson.model.Slave.<init>(Slave.java:185)
        at hudson.slaves.AbstractCloudSlave.<init>(AbstractCloudSlave.java:51)
        at com.github.kostyasha.yad.DockerSlave.<init>(DockerSlave.java:58)
        at com.github.kostyasha.yad.DockerCloud.provisionWithWait(DockerCloud.java:227)
        at com.github.kostyasha.yad.DockerCloud.lambda$provision$0(DockerCloud.java:120)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Plugin version: 0.1.0-rc23

@KostyaSha
Copy link
Owner

Very strange...
@Reonaydo please add logging for com.github.kostyasha.yad. class in jenkins global settings.

@KostyaSha
Copy link
Owner 

at jenkinsci.plugin.browseraxis.label.BrowserFinder.checkNodes(BrowserFinder.java:44)
        at jenkinsci.plugin.browseraxis.label.BrowserFinder.findLabels(BrowserFinder.java:72)

It not the same.

@KostyaSha
Copy link
Owner

please fill separate issue

@Reonaydo
Copy link

Reonaydo commented Sep 9, 2016

@KostyaSha
ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@KostyaSha @Reonaydo @antoineco @dasJ