Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

web and nginx slaves #1

Closed
wants to merge 4 commits into from
Closed

Conversation

atarkowska
Copy link

This PR improves existing web infrastructure introduced in ome#19

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                   NAMES
2036d9641125        devspace_web        "/usr/sbin/init"         2 minutes ago       Up 2 minutes        22/tcp, 0.0.0.0:4080->4080/tcp          devspace_web_1
5f94f815fe7b        devspace_slave      "/tmp/jenkins-slave.s"   2 minutes ago       Up 2 minutes                                                devspace_slave_1
a773bc06f903        devspace_nginx      "/usr/sbin/init"         2 minutes ago       Up 2 minutes        22/tcp, 443/tcp, 0.0.0.0:8888->80/tcp   devspace_nginx_1
41121d7114bd        devspace_jenkins    "/bin/tini -- /usr/lo"   2 minutes ago       Up 2 minutes        0.0.0.0:8080->8080/tcp, 50000/tcp       devspace_jenkins_1
89b0a8f675b1        postgres            "/docker-entrypoint.s"   2 minutes ago       Up 2 minutes        5432/tcp                                devspace_pg_1
0d64ddcb5ce7        redis               "/entrypoint.sh redis"   2 minutes ago       Up 2 minutes        0.0.0.0:6379->6379/tcp                  devspace_redis_1

nginx is available on http://$(docker-machine ip omerodev):8888

  • use swarm plugin 2.0
  • use latest omero-install and fix Dockerfile
  • make nginx a jenkins node
  • allow systemd to start services on web and nginx
  • allow linking nginx config and statics
  • add redis node
jenkins_1 | Mar 26, 2016 9:41:49 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
jenkins_1 | INFO: Accepted connection #1 from /172.17.0.7:58020
jenkins_1 | Mar 26, 2016 9:41:49 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
jenkins_1 | INFO: Accepted connection #2 from /172.17.0.6:38142
jenkins_1 | Mar 26, 2016 9:41:49 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
jenkins_1 | INFO: Accepted connection #3 from /172.17.0.5:42540

NOTE:

 - use swarm plugin 2.0
 - use omero-install and fix Dockerfile
@atarkowska
Copy link
Author

@sbesson could you advice why BF job is failing?

2016-03-24 19:38:07,333 [   scc.merge] INFO  Merged PRs:
2016-03-24 19:38:07,333 [   scc.merge] INFO    # PR 1828 melissalinkert 'JPEG: add basic support for CMYK images'
...
2016-03-24 19:38:07,334 [   scc.merge] INFO    # PR 2305 melissalinkert 'Zeiss CZI: fixes for files with multiple mosaics and positions'
2016-03-24 19:38:07,334 [   scc.merge] INFO  

/home/slave/.local/lib/python2.6/site-packages/scc/git.py:79: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  if exception.message != "rc=128":

Traceback (most recent call last):
  File "/home/slave/.local/lib/python2.6/site-packages/scc/main.py", line 76, in entry_point
    (UpdateSubmodules.NAME, UpdateSubmodules),
  File "/home/slave/.local/lib/python2.6/site-packages/yaclifw/framework.py", line 197, in main
    ns.func(ns)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 2912, in __call__
    self.push(args, self.main_repo)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 1984, in push
    main_repo.rpush(branch_name, remote, force=True)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 1833, in rpush
    self.push_branch(branch_name, remote=full_remote, force=force)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 98, in wrapper
    return func(*args, **kwargs)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 1147, in push_branch
    self.call("git", "push", "-f", remote, name)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 1013, in call
    return self.wrap_call(self.debugWrap, *command, **kwargs)
  File "/home/slave/.local/lib/python2.6/site-packages/scc/git.py", line 1038, in wrap_call
    raise Exception("rc=%s" % rc)
Exception: rc=128
Build step 'Execute shell' marked build as failure

Finished: FAILURE

@sbesson
Copy link

sbesson commented Mar 24, 2016

@aleksandra-tarkowska: this looks like a permission issue while pushing the merge branch. Did you set up the SSH keys and .gitconfig on the slave workspace as described in the README?

@atarkowska
Copy link
Author

I could forget but after coping rebuilding is the same

Type=forking
Restart=on-failure
RestartSec=10
ExecStart=/tmp/jenkins-slave.sh
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this start jenkins as a background (daemon) process, or in the foreground? If it's in the foreground you can use Type simple instead and omit the ExecStop, e.g. see https://github.com/ome/omero-install/blob/develop/linux/omero-systemd.service

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean https://documentation.cloudbees.com/docs/dev-at-cloud/On-Premise+Executors.html?query=systemd#running-the-slave-with-systemd-on-linux
???

[root@102787622c66 ~]# systemctl status jenkins.service
● jenkins.service - Jenkins slave
   Loaded: loaded (/etc/systemd/system/jenkins.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2016-03-25 16:04:12 UTC; 1min 14s ago
     Docs: https://developer.cloudbees.com/bin/view/DEV/On-Premise+Executors
 Main PID: 216 (java)
   CGroup: /system.slice/jenkins.service
           └─216 java -jar /tmp/swarm-client-2.0-jar-with-dependencies.jar -fsroot /home/omero -labels centos7 -labels java -name web -disableClientsUniq...

Mar 25 16:05:03 102787622c66 jenkins-slave.sh[216]: Retrying in 10 seconds
Mar 25 16:05:16 102787622c66 systemd[1]: Started Jenkins slave.
Mar 25 16:05:18 102787622c66 jenkins-slave.sh[216]: Failed to receive a reply to broadcast.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

- make nginx a jankins node
- allow linking nginx config and statics
- add redis node
@atarkowska atarkowska changed the title fix web slave web and nginx slave using systemd Mar 26, 2016
@atarkowska atarkowska changed the title web and nginx slave using systemd web and nginx slaves Mar 26, 2016
@@ -1,6 +1,8 @@
#!/bin/bash
# https://raw.githubusercontent.com/carlossg/jenkins-swarm-slave-docker/master/jenkins-slave.sh

eval `(sudo cat /proc/1/environ; echo) | tr '\000' '\n' | while read line; do if [[ $line == JENKINS_PORT_8080_TCP_* ]] ; then echo export $line; fi done`
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manics indeed dirty hack works ;-)

@atarkowska atarkowska mentioned this pull request Mar 26, 2016
2 tasks
@atarkowska
Copy link
Author

Could anyone suggest why am I getting:

Building remotely on slave (java swarm) in workspace /home/slave/workspace/OMERO-push
Installing JDK jdk-8u74-oth-JPR
Downloading JDK from http://download.oracle.com/otn/java/jdk/8u74-b02/jdk-8u74-linux-x64.tar.gz
Your Oracle account doesn't appear valid. Please specify a valid username/password
ERROR: Unable to install JDK unless a valid username/password is provided.
Finished: FAILURE

- JENKINS_MODE=exclusive
- SERVICE_NAME=SPACENAME-omero
ports:
- "4080:4080"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be in the overwrite file (e.g. docker-compose.osx.yml) since otherwise only one devspace can run on a single docker host.

@atarkowska
Copy link
Author

Clsing in favor of ome#20

@atarkowska atarkowska closed this Mar 27, 2016
joshmoore pushed a commit that referenced this pull request Feb 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants