New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade omero-install #63

Merged
merged 18 commits into from Jan 25, 2017
Jump to file or symbol
Failed to load files and symbols.
+85 −135
Diff settings

Always

Just for now

View
@@ -1,19 +1,13 @@
sudo: required
language: java
language: python
services:
- docker
env:
global:
- DOCKER_COMPOSE_VERSION=1.7.0
before_install:
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
- docker --version
- pip install docker-compose
- docker-compose -v
- export USER_ID=`$UID`
View
102 README.md
@@ -5,75 +5,75 @@ an automation framework that runs repeated jobs. The default deployment
initializes a Jenkins CI master with a predefined set of jobs.
Running and maintaining Devspace require:
* brief understanding of ansible http://docs.ansible.com/ansible/intro_getting_started.html
Running and maintaining Devspace requires brief understanding of:
* Docker engine https://docs.docker.com/
* Docker compose
Running and maintaining Devspace in OpenStack requires brief understanding of:
* Docker engine https://docs.docker.com/ and Docker compose
* ansible http://docs.ansible.com/ansible/intro_getting_started.html
* inventory http://docs.ansible.com/ansible/intro_inventory.html
* playbook http://docs.ansible.com/ansible/playbooks.html
* access to openstack tenancy
* access to openstack tenancy
* own ssh key set in openstack tenancy, that name will be used as `vm_key_name`
* openrc.sh http://docs.openstack.org/user-guide/common/cli-set-environment-variables-using-openstack-rc.html
* snoopy ssh key and gitconfig
* snoopy ssh key and gitconfig (optional)
## Requirements
## Installation
The following prerequisites are required for deploying a Jenkins devspace:
#### Manual
Client machine:
* Ansible 2.1+
* Shade
Install following prerequesits:
Create virtualenv:
* Docker engine https://docs.docker.com/engine/installation/
* Docker compose
$ virtualenv dev
$ source dev/bin/activate
(dev) $ pip install ansible
(dev) $ pip install shade
$ pip install docker-compose
Clone infrastructure repository where all ansible playbooks and roles are
Clone devspace repository and start devspace
(dev) $ git clone https://github.com/openmicroscopy/infrastructure.git
(dev) $ cd infrastructure/ansible
$ docker-compose -f docker-compose.yml up --build
## Deployment
#### OpenStack
Ansible playbooks are available in https://github.com/openmicroscopy/infrastructure/tree/master/ansible
Clone infrastructure repository:
* It is recommended to use devspace playbook to install devspace on a Virtual Machine like OpenStack
* create new vm
(dev) $ source path/to/openrc.sh
# vm_key_name is a name of ssh key in openstack
# vm_size (default 50GB) is a size of the volume vm boot from. You no longer have to attach additional volumes!
(dev) $ ansible-playbook os-devspace.yml -e vm_name=my-devspace -e vm_key_name=mysshkey
$ git clone https://github.com/openmicroscopy/infrastructure.git
$ virtualenv dev
$ source dev/bin/activate
(dev) $ pip install -r requirements.txt
(dev) $ source tenancy.rc
(dev) $ cd infrastructure/ansible
NOTE: VM will boot from volume, you no longer have to attach additional volumes. Size of the volume can be set by `-e vm_size=100`
NOTE: VM will boot from volume, you no longer have to attach additional volumes. Size of the volume can be set by `-e vm_size=100`
* create inventory
$ tree /path/to/inventory
devspace
├── devspace-hosts
├── group_vars
│ └── devspace
└── snoopy
├── .gitconfig
└── .ssh
(dev) $ ansible-playbook os-devspace.yml -e vm_name=devspace-test -e vm_key_name=your_key
(dev) $ ansible-playbook -l devspace-test -u centos devspace.yml
/path/to/inventory/devspace/group_vars/devspace
To deploy devspace from custom branch, first set up inventory:
openstack_ip: 10.0.50.100
omero_branch: develop
snoopy_dir_path: "/path/to/ssh_keys/"
$ tree /path/to/inventory
inventory
├── group_vars
│ └── devspace
└── snoopy
├── .gitconfig
└── .ssh
/path/to/inventory/devspace/devspace-hosts
* add variables to group_vars/devspace:
[devspace]
10.0.50.100
omero_branch: develop
snoopy_dir_path: "/path/to/snoopy"
NOTE:
git_repo: "https://github.com/user_name/devspace.git"
version: "your_branch"
NOTE:
`omero_branch` is a name of the git branch all the jobs will be using. By default it is using `https://github.com/openmicroscopy/openmicroscopy/tree/develop`.
If you wish to use your own fork please adjust the jobs manually.
@@ -84,11 +84,9 @@ Ansible playbooks are available in https://github.com/openmicroscopy/infrastruct
-rwx------. 1 1674 Sep 13 15:25 snoopycrimecop_github
-rwx------. 1 405 Sep 13 15:25 snoopycrimecop_github.pub
* install prerequisites as default user with sudo privileges (as user `centos`)
ansible-playbook -i /path/to/inventory/devspace -u centos devspace.yml
devspace should be already started at https://10.0.50.100:8443
Devspace should be already started at https://your_host:8443
## ADVANCE: Multiply containers
@@ -170,7 +168,7 @@ they are associated with and a short description of the job:
| -----------------------|-----------------| ------------------------------------------|
| Trigger | | Runs all the following jobs in order |
| BIOFORMATS-push | testintegration | Merges all Bio-Formats PRs |
| BIOFORMATS-ant | bf | Builds Bio-Formats and runs unit tests |
| BIOFORMATS-ant | testintegration | Builds Bio-Formats and runs unit tests |
| BIOFORMATS-maven | testintegration | Builds Bio-Formats and runs unit tests |
| OMERO-push | testintegration | Merges all OMERO PRs |
| OMERO-build | testintegration | Builds OMERO artifacts (server, clients) |
@@ -223,8 +221,6 @@ Then fetch custom omero-install branch by updating each Dockerfile
--webhost "10.0.50.100"
* TestNG requires setting correct jenkins hostname in Jenkins Configuration / Jenkins Location / Jenkins URL.
## Upgrade
View

This file was deleted.

Oops, something went wrong.
View
@@ -4,15 +4,6 @@
# web is a container with basic omero web dependances
# nginx is a nginx container for omero web
basebf:
build: ./bf
volumes:
- ./bf:/home/omero
environment:
- SLAVE_NAME=basebf
- SLAVE_EXECUTORS=2
- SLAVE_PARAMS=-labels centos7 -labels java -disableClientsUniqueId
baseslave:
build: ./slave
volumes:
View
@@ -15,18 +15,6 @@ pg:
volumes:
- ./pgdata:/var/lib/postgresql/data
bf:
extends:
file: common-services-v1.yml
service: basebf
links:
- jenkins
volumes:
- ./bf:/home/omero
environment:
- SLAVE_NAME=bf
- SLAVE_PARAMS=-labels centos7 -labels java18 -disableClientsUniqueId
testintegration:
extends:
file: common-services-v1.yml
View
@@ -22,7 +22,7 @@
<hudson.tools.InstallSourceProperty>
<installers>
<hudson.tools.JDKInstaller>
<id>jdk-8u112-oth-JPR</id>
<id>jdk-8u121-oth-JPR</id>
<acceptLicense>true</acceptLicense>
</hudson.tools.JDKInstaller>
</installers>
@@ -19,6 +19,16 @@
<description></description>
<defaultValue>/opt/findbugs</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>testng.runWriterSaveBytesTests</name>
<description></description>
<defaultValue>50</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>testng.runWriterTilingTests</name>
<description></description>
<defaultValue>2</defaultValue>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
</properties>
@@ -40,7 +50,7 @@
<hudson.plugins.git.extensions.impl.CleanCheckout/>
</extensions>
</scm>
<assignedNode>bf</assignedNode>
<assignedNode>testintegration</assignedNode>
<canRoam>false</canRoam>
<disabled>false</disabled>
<blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
@@ -49,13 +59,13 @@
<concurrentBuild>false</concurrentBuild>
<builders>
<hudson.tasks.Ant plugin="ant@1.2">
<targets>clean release jars tools utils dist-bftools dist-matlab docs docs-sphinx test findbugs</targets>
<targets>clean release tools utils docs docs-sphinx dist-bftools dist-matlab dist-octave test findbugs</targets>
<antName>Ant 1.9</antName>
</hudson.tasks.Ant>
</builders>
<publishers>
<hudson.tasks.ArtifactArchiver>
<artifacts>artifacts/*.jar, artifacts/*.zip</artifacts>
<artifacts>artifacts/*.jar, artifacts/*.zip, artifacts/*.tar.*</artifacts>
<allowEmptyArchive>false</allowEmptyArchive>
<onlyIfSuccessful>false</onlyIfSuccessful>
<fingerprint>false</fingerprint>
@@ -16,6 +16,11 @@
<description></description>
<defaultValue>14064</defaultValue>
</hudson.model.StringParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>WEBHOST</name>
<description></description>
<defaultValue>SPACENAME-ci.openmicroscopy.org</defaultValue>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
</properties>
@@ -75,7 +80,7 @@ cd $SRC/dist
bash ../components/tests/ui/robot_setup.sh
mv $SRC/dist/robot_ice.config $SRC/components/tests/ui/resources/
export ICE_CONFIG=$SRC/components/tests/ui/resources/robot_ice.config
$SRC/dist/bin/omero --path $SRC/components/tests/ui/plugins robot config --protocol http --webhost &apos;SPACENAME-ci.openmicroscopy.org&apos; --remoteurl &apos;http://seleniumhub:4444/wd/hub&apos; --dc &apos;browserName:${BROWSER},javascriptEnabled:True&apos; &gt;$SRC/components/tests/ui/resources/config.txt
$SRC/dist/bin/omero --path $SRC/components/tests/ui/plugins robot config --protocol http --webhost $WEBHOST --remoteurl &apos;http://seleniumhub:4444/wd/hub&apos; --dc &apos;browserName:${BROWSER},javascriptEnabled:True&apos; &gt;$SRC/components/tests/ui/resources/config.txt
sed -i.bak -re &apos;s/(\{SERVER_ID\}.*1)/\{SERVER_ID\} 2/&apos; $SRC/components/tests/ui/resources/config.txt
@@ -80,9 +80,6 @@ virtualenv $WORKSPACE/omero-virtualenv --system-site-packages
source $WORKSPACE/omero-virtualenv/bin/activate
pip install omego
# TODO: use RPM
pip install --upgrade &quot;Pillow&lt;3&quot;
## LOAD CONFIG
@@ -100,10 +100,6 @@ source $WORKSPACE/omero-virtualenv/bin/activate
# install web dependences
pip install -r $OMERO_DIST/share/web/requirements-py27-nginx.txt
# TODO: use RPM
pip install --upgrade &quot;Pillow&lt;3&quot;
# START OMERO
BUILD_ID=DONT_KILL_ME $OMERO_DIST/bin/omero admin start
$OMERO_DIST/bin/omero admin waitup
@@ -65,9 +65,6 @@ if [ $ICEVERSION = "noice" ]; then
pip install &quot;zeroc-ice&gt;3.5,&lt;3.7&quot;
fi
# TODO: use RPM
pip install --upgrade &quot;Pillow&lt;3&quot;
#start-config
$OMERO_DIST/bin/omero config set omero.web.application_server &apos;wsgi-tcp&apos;
$OMERO_DIST/bin/omero config set omero.web.application_server.max_requests 0
View
@@ -1,4 +1,4 @@
FROM openmicroscopy/devslave-c7:0.2.1
FROM openmicroscopy/devslave-c7:0.3.0
MAINTAINER OME
View
@@ -1,4 +1,4 @@
FROM openmicroscopy/devslave-c7:0.2.1
FROM openmicroscopy/devslave-c7:0.3.0
MAINTAINER OME
@@ -45,10 +45,6 @@ WORKDIR /tmp/omero-install/linux
RUN JAVAVER=$JAVAVER ICEVER=$ICEVER PGVER=nopg bash install_centos7_nginx.sh
# TODO: use RPM
#RUN yum -y install python-pillow \
# && yum clean all
# install postgres tools
RUN yum -y install http://yum.postgresql.org/9.4/redhat/rhel-7-x86_64/pgdg-centos94-9.4-2.noarch.rpm \
&& yum clean all
View
@@ -1,4 +1,4 @@
FROM openmicroscopy/devslave-c7:0.2.1
FROM openmicroscopy/devslave-c7:0.3.0
MAINTAINER OME
@@ -45,12 +45,16 @@ WORKDIR /tmp/omero-install/linux
RUN JAVAVER=$JAVAVER ICEVER=$ICEVER PGVER=nopg bash install_centos7_nginx.sh
# TODO: use RPM
#RUN yum -y install python-pillow \
# && yum clean all
# install sphinx - minimum required
RUN pip install sphinx==1.2.3
# install sphinx
RUN pip install sphinx
# Install FindBugs
ENV FINDBUGS_VERSION 3.0.0
ENV FINDBUGS_HOME /opt/findbugs
RUN mkdir -p $FINDBUGS_HOME
RUN curl -fSLO http://downloads.sourceforge.net/project/findbugs/findbugs/$FINDBUGS_VERSION/findbugs-$FINDBUGS_VERSION.tar.gz && \
tar xzf findbugs-$FINDBUGS_VERSION.tar.gz --strip-components 1 -C $FINDBUGS_HOME && \
rm findbugs-$FINDBUGS_VERSION.tar.gz
# install postgres tools
RUN yum -y install http://yum.postgresql.org/9.4/redhat/rhel-7-x86_64/pgdg-centos94-9.4-2.noarch.rpm \
Oops, something went wrong.
ProTip! Use n and p to navigate between commits in a pull request.