DevTools for integration automation, QA/CI maintenance...
More usage details are available in the scripts.
cleanup-registry.groovy: Jenkins pipeline to delete cache and temporary tags (working branches, closed PRs...).purge.sh: Purge tags from Docker repositories based on their name and labels.
GITHUB_TOKEN=$(git config --get github.token) ./purge.shdeleteCache.sh: Delete Docker repositories which name ends with "/cache*", tag by tag.deleteTags.sh: Bulk delete Docker repository tags by name
https://jenkins.ai.dev.nuxeo.com/job/nuxeo/job/cleanup-registry/
Utility script for downloading a distribution.
Other related scripts:
upload.shused by /Deploy/IT-nuxeo-master-tests-cap-tomcat
Used in jobs like:
- /Deploy/IT-nuxeo-master-tests-cap-tomcat
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-sqlserver
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-postgresql
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-mysql
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-jre
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-sqlserver2014
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-mariadb
- /Deploy/IT-release-on-demand-tests-cap-tomcat
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-mongodb
- /Deploy/IT-nuxeo-master-tests-cap-tomcat-oracle12
Refines the released distribution: repackage, fix details, finalize for easy-use out of the box.
/Deploy/IT-release-on-demand-build /Deploy/IT-release-nuxeo.io /Deploy/IT-nuxeo-10.10-build /Deploy/IT-nuxeo-11.1-build /Deploy/IT-nuxeo-master-build
QA/System/git-branches-cleanup job runs
git-branches-cleanup.sh following CORG/Git Usage.
The principle is to delete Git branches older than 3 days and which JIRA ticket is resolved or closed, and with no backport-* tag.
In case of doubt, the branch is kept forever.
https://qa.nuxeo.org/jenkins/job/System/job/git-branches-cleanup/
DEPRECATED. Server management and related CI tooling.
Various WS cleanup tricks.
https://jira.nuxeo.com/browse/NXBT-1915
Platform repositories consolidation: lists the repositories to be assembled along with the Platform, in completion to the main structure and the addons. Those repositories contain code for building a Nuxeo Package or a Nuxeo Plugin (= package + addon).
It is mainly used by the Git and release tooling.
Synchronizes local caches for network optimization.
https://qa.nuxeo.org/jenkins/job/Deploy/job/nexus-sync-snapshots/ https://qa.nuxeo.org/jenkins/job/Deploy/job/nexus-sync-releases/
Used by T&P jobs to generate metrics.
https://qa.nuxeo.org/jenkins/job/System/job/ondemand-testandpush-metrics/
TODO
├── qa.groovy // Development [QA](https://qa.nuxeo.org/jenkins/)
├── qa2.groovy // Maintenance [QA2](https://qa2.nuxeo.org/jenkins/)
├── update_static_slaves.groovy // main update script
├── common
│ ├── jenkins_workspace_cleanup.sh
│ ├── kill_remote.sh
│ └── pull_images.sh
├── qa-ovh01 // Host qa-ovh01
│ ├── manual // Legacy local maintenance scripts
│ ├── start_remote_priv.sh
│ └── start_remote.sh
├── qa-ovh02 // Host qa-ovh02
│ ├── manual // Legacy local maintenance scripts
│ ├── start_remote_priv.sh
│ └── start_remote.sh
└── qa-ovh03 // Host qa-ovh03
├── manual // Legacy local maintenance scripts
├── start_remote_priv.sh
└── start_remote.sh
jenkins_workspace_cleanup.sh Cleanup Jenkins hosts, Docker and workspaces:
- Prune stopped images
- Prune stopped volumes
- Delete "exited" containers
- Delete T&P jobs older than 3 days
- Delete Nuxeo server ZIP files and unzipped folders older than 3 days
- Remove Git repositories parent folders older than 5 days
- Remove Git repositories parent folders older than 2 days and bigger than 100M
- Remove files that the Workspace Cleanup plugin has no permission to delete (NXBT-2205, JENKINS-24824)
/System/update_static_slaves is triggered by /System/build-slave-images and co
Fetch "static" slaves list by label from Jenkins masters QA, QA2.
Works on idle and online slaves labelled STATIC.
update_static_slaves.groovy will pull_images.sh and kill_remotes.sh static slaves then start_remote.sh and start_remote_priv.
See getLabelsBySlaves.groovy for the extraction of existing labels.
Naming convention is not yet enforced but tends to pattern like prefix[n]-<host>[-<ID>] to interprete as:
[n]: if dedicated toqa<n>.nuxeo.orginstead of defaultqa.nuxeo.org<host>: the Docker host within the Docker swarm, ieqa-ovh01.nuxeo.com,qa-ovh02.nuxeo.com...[-<ID>]: increment when there are multiple instance with the same name
$ docker -H tcp://swarm-qa.nuxeo.org:4000 ps -a --format "table {{.Names}}\t{{.Ports}}\t{{.Image}}"|sort
NAMES PORTS IMAGE
qa-ovh01/itslave01 51.254.42.78:2301->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-it
qa-ovh01/itslavepriv01 51.254.42.78:3401->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-itpriv
qa-ovh01/matrix01 51.254.42.78:2302->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave
qa-ovh01/priv-01-1 51.254.42.78:3311->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh01/priv-01-2 51.254.42.78:3312->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh01/priv2-01-1 51.254.42.78:4401->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh01/static01 51.254.42.78:2201->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave
qa-ovh02/itslave910 51.254.197.210:2304->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-it-9.10
qa-ovh02/itslave1010 51.254.197.210:2304->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-it-10.10
qa-ovh02/matrix02 51.254.197.210:2302->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave
qa-ovh02/priv-02-1 51.254.197.210:3301->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh02/priv-02-2 51.254.197.210:3302->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh02/priv2-02-1 51.254.197.210:4401->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh02/slavepriv2-910-1 51.254.197.210:5701->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv-9.10
qa-ovh02/slavepriv2-1010-1 51.254.197.210:5701->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv-10.10
qa-ovh02/static910 51.254.197.210:2203->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave-9.10
qa-ovh02/static1010 51.254.197.210:2203->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave-10.10
qa-ovh03/itslave03 151.80.31.37:2301->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-it
qa-ovh03/itslavepriv03 151.80.31.37:3401->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-itpriv
qa-ovh03/matrix03 151.80.31.37:2302->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave
qa-ovh03/priv-03-1 151.80.31.37:3301->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh03/priv-03-2 151.80.31.37:3302->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh03/priv2-03-1 151.80.31.37:4401->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slavepriv
qa-ovh03/static03 151.80.31.37:2201->22/tcp dockerpriv.nuxeo.com:443/nuxeo/jenkins-slave
TODO
TODO
├── jenkins_perform.sh
├── jenkins_release.sh
└── post-release-new-jobs.sh
TODO
bower_deploy
build-and-bench, build-and-test
https://jira.nuxeo.com/browse/NXBT/component/11326 https://jira.nuxeo.com/secure/CreateIssue!default.jspa?project=NXBT
The Nuxeo Platform is an open source customizable and extensible content management platform for building business applications. It provides the foundation for developing document management, digital asset management, case management application and knowledge management. You can easily add features using ready-to-use addons or by extending the platform using its extension point system.
The Nuxeo Platform is developed and supported by Nuxeo, with contributions from the community.
Nuxeo dramatically improves how content-based applications are built, managed and deployed, making customers more agile, innovative and successful. Nuxeo provides a next generation, enterprise ready platform for building traditional and cutting-edge content oriented applications. Combining a powerful application development environment with SaaS-based tools and a modular architecture, the Nuxeo Platform and Products provide clear business value to some of the most recognizable brands including Verizon, Electronic Arts, Sharp, FICO, the U.S. Navy, and Boeing. Nuxeo is headquartered in New York and Paris. More information is available at www.nuxeo.com.