Release 2020.10.0

blackducksoftware · Nov 5, 2020 · d6b5cc9 · d6b5cc9
1 parent 5e71336
commit d6b5cc9
Show file tree

Hide file tree

Showing 25 changed files with 196 additions and 92 deletions.
diff --git a/README.md b/README.md
@@ -2,13 +2,13 @@
 
 This repository contains orchestration files and documentation for deploying Black Duck Docker containers. 
 
-## Location of Black Duck 2020.8.1 archive:
+## Location of Black Duck 2020.10.0 archive:
 
-https://github.com/blackducksoftware/hub/archive/v2020.8.1.tar.gz
+https://github.com/blackducksoftware/hub/archive/v2020.10.0.tar.gz
 
-## Important Upgrade Announcement
+NOTE:
 
-Customers upgrading from a version prior to 2018.12.0 will experience a longer than usual upgrade time due to a data migration needed to support new features in this release. Upgrade times will depend on the size of the Black Duck database. If you would like to monitor the process of the upgrade, please contact Synopsys Customer Support for instructions.
+Customers upgrading from a version prior to 2018.12.0 will experience a longer than usual upgrade time due to a data migration needed to support new features in subsequent releases. Upgrade times will depend on the size of the Black Duck database. If you would like to monitor the process of the upgrade, please contact Synopsys Customer Support for instructions.
 
 Customers upgrading from a version prior to 4.2, will need to perform a data migration as part of their upgrade process.  A high level description of the upgrade is located in the Important_Upgrade_Announcement.md file in the root directory of this package.  Detailed instructions to perform the migration located in the individual README.md doc file in the directory for the each orchestration method folder.
 
@@ -31,15 +31,15 @@ https://github.com/blackducksoftware/hub/releases
 * https://hub.docker.com/r/blackducksoftware/blackduck-scan/
 * https://hub.docker.com/r/blackducksoftware/blackduck-webapp/
 * https://hub.docker.com/r/blackducksoftware/blackduck-upload-cache/
+* https://hub.docker.com/r/blackducksoftware/blackduck-redis/
 * https://hub.docker.com/r/sigsynopsys/bdba-worker/
 * https://hub.docker.com/r/blackducksoftware/rabbitmq/
 
 # Running Black Duck in Docker
 
-Swarm (mode), Compose, Kubernetes, and OpenShift are supported as of Black Duck (Hub) 4.2.0. Instructions for running each can be found in the archive bundle:
+Swarm (mode), Kubernetes, and OpenShift are supported as of Black Duck (Hub) 4.2.0. Instructions for running each can be found in the archive bundle:
 
 * docker-swarm - Instructions and files for running Black Duck with 'docker swarm mode'
-* docker-compose - Instructions and files for running Black Duck with 'docker-compose'
 * kubernetes - Instructions and files for running Black Duck with Kubernetes and OpenShift
 
 ## Requirements
@@ -48,33 +48,23 @@ Swarm (mode), Compose, Kubernetes, and OpenShift are supported as of Black Duck
 
 Black Duck supports the following orchestration environments:
 
-* Docker 17.09.x
-* Docker 17.12.x
 * Docker 18.03.x
 * Docker 18.06.x
 * Docker 18.09.x
-* Kubernetes 1.6
-* Kubernetes 1.7
-* Kubernetes 1.8
-* Kubernetes 1.9
-* Kubernetes 1.10
-* Red Hat OpenShift Container Platform 3.6
-* Red Hat OpenShift Container Platform 3.7
-* Red Hat OpenShift Container Platform 3.8
-* Red Hat OpenShift Container Platform 3.9
-* Red Hat OpenShift Container Platform 3.10
+* Docker 19.03.x (CE or EE)
+* Kubernetes 1.9.x-1.17
+* Red Hat OpenShift Container Platform 3.8-3.11
+* Red Hat OpenShift Container Platform 4.1
+* Red Hat OpenShift Container Platform 4.3
+* Red Hat OpenShift Container Platform 4.4
 
 ### Minimum Hardware Requirements
 
 This is the minimum hardware that is needed to run a single instance of each container. The sections below document the individual requirements for each container if they will be running on different machines or if more than one instance of a container will be run (right now only Job Runners support this).
 
-For Docker Compose:
-* 4 CPUs
-* 16 GB RAM
-
-For Docker Swarm, Kubernetes, and OpenShift:
 * 5 CPUs
-* 20 GB RAM
+* 21 GB RAM
+* 250 GB DISK SPACE
 
 Please note there that these are the minimum hardware requirements. These will likely need to be increased with larger or multiple concurrent scans.
 
@@ -85,5 +75,6 @@ Also, for Swarm, Kubernetes and OpenShift, note that these requirements are only
 There are variations of the orchestration files that will add additional containers for use in Binary Scanning. If these additional containers
 are added, then the following additional resources would be required:
 
-* 1 CPU
-* 4 GB RAM
+* 1 ADDITIONAL CPU
+* 4 GB ADDITIONAL RAM
+* 100 GB ADDITIONAL DISK SPACE
diff --git a/docker-swarm/bin/system_check.sh b/docker-swarm/bin/system_check.sh
@@ -34,7 +34,7 @@ set -o noglob
 
 readonly NOW="$(date +"%Y%m%dT%H%M%S%z")"
 readonly NOW_ZULU="$(date -u +"%Y%m%dT%H%M%SZ")"
-readonly HUB_VERSION="${HUB_VERSION:-2020.8.1}"
+readonly HUB_VERSION="${HUB_VERSION:-2020.10.0}"
 readonly OUTPUT_FILE="${SYSTEM_CHECK_OUTPUT_FILE:-system_check_${NOW}.txt}"
 readonly PROPERTIES_FILE="${SYSTEM_CHECK_PROPERTIES_FILE:-${OUTPUT_FILE%.txt}.properties}"
 readonly SUMMARY_FILE="${SYSTEM_CHECK_SUMMARY_FILE:-${OUTPUT_FILE%.txt}_summary.properties}"
@@ -3570,12 +3570,14 @@ generate_report_section() {
 }
 
 ################################################################
-# Save a full report to disk.
+# Save a full report to disk.  Assumes that all data has been
+# collected and is available in global variables.
 #
 # Globals:
 #   OUTPUT_FILE -- (in) default output file path.
 #   FAILURES -- (out) list of failures reported.
 #   WARNINGS -- (out) list of warnings reported.
+#   ... -- (in) everything.
 # Arguments:
 #   $1 - output file path, default "${OUTPUT_FILE}"
 # Returns:
@@ -3592,6 +3594,8 @@ generate_report() {
     local -r header="${REPORT_SEPARATOR}
 System check version $HUB_VERSION report for Black Duck version ${RUNNING_HUB_VERSION}
   generated at $NOW on $(hostname -f)
+
+Approximate installation size: ${INSTALLATION_SIZE}
 "
     local -r report=$(cat <<END
 $(generate_report_section "Operating System information")

diff --git a/docker-swarm/blackduck-config.env b/docker-swarm/blackduck-config.env
@@ -21,7 +21,7 @@ BLACKDUCK_CORS_ALLOWED_HEADERS_PROP_NAME=
 BLACKDUCK_CORS_EXPOSED_HEADERS_PROP_NAME=
 
 # Do not change
-HUB_VERSION=2020.8.1
+HUB_VERSION=2020.10.0
 
 # Specify any property-specific overrides here
 #
@@ -87,9 +87,9 @@ BLACKDUCK_REDIS_LOGLEVEL=notice
 # Whether or not to enforce TLS/SSL connections between Redis client and server.
 BLACKDUCK_REDIS_TLS_ENABLED=false
 # Maximum number of concurrent client connections that can be connected to Redis
-BLACKDUCK_REDIS_MAX_TOTAL_CONN=16
+BLACKDUCK_REDIS_MAX_TOTAL_CONN=128
 # Maximum number of concurrent client connections that can remain idle in the pool, without extra ones being released
-BLACKDUCK_REDIS_MAX_IDLE_CONN=8
+BLACKDUCK_REDIS_MAX_IDLE_CONN=128
 
 # All following sentinel configures are ignored if redis is in "standalone" mode.
 # The port that Redis sentinel  instance will run on. Default is 28379.

diff --git a/docker-swarm/docker-compose.bdba.yml b/docker-swarm/docker-compose.bdba.yml
@@ -5,7 +5,7 @@ version: '3.6'
 
 services:
   binaryscanner:
-    image: sigsynopsys/bdba-worker:2020.06-2
+    image: sigsynopsys/bdba-worker:2020.09
     env_file: [hub-bdba.env]
     entrypoint: /docker-entrypoint.sh
     deploy:
@@ -15,7 +15,7 @@ services:
         limits: {memory: 2048M, cpus: '1'}
         reservations: {memory: 2048M, cpus: '1'}
   rabbitmq:
-    image: blackducksoftware/rabbitmq:1.2.1
+    image: blackducksoftware/rabbitmq:1.2.2
     depends_on: [cfssl]
     tmpfs: [/opt/blackduck/rabbitmq/security]
     volumes: ['rabbitmq-data-volume:/var/lib/rabbitmq']

diff --git a/docker-swarm/docker-compose.externaldb.yml b/docker-swarm/docker-compose.externaldb.yml
@@ -6,7 +6,7 @@ version: '3.6'
 services:
   authentication:
     user: authentication:root
-    image: blackducksoftware/blackduck-authentication:2020.8.1
+    image: blackducksoftware/blackduck-authentication:2020.10.0
     volumes:
     - authentication-volume:/opt/blackduck/hub/hub-authentication/ldap
     - {type: tmpfs, target: /opt/blackduck/hub/hub-authentication/security}
@@ -28,7 +28,7 @@ services:
         reservations: {memory: 1024M}
   webapp:
     user: webapp:root
-    image: blackducksoftware/blackduck-webapp:2020.8.1
+    image: blackducksoftware/blackduck-webapp:2020.10.0
     volumes:
     - log-volume:/opt/blackduck/hub/logs
     - webapp-volume:/opt/blackduck/hub/hub-webapp/ldap
@@ -51,7 +51,7 @@ services:
         reservations: {cpus: '1', memory: 2560M}
   scan:
     user: scan:root
-    image: blackducksoftware/blackduck-scan:2020.8.1
+    image: blackducksoftware/blackduck-scan:2020.10.0
     env_file: [blackduck-config.env , hub-postgres.env]
     healthcheck:
       test: [CMD, /usr/local/bin/docker-healthcheck.sh, 'https://localhost:8443/api/health-checks/liveness',
@@ -72,7 +72,7 @@ services:
         reservations: {cpus: '1', memory: 2560M}
   jobrunner:
     user: jobrunner:root
-    image: blackducksoftware/blackduck-jobrunner:2020.8.1
+    image: blackducksoftware/blackduck-jobrunner:2020.10.0
     env_file: [blackduck-config.env , hub-postgres.env]
     healthcheck:
        test: [CMD, /usr/local/bin/docker-healthcheck.sh, 'https://localhost:8443/health-checks/liveness',
@@ -125,7 +125,7 @@ services:
         limits: {memory: 1024M}
         reservations: {memory: 1024M}
   registration:
-    image: blackducksoftware/blackduck-registration:2020.8.1
+    image: blackducksoftware/blackduck-registration:2020.10.0
     volumes:
     - config-volume:/opt/blackduck/hub/hub-registration/config
     - {type: tmpfs, target: /opt/blackduck/hub/hub-registration/security}
@@ -145,7 +145,7 @@ services:
         limits: {memory: 640M}
         reservations: {memory: 640M}
   webserver:
-    image: blackducksoftware/blackduck-nginx:1.0.25
+    image: blackducksoftware/blackduck-nginx:1.0.26
     ports: ['443:8443']
     env_file: [hub-webserver.env, blackduck-config.env]
     healthcheck:
@@ -164,7 +164,7 @@ services:
         limits: {memory: 512M}
         reservations: {memory: 512M}
   documentation:
-    image: blackducksoftware/blackduck-documentation:2020.8.1
+    image: blackducksoftware/blackduck-documentation:2020.10.0
     env_file: [blackduck-config.env]
     user: documentation:root
     healthcheck:
@@ -201,7 +201,7 @@ services:
           limits: {memory: 512M}
           reservations: {memory: 512M}
   redis:
-    image: blackducksoftware/blackduck-redis:2020.8.1
+    image: blackducksoftware/blackduck-redis:2020.10.0
     env_file: [blackduck-config.env]
     user: redis:root
     stop_grace_period: 60s
@@ -213,6 +213,8 @@ services:
       interval: 30s
       timeout: 10s
       retries: 5
+    volumes:
+    - {type: tmpfs, target: /opt/blackduck/hub/blackduck-redis/security}
     deploy:
       restart_policy: {condition: any}
       resources:

diff --git a/docker-swarm/docker-compose.readonly.yml b/docker-swarm/docker-compose.readonly.yml
@@ -32,13 +32,15 @@ services:
       - /opt/blackduck/hub/jobrunner/temp
       - /opt/blackduck/hub/filebeat
       - /opt/blackduck/hub/jobrunner/logs
+      - /tmp
   cfssl:
     read_only: true
   logstash:
     read_only: true
     volumes:
       - /usr/share/logstash
       - /var/log
+      - /tmp
   registration:
     read_only: true
     volumes:
@@ -65,3 +67,8 @@ services:
     volumes:
       - /opt/blackduck/hub/filebeat
       - /opt/blackduck/hub/blackduck-upload-cache/logs
+  redis:
+    read_only: true
+    volumes:
+      - /opt/blackduck/hub/blackduck-redis/logs
+      - /opt/blackduck/hub/filebeat
diff --git a/docker-swarm/docker-compose.redis.sentinel.yml b/docker-swarm/docker-compose.redis.sentinel.yml
@@ -1,7 +1,7 @@
 version: '3.6'
 services:
   redisslave:
-    image: blackducksoftware/blackduck-redis:2020.8.1
+    image: blackducksoftware/blackduck-redis:2020.10.0
     env_file: [blackduck-config.env]
     user: redis:root
     stop_grace_period: 60s
@@ -15,6 +15,8 @@ services:
       interval: 30s
       timeout: 10s
       retries: 5
+    volumes:
+    - {type: tmpfs, target: /opt/blackduck/hub/blackduck-redis/security}
     deploy:
       mode: replicated
       replicas: 2
@@ -23,7 +25,7 @@ services:
         limits: {memory: 1024M}
         reservations: {memory: 1024M}
   redissentinel1:
-    image: blackducksoftware/blackduck-redis:2020.8.1
+    image: blackducksoftware/blackduck-redis:2020.10.0
     env_file: [blackduck-config.env]
     hostname: redissentinel1
     user: redis:root
@@ -37,13 +39,15 @@ services:
       interval: 30s
       timeout: 10s
       retries: 5
+    volumes:
+    - {type: tmpfs, target: /opt/blackduck/hub/blackduck-redis/security}
     deploy:
       restart_policy: {condition: any}
       resources:
         limits: {memory: 32M}
         reservations: {memory: 32M}
   redissentinel2:
-    image: blackducksoftware/blackduck-redis:2020.8.1
+    image: blackducksoftware/blackduck-redis:2020.10.0
     env_file: [blackduck-config.env]
     hostname: redissentinel2
     user: redis:root
@@ -57,13 +61,15 @@ services:
       interval: 30s
       timeout: 10s
       retries: 5
+    volumes:
+    - {type: tmpfs, target: /opt/blackduck/hub/blackduck-redis/security}
     deploy:
       restart_policy: {condition: any}
       resources:
         limits: {memory: 32M}
         reservations: {memory: 32M}
   redissentinel3:
-    image: blackducksoftware/blackduck-redis:2020.8.1
+    image: blackducksoftware/blackduck-redis:2020.10.0
     env_file: [blackduck-config.env]
     hostname: redissentinel3
     user: redis:root
@@ -77,6 +83,8 @@ services:
       interval: 30s
       timeout: 10s
       retries: 5
+    volumes:
+    - {type: tmpfs, target: /opt/blackduck/hub/blackduck-redis/security}
     deploy:
       restart_policy: {condition: any}
       resources: