Release 2024.4.0

README.containers.md

@@ -18,6 +18,7 @@ There are a number of containers that make up the application. Here are quick de
 12. [Storage Container (blackduck-storage)](#-storage-container-blackduck-storage)
 13. [Web App Container (blackduck-webapp)](#-web-app-container-blackduck-webapp)
 14. [Web Server Container (blackduck-nginx)](#-web-server-container-blackduck-nginx)
+15. [RL Service Container (rl-service)](#-rl-service-container-rl-service)
 
 # Web App Container (blackduck-webapp)
 ----
@@ -650,3 +651,40 @@ This container will need to expose port 8443 to other containers that will link
 
 This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing its main process.
 This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).
+
+# RL Service Container (rl-service)
+----
+
+## Container Description
+
+This container analyzes binary files for malware.
+This container is only used if Black Duck - ReversingLabs is enabled.
+
+## Scalability
+
+This container can be scaled.
+
+## Links/Ports
+
+This container needs to connect to these containers/services:
+* cfssl
+* logstash
+* rabbitmq
+* storage
+* scan
+* registration
+
+## Alternate Host Name Environment Variables
+
+It may be useful to set host names for these containers, that are not the Docker Swarm defaults, when running in other types of orchestrations. These environment variables can be set to override the default host names:
+
+* cfssl: $HUB_CFSSL_HOST
+* logstash: $HUB_LOGSTASH_HOST
+* rabbitmq: $RABBIT_MQ_HOST
+* storage: $BLACKDUCK_STORAGE_HOST
+* scan: $HUB_SCAN_HOST
+* registration: $HUB_REGISTRATION_HOST
+
+## Users/Groups
+
+This container runs as UID 1000 (rlservice username)

README.md

@@ -2,9 +2,9 @@
 
 This repository contains orchestration files and documentation for deploying Black Duck Docker containers.
 
-## Location of Black Duck 2024.1.1 archive:
+## Location of Black Duck 2024.4.0 archive:
 
-https://github.com/blackducksoftware/hub/archive/v2024.1.1.tar.gz
+https://github.com/blackducksoftware/hub/archive/v2024.4.0.tar.gz
 
 NOTE:
 
@@ -44,6 +44,7 @@ https://github.com/blackducksoftware/hub/releases
 * https://hub.docker.com/r/sigsynopsys/bdba-worker/
 * https://hub.docker.com/r/blackducksoftware/rabbitmq/
 * https://hub.docker.com/r/blackducksoftware/blackduck-webui/
+* https://hub.docker.com/r/blackducksoftware/rl-service/
 
 # Running Black Duck in Docker
 

docker-swarm/README.md

@@ -95,6 +95,21 @@ them unless this flag is added to the command above:
 --with-registry-auth
 ```
 
+## Running with ReversingLabs Enabled
+
+Note: This command might require being run as either a root user, a user in the docker group, or with 'sudo'.
+
+```
+docker stack deploy --compose-file docker-compose.yml -c docker-compose.rl.yml hub
+```
+
+There are some versions of docker where if the images live in a private repository, docker stack will not pull
+them unless this flag is added to the command above:
+
+```
+--with-registry-auth
+```
+
 ## Running with External PostgreSQL
 
 Hub can be run using a PostgreSQL instance other than the provided hub-postgres docker image.
@@ -231,6 +246,24 @@ Added definition:
         reservations: {cpus: '1', memory: 4096M}
 ```
 
+### Changing the Default ReversingLabs Memory Limits
+
+The default memory limits allow files up to 6GB to successfully scan. Additional memory and CPUs will potentially speed up scan times.
+
+The following configuration example will update the container memory limits from 6GB to 8GB. These configuration values can be changed 
+in the 'docker-compose.rl.yml':
+
+
+Added definition:
+
+```
+  rlservice:
+    deploy:
+      resources:
+        limits: {cpus: '2', memory: 8192M}
+        reservations: {cpus: '2', memory: 8192M}
+```
+
 ## Configuration
 
 There are several additional options that can be user-configured. This section describes these:
@@ -266,6 +299,7 @@ There are currently several containers that need access to services hosted by Bl
 * registration
 * scan
 * webapp
+* rl-service
 
 If a proxy is required for external internet access you'll need to configure it. 
 
@@ -291,6 +325,7 @@ There are several containers that will require the proxy password:
 * registration
 * scan
 * webapp
+* rl-service
 
 #### LDAP Trust Store Password
 
@@ -312,6 +347,7 @@ The proxy password secret will need to be added to the services:
 * registration
 * scan
 * webapp
+* rl-service
 
 In each of these service sections, you'll need to add:
 
@@ -524,6 +560,8 @@ For each of the services below, add the secret by
 * webapp
 * registration
 
+Note: The rl-service does not support proxies using certificates.
+
 ```
 secrets:
   - HUB_PROXY_CERT_FILE

docker-swarm/bin/hub_add_replication_user.sh

@@ -3,7 +3,7 @@
 set -e
 
 TIMEOUT=${TIMEOUT:-10}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.21}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.22}
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
 
 function fail() {

docker-swarm/bin/hub_create_data_dump.sh

@@ -5,8 +5,8 @@
 #  2. The database container has been properly initialized.
 
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.21}
-HUB_VERSION=${HUB_VERSION:-2024.1.1}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.22}
+HUB_VERSION=${HUB_VERSION:-2024.4.0}
 OPT_FORCE=
 OPT_LIVE_SYSTEM=
 OPT_MAX_CPU=${MAX_CPU:-1}

docker-swarm/bin/hub_db_migrate.sh

@@ -14,7 +14,7 @@
 set -o errexit
 
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.21}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.22}
 OPT_MAX_CPU=${MAX_CPU:-1}
 OPT_NO_DATABASE=${NO_DATABASE:-}
 OPT_NO_STORAGE=${NO_STORAGE:-}

docker-swarm/bin/hub_replication_changepassword.sh

@@ -3,7 +3,7 @@
 set -e
 
 TIMEOUT=${TIMEOUT:-10}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.21}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.22}
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
 
 function fail() {

docker-swarm/bin/hub_reportdb_changepassword.sh

@@ -3,7 +3,7 @@
 set -e
 
 TIMEOUT=${TIMEOUT:-10}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.21}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.22}
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
 
 function fail() {

docker-swarm/bin/system_check.sh

@@ -41,7 +41,7 @@ set -o noglob
 
 readonly NOW="$(date +"%Y%m%dT%H%M%S%z")"
 readonly NOW_ZULU="$(date -u +"%Y%m%dT%H%M%SZ")"
-readonly HUB_VERSION="${HUB_VERSION:-2024.1.1}"
+readonly HUB_VERSION="${HUB_VERSION:-2024.4.0}"
 readonly OUTPUT_FILE="${SYSTEM_CHECK_OUTPUT_FILE:-system_check_${NOW}.txt}"
 readonly PROPERTIES_FILE="${SYSTEM_CHECK_PROPERTIES_FILE:-${OUTPUT_FILE%.txt}.properties}"
 readonly SUMMARY_FILE="${SYSTEM_CHECK_SUMMARY_FILE:-${OUTPUT_FILE%.txt}_summary.properties}"
@@ -89,7 +89,6 @@ declare -ar REQ_CONTAINER_SIZES_G4=(
     "hub_uploadcache=512 512 512 1024 1536 2048 2048"
     "hub_webapp=3584 4048 5120 6144 20480 20480 20480"
     "hub_webserver=512 512 512 1024 2048 2048 2048"
-    "hub_webui=512 512 512 1024 1536 1536 1536"
 )
 declare -ar REQ_CONTAINER_SIZES_G3=(
     # "SERVICE=10sph 120sph 250sph 500sph 1000sph 1500sph 2000sph"
@@ -114,7 +113,6 @@ declare -ar REQ_CONTAINER_SIZES_G3=(
     "hub_storage=1024 1024 1024 1024 1024 1024 1024"
     "hub_webapp=3584 5120 8192 11264 15360 18432 18432"
     "hub_webserver=512 512 512 1024 2048 3072 3072"
-    "hub_webui=512 512 512 1024 1536 2048 2048"
 )
 declare -ar REQ_CONTAINER_SIZES_G2=(
     # "SERVICE=compose swarm kubernetes"
@@ -178,7 +176,7 @@ declare -ar SPH_MEM_SIZES_G4=(
     "hub_redis=900 900 1844 3687 4608 7373 9216"
     "hub_registration=922 1200 1200 1844 2765 2765 2765"
     "hub_scan=4608 9216 9216 9216 13824 13824 13824"
-    "hub_storage=512 2304 2765 3687 7373 7373 9100"
+    "hub_storage=512 1536 1996 3072 6554 6554 8192"
     "hub_webapp=3226 3608 4608 5530 18432 18432 18432"
 )
 declare -ar SPH_MEM_SIZES_G3=(
@@ -210,7 +208,6 @@ declare -ar TS_MEM_SIZES_G2=(
     "hub_scan=2048 2048 8192"  # sic
     "hub_webapp=2048 4096 8192"
     "hub_webserver=512 2048 2048"
-    "hub_webui=640 640 1024"
 )
 declare -ar TS_MEM_SIZES_G1=(
     # "SERVICE=small medium large" # in MB
@@ -317,7 +314,6 @@ declare -ar REPLICABLE=(
     "hub_storage=$FAIL"
     "hub_webapp=$FAIL"
     "hub_webserver=$WARN"
-    #"hub_webui=$PASS"
 )
 
 readonly MB=1048576
@@ -337,13 +333,13 @@ USE_NETWORK_TESTS="$TRUE"
 readonly NETWORK_TESTS_SKIPPED="*** Network Tests Skipped at command line ***"
 
 # Hostnames Black Duck uses within the docker network
-readonly HUB_RESERVED_HOSTNAMES="postgres postgres-upgrader postgres-waiter authentication webapp webui scan jobrunner cfssl logstash \
+readonly HUB_RESERVED_HOSTNAMES="postgres postgres-upgrader postgres-waiter authentication webapp scan jobrunner cfssl logstash \
 registration webserver documentation redis bomengine rabbitmq matchengine integration"
 
 readonly CONTAINERS_WITHOUT_CURL="nginx|postgres|postgres-upgrader|postgres-waiter|alert-database|cadvisor"
 
 # Versioned (not "1.0.x") blackducksoftware images
-readonly VERSIONED_HUB_IMAGES="blackduck-authentication|blackduck-bomengine|blackduck-documentation|blackduck-jobrunner|blackduck-matchengine|blackduck-redis|blackduck-registration|blackduck-scan|blackduck-storage|blackduck-webapp|blackduck-webui"
+readonly VERSIONED_HUB_IMAGES="blackduck-authentication|blackduck-bomengine|blackduck-documentation|blackduck-jobrunner|blackduck-matchengine|blackduck-redis|blackduck-registration|blackduck-scan|blackduck-storage|blackduck-webapp"
 readonly VERSIONED_BDBA_IMAGES="bdba-worker"
 readonly VERSIONED_ALERT_IMAGES="blackduck-alert"
 
@@ -2510,7 +2506,7 @@ _get_container_size_info() {
             case "$hub_service" in
                 (hub_redis*)
                     if [[ "$hub_service" == hub_redissentinel* ]]; then memvar="container_memory"; else memvar="BLACKDUCK_REDIS_MAXMEMORY"; fi;;
-                (hub_postgres* | hub_cfssl | hub_rabbitmq | hub_webserver | hub_webui)
+                (hub_postgres* | hub_cfssl | hub_rabbitmq | hub_webserver)
                     memvar="container_memory";;
                 (*)
                     memvar="HUB_MAX_MEMORY";;
@@ -2580,8 +2576,6 @@ _get_container_size_info() {
                     service="hub_storage";;
                 (blackducksoftware/blackduck-webapp*)
                     service="hub_webapp";;
-                (blackducksoftware/blackduck-webui*)
-                    service="hub_webui"; memvar="container_memory";;
                 (blackducksoftware/blackduck-nginx*)
                     service="hub_webserver"; memvar="container_memory";;
                 (blackducksoftware/blackduck-alert*)

docker-swarm/blackduck-config.env

@@ -24,7 +24,7 @@ BLACKDUCK_CORS_ALLOW_CREDENTIALS_PROP_NAME=
 
 # Do not change
 HUB_PRODUCT_NAME=BLACK_DUCK
-HUB_VERSION=2024.1.1
+HUB_VERSION=2024.4.0
 
 # Specify any property-specific overrides here
 #
@@ -117,6 +117,9 @@ BLACKDUCK_DETECT_MAJOR_VERSION_MIN=8
 # pass-thru ev for toggling JSON structured logging
 STRUCTURED_LOGGING
 
+# Enable/disable the alerting of certificate expiration
+BLACKDUCK_CERT_EXPIRATION_ALERT_ENABLED=true
+
 # Enable/disable secrets encryption. Once enabled, it can never be disabled. Use docker-compose.encryption.yml to
 # specify the root, backup and previous seeds, once secrets encryption is enabled.
 SYNOPSYS_CRYPTO_ENABLED=false

docker-swarm/docker-compose.bdba.yml

@@ -5,7 +5,7 @@ version: '3.6'
 
 services:
   binaryscanner:
-    image: sigsynopsys/bdba-worker:2023.12.3
+    image: sigsynopsys/bdba-worker:2024.3.0
     env_file: [hub-bdba.env]
     entrypoint: /docker-entrypoint.sh
     healthcheck:

docker-swarm/docker-compose.dbmigrate.yml

@@ -1,7 +1,7 @@
 version: '3.6'
 services:
   cfssl:
-    image: blackducksoftware/blackduck-cfssl:1.0.25
+    image: blackducksoftware/blackduck-cfssl:1.0.26
     volumes:
       - cert-volume:/etc/cfssl
     healthcheck:
@@ -12,7 +12,7 @@ services:
     user: 'cfssl:root'
 
   logstash:
-    image: blackducksoftware/blackduck-logstash:1.0.35
+    image: blackducksoftware/blackduck-logstash:1.0.36
     volumes:
       - log-volume:/var/lib/logstash/data
     healthcheck:
@@ -23,7 +23,7 @@ services:
     user: 'logstash:root'
 
   postgres:
-    image: blackducksoftware/blackduck-postgres:14-1.21
+    image: blackducksoftware/blackduck-postgres:14-1.22
     volumes:
       - postgres96-data-volume:/bitnami/postgresql
       - postgres-conf-volume:/opt/bitnami/postgresql/conf