diff --git a/README.md b/README.md index a0e32e7..4e5a432 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,41 @@ [![default profile](https://github.com/OHDSI/Broadsea/actions/workflows/default.yml/badge.svg?branch=develop)](https://github.com/OHDSI/Broadsea/actions/workflows/default.yml) [![perseus profile](https://github.com/OHDSI/Broadsea/actions/workflows/perseus.yml/badge.svg?branch=develop)](https://github.com/OHDSI/Broadsea/actions/workflows/perseus.yml) [![openldap profile](https://github.com/OHDSI/Broadsea/actions/workflows/openldap.yml/badge.svg?branch=develop)](https://github.com/OHDSI/Broadsea/actions/workflows/openldap.yml) [![solr-vocab Profile](https://github.com/OHDSI/Broadsea/actions/workflows/solr-vocab.yml/badge.svg?branch=develop)](https://github.com/OHDSI/Broadsea/actions/workflows/solr-vocab.yml) [![achilles Profile](https://github.com/OHDSI/Broadsea/actions/workflows/achilles.yml/badge.svg?branch=develop)](https://github.com/OHDSI/Broadsea/actions/workflows/achilles.yml) +## Contents + +- [Introduction](#introduction) + - [A Note on Docker Compose V2](#a-note-on-docker-compose-v2) + - [Broadsea Dependencies](#broadsea-dependencies) + - [Mac Silicon](#mac-silicon) +- [Broadsea - Quick start](#broadsea---quick-start) +- [Broadsea - Advanced Usage](#broadsea---advanced-usage) + - [.env file](#env-file) + - [Docker Secrets (New for 3.1)](#docker-secrets-new-for-31) + - [Remote Servers](#remote-servers) + - [Docker Profiles](#docker-profiles) + - [Traefik Dashboard](#traefik-dashboard) + - [SSL](#ssl) + - [Broadsea Content Page](#broadsea-content-page) + - [Vocabulary Loading](#vocabulary-loading) + - [OHDSI Web Applications](#ohdsi-web-applications) + - [CDM ETL Design and Execution](#cdm-etl-design-and-execution) + - [CDM Post Processing {#cdm-post-processing}](#cdm-post-processing-cdm-post-processing) + - [Evidence Generation](#evidence-generation) + - [Evidence Dissemination](#evidence-dissemination) +- [Shutdown Broadsea](#shutdown-broadsea) +- [Broadsea Intended Uses](#broadsea-intended-uses) +- [Troubleshooting](#troubleshooting) + - [View the status of the running Docker containers](#view-the-status-of-the-running-docker-containers) + - [Viewing Log Files](#viewing-log-files) +- [Hardware/OS Requirements for Installing Docker](#hardwareos-requirements-for-installing-docker) + - [Mac OS X](#mac-os-x) + - [Windows](#windows) + - [Docker for Windows Requirements](#docker-for-windows-requirements) + - [Docker Toolbox Windows Requirements](#docker-toolbox-windows-requirements) + - [Linux](#linux) + - [Linux Requirements](#linux-requirements) +- [License](#license) + ## Introduction Broadsea runs the core OHDSI technology stack using cross-platform Docker container technology. @@ -10,41 +45,41 @@ Broadsea runs the core OHDSI technology stack using cross-platform Docker contai This repository contains the Docker Compose file used to launch the OHDSI Broadsea Docker containers: -- OHDSI R HADES - in RStudio Server - - [OHDSI Broadsea R HADES GitHub repository](https://github.com/OHDSI/Broadsea-Hades/ "OHDSI Broadsea R HADES GitHub Repository") - - [OHDSI Broadsea R HADES Docker Hub container image](https://hub.docker.com/r/ohdsi/broadsea-hades "OHDSI Broadsea HADES Docker Image Repository") -- OHDSI Atlas - including WebAPI REST services - - [Atlas GitHub repository](https://github.com/OHDSI/Atlas "OHDSI Atlas GitHub Repository") - - [Atlas Docker Hub container image](https://hub.docker.com/r/ohdsi/atlas "OHDSI Atlas Docker Image Repository") - - [WebAPI GitHub repository](https://github.com/OHDSI/WebAPI "OHDSI WebAPI GitHub Repository") - - [WebAPI Docker Hub container image](https://hub.docker.com/r/ohdsi/webapi "OHDSI WebAPI Docker Image Repository") - - [Atlas application PostgreSQL database GitHub repository](https://github.com/OHDSI/Broadsea-Atlasdb "OHDSI Broadsea Atlas application PostgreSQL database GitHub Repository")cac - - [Atlas application PostgreSQL databbase Docker Hub container image](https://hub.docker.com/repository/docker/ohdsi/broadsea-atlasdb "OHDSI Broadsea Atlas application PostgreSQL database Docker Image Repository") - - SOLR based OMOP Vocab search -- OHDSI Ares - - [Ares GitHub repository](https://github.com/OHDSI/Ares "OHDSI Ares GitHub Repository") -- OHDSI Perseus (Experimental) - - [Perseus GitHub repository](https://github.com/OHDSI/Perseus "OHDSI Perseus GitHub Repository") +- OHDSI R HADES - in RStudio Server + - [OHDSI Broadsea R HADES GitHub repository](https://github.com/OHDSI/Broadsea-Hades/ "OHDSI Broadsea R HADES GitHub Repository") + - [OHDSI Broadsea R HADES Docker Hub container image](https://hub.docker.com/r/ohdsi/broadsea-hades "OHDSI Broadsea HADES Docker Image Repository") +- OHDSI Atlas - including WebAPI REST services + - [Atlas GitHub repository](https://github.com/OHDSI/Atlas "OHDSI Atlas GitHub Repository") + - [Atlas Docker Hub container image](https://hub.docker.com/r/ohdsi/atlas "OHDSI Atlas Docker Image Repository") + - [WebAPI GitHub repository](https://github.com/OHDSI/WebAPI "OHDSI WebAPI GitHub Repository") + - [WebAPI Docker Hub container image](https://hub.docker.com/r/ohdsi/webapi "OHDSI WebAPI Docker Image Repository") + - [Atlas application PostgreSQL database GitHub repository](https://github.com/OHDSI/Broadsea-Atlasdb "OHDSI Broadsea Atlas application PostgreSQL database GitHub Repository") + - [Atlas application PostgreSQL databbase Docker Hub container image](https://hub.docker.com/repository/docker/ohdsi/broadsea-atlasdb "OHDSI Broadsea Atlas application PostgreSQL database Docker Image Repository") + - SOLR based OMOP Vocab search +- OHDSI Ares + - [Ares GitHub repository](https://github.com/OHDSI/Ares "OHDSI Ares GitHub Repository") +- OHDSI Perseus (Experimental) + - [Perseus GitHub repository](https://github.com/OHDSI/Perseus "OHDSI Perseus GitHub Repository") Additionally, Broadsea offers limited support for services not specifically needed for OHDSI applications that often are useful: -- OpenLDAP for testing security in Atlas -- Open Shiny Server for deploying Shiny apps without a commercial license -- Posit Connect for sites with commercial Posit licenses, for deploying Shiny apps -- DBT for ETL design +- OpenLDAP for testing security in Atlas +- Open Shiny Server for deploying Shiny apps without a commercial license +- Posit Connect for sites with commercial Posit licenses, for deploying Shiny apps +- DBT for ETL design ### A Note on Docker Compose V2 -Throughout this README, we will show docker compose commands with the convention of `docker compose` (no hyphen), per the new Docker Compose V2 standard outlined by [Docker](https://docs.docker.com/compose/migrate/#docker-compose-vs-docker-compose). +Throughout this README, we will show docker compose commands with the convention of `docker compose` (no hyphen), per the new Docker Compose V2 standard outlined by [Docker](https://docs.docker.com/compose/migrate/#docker-compose-vs-docker-compose). **For Broadsea 3.1, you will need Docker version 1.27.0+.** ### Broadsea Dependencies -- Linux, Mac, or Windows with WSL -- Docker 1.27.0+ -- Git -- Chromium-based web browser (Chrome, Edge, etc.) +- Linux, Mac, or Windows with WSL +- Docker 1.27.0+ +- Git +- Chromium-based web browser (Chrome, Edge, etc.) ### Mac Silicon @@ -52,23 +87,23 @@ If using Mac Silicon (M1, M2), you **may** need to set the DOCKER_ARCH variable ## Broadsea - Quick start -- Download and install Docker. See the installation instructions at the [Docker Web Site](https://docs.docker.com/engine/installation/ "Install Docker") -- git clone this GitHub repo: +- Download and install Docker. See the installation instructions at the [Docker Web Site](https://docs.docker.com/engine/installation/ "Install Docker") +- git clone this GitHub repo: -``` +```shell git clone https://github.com/OHDSI/Broadsea.git ``` -- In a command line / terminal window - navigate to the directory where this README.md file is located and start the Broadsea Docker Containers using the below command. On Linux you may need to use 'sudo' to run this command. Wait up to one minute for the Docker containers to start. +- In a command line / terminal window - navigate to the directory where this README.md file is located and start the Broadsea Docker Containers using the below command. On Linux you may need to use 'sudo' to run this command. Wait up to one minute for the Docker containers to start. -``` +```shell docker compose --profile default up -d ``` -- In your web browser open the URL: `"http://127.0.0.1"` -- Click on the Atlas link to open Atlas in a new browser window -- Click on the Hades link to open HADES (RStudio) in a new browser window. - - The default RStudio userid is 'ohdsi' and the default password is located in the `./secrets/hades/HADES_PASSWORD` file. +- In your web browser open the URL: `"http://127.0.0.1"` +- Click on the Atlas link to open Atlas in a new browser window +- Click on the Hades link to open HADES (RStudio) in a new browser window. + - The default RStudio userid is 'ohdsi' and the default password is located in the `./secrets/hades/HADES_PASSWORD` file. ## Broadsea - Advanced Usage @@ -80,133 +115,66 @@ The .env file that comes with Broadsea has default and sample values. For advanc Broadsea leverages [Docker Secrets](https://docs.docker.com/engine/swarm/secrets/ "Docker Secrets") to handle sensitive passwords and secret keys. -*(In Broadsea 3.0, these were handled via plain-text environment variables, which is not best security practice)* +> In Broadsea 3.0, these were handled via plain-text environment variables, which is not best security practice Now in Broadsea 3.1, each sensitive password or secret key is to be stored in a file; the paths to these files is then set in the .env file per Section. Please refer to the default `./secrets` folder for examples on how to set up these files for your site. -#### Run Broadsea on a remote server +### Remote Servers In Section 1 of the .env file, set BROADSEA_HOST as the IP address or host name **(without http/https)** of the remote server. -### Docker profiles +### Docker Profiles Broadsea makes use of [Docker profiles](https://docs.docker.com/compose/profiles/ "Docker Profiles") to allow for either a full default deployment ("default"), or a more a-la-carte approach in which you can pick and choose which services you'd like to deploy. You can use this syntax for this approach, substituting profile names in: -``` -docker-compose --env-file .env --profile profile1 --profile profile2 .... up -d +```shell +docker compose --env-file .env --profile profile1 --profile profile2 ... up -d ``` -Here are the profiles available: - -- default - - atlas ("/atlas") - - WebAPI ("/WebAPI") - - AtlasDB (a Postgres instance for Atlas/WebAPI) - - HADES ("/hades") - - A splash page for Broadsea ("/") -- atlas-from-image - - Pulls the standard Atlas image from Docker Hub -- atlas-from-git - - Builds Atlas from a Git repo - - Useful for testing new versions of Atlas that aren't in Docker Hub -- webapi-from-image: - - Pulls the standard WebAPI image from Docker Hub - - Mac Silicon users, see "Mac Silicon" section above -- webapi-from-git - - Builds WebAPI from a Git repo - - Useful for testing new versions of WebAPI that aren't in Docker Hub - - Mac Silicon users, see "Mac Silicon" section above -- atlasdb - - Pulls the standard Atlas DB image, a Postgres instance for Atlas/WebAPI - - Useful if you do not have an existing Postgres instance for Atlas/WebAPI -- solr-vocab-no-import - - Pulls the standard SOLR image from Docker Hub - - Initializes a core for the OMOP Vocabulary specified in the .env file - - No data is imported into the core, left to you to run through the SOLR Admin GUI at "/solr" -- solr-vocab-with-import - - Pulls the standard SOLR image from Docker Hub - - Initializes a core for the OMOP Vocabulary specified in the .env file - - Runs the data import for that core - - Once complete, the solr-run-import container will finish with an exit status; you can remove this container -- ares - - Builds Ares web app from Ares GitHub repo - - Exposes a volume mount point for adding Ares files (see [Ares GitHub IO page](https://ohdsi.github.io/Ares/ "Ares GitHub IO")) -- content - - A splash page for Broadsea ("/broadsea") -- omop-vocab-pg-load - - Using OMOP Vocab files downloaded from Athena, this can load them into a Postgres instance (can be Broadsea's atlasdb or an external one) - - Rebuilds the CPTs using the CPT jar file from Athena, with UMLS API Key (see .env file Section 9) - - Creates the schema if necessary - - Runs copy command for each vocabulary CSV file - - Creates all necessary Postgres indices - - Once complete, the omop-vocab-load container will finish with an exit status; you can remove this container -- phoebe-pg-load - - For Atlas 2.12+, which offers Concept Recommendation options based on the [Phoebe project](https://forums.ohdsi.org/t/phoebe-2-0/17410 "Phoebe Project") - - Loads Phoebe files into an existing OMOP Vocabulary hosted in a Postgres instance (can be Broadsea's atlasdb or an external one) - - Note: your Atlas instance must use this OMOP Vocabulary as its default vocabulary source in order to use this feature - - Once complete, the phoebe-load container will finish with an exit status; you can remove this container -- openldap - - For testing security in Atlas, this Open LDAP container can be used to assess security needs - - You can specify a comma separated list of user ids and passwords - - This is not recommended for any production level setup -- open-shiny-server - - An open source version of Shiny Server, where you can drop shiny apps into a mounted folder. - - Recommended if your organization does not have a Posit Connect license. -- posit-connect - - For sites with commercial Posit Connect licenses, this can be useful for convenient publication of Shiny apps -- cdm-postprocessing - - For a specified CDM database, executes Achilles and DataQualityDashboard, then AresIndexer - - Useful for Atlas Data Sources reports and populating the files needed for the Ares application -- achilles - - Executes only Achilles for a specified CDM database -- dqd - - Executes only DataQualityDashboard for a specified CDM database -- aresindexer - - Executes only AresIndexer for a specified CDM database - - Only run this if Achilles and DataQualityDashboard have been executed -- dbt - - Sets up the dbt command-line tool for ETL design - +#### Standard Profiles + +| Profile | Description | +|----------------------|-------------| +| default | | +| atlas-from-image | | +| atlas-from-git | | +| webapi-from-image | | +| webapi-from-git | | +| atlasdb | | +| solr-vocab-no-import | | +| solr-vocab-with-import | | +| ares | | +| content | | +| omop-vocab-pg-load | | +| phoebe-pg-load | | +| openldap | | +| open-shiny-server | | +| posit-connect |