Skip to content

Commit

Permalink
Resolve all Job Manager HIGHs & update docs [BW-752] (#729)
Browse files Browse the repository at this point in the history
  • Loading branch information
aednichols committed Jul 29, 2021
1 parent 7ea6445 commit b713321
Show file tree
Hide file tree
Showing 2 changed files with 27 additions and 37 deletions.
60 changes: 25 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,21 @@
[![Github](https://img.shields.io/badge/Docker%20Image-dsub%20shim%20API-blue.svg?style=flat-square)](https://cloud.docker.com/u/databiosphere/repository/docker/databiosphere/job-manager-api-dsub)
[![Github](https://img.shields.io/badge/Docker%20Image-cromwell%20shim%20API-blue.svg?style=flat-square)](https://cloud.docker.com/u/databiosphere/repository/docker/databiosphere/job-manager-api-cromwell)

## Lifecycle notice

Job Manager is in maintenance mode and is not recommended for new projects.

dsub support is deprecated and will be removed in a future release.

The last release of Job Manager to support dsub without compromise is [1.5.7](https://github.com/DataBiosphere/job-manager/releases/tag/v1.5.7).

## User facing documentation

Welcome to the Job Manager repository! If you're a developer you're in the right place.

However, if you just want to try out or deploy Job Manager, you will probably find our user and deployment focused content in
our ReadTheDocs pages: https://data-biosphere-job-manager.readthedocs.io/en/latest/

### Try it out, NOW!

The easiest way to try out Job Manager is to use the [getting started script](https://data-biosphere-job-manager.readthedocs.io/en/stable/GettingStarted/QuickStart)

## Welcome

See the [development guide](#development) below.
Expand All @@ -36,20 +40,6 @@ The Job Manager aspires to bring ease and efficiency to developing and debugging
* Aborting workflows
* Clean, intuitive UX based on material design principles

### Future Features
* Dynamic grouping, filtering, and drill-down
* Re-launching workflows
* Simplified troubleshooting of failed workflows
* Improved UI design

## Roadmap

The current code is a work in progress towards an alpha release and as such has started with core features: connecting to both backends, visualizing workflow and task status and metadata, quick access to log files, and simple filtering.

The near-term roadmap includes improvements to failure troubleshooting, creating a robust dashboard for grouping jobs and seeing status overviews, and improving handling of widely scattered workflows.

We envision a product with user-customizable views of jobs running, insights into workflow compute cost, the ability to re-launch jobs, and the potential to make custom reports about the jobs that have been run.

## Architecture Overview

The Job Manager [defines an API](api/jobs.yaml) via OpenAPI. An Angular2 UI is provided over the autogenerated Typescript bindings for this API. The UI is configurable at compilation time to support various deployment environments (see [environment.ts](ui/src/environments/environment.ts)), including auth, cloud projects, and label columns.
Expand Down Expand Up @@ -175,31 +165,31 @@ For `cromwell` server documentation, see [servers/cromwell](servers/cromwell/REA
### How to build
From v0.2.0, Job Manager starts to release stock docker images on [DockerHub](https://hub.docker.com/u/databiosphere)
- Setting the docker tag first in bash, e.g. `TAG="v0.1.0"`
Starting with release v1.6.0, Job Manager docker images are on [GCR](https://console.cloud.google.com/gcr/images/broad-dsp-gcr-public).
- To build the `job-manager-ui` image with `$TAG` from the root of this Github repository:
- Configure Docker to authenticate with GCR
```
docker build -t job-manager-ui:$TAG . -f ui/Dockerfile
gcloud config set account username@broadinstitute.org
gcloud auth configure-docker us.gcr.io
```
- **Cromwell:** To build the `job-manager-api-cromwell` image with `$TAG` from the root of this Github repository:
- Set the Docker tag first in bash, e.g. `TAG="v0.1.0"`
- To publish the `job-manager-ui` image with `$TAG` from the root of this Github repository:
```
docker build -t job-manager-api-cromwell:$TAG . -f servers/cromwell/Dockerfile
docker build -t us.gcr.io/broad-dsp-gcr-public/job-manager-ui:$TAG . -f ui/Dockerfile
docker push us.gcr.io/broad-dsp-gcr-public/job-manager-ui:$TAG
```
- **dsub:** To build the `job-manager-api-dsub` image with `$TAG` from the root of this Github repository:
- To publish the `job-manager-api-cromwell` image with `$TAG` from the root of this Github repository:
```
docker build -t job-manager-api-dsub:$TAG . -f servers/dsub/Dockerfile
docker build -t us.gcr.io/broad-dsp-gcr-public/job-manager-api-cromwell:$TAG . -f servers/cromwell/Dockerfile
docker push us.gcr.io/broad-dsp-gcr-public/job-manager-api-cromwell:$TAG
```
### Add a github release pointing to the dockerhub images
From v0.2.0, each release in Github will also release 3 corresponding docker images on Docker Hub:
### Add a Github release pointing to the GCR images
- [job-manager-ui](https://hub.docker.com/r/databiosphere/job-manager-ui/)
- [job-manager-api-cromwell](https://hub.docker.com/r/databiosphere/job-manager-api-cromwell/)
- [job-manager-api-dsub](https://hub.docker.com/r/databiosphere/job-manager-api-dsub/)
From v1.6.0, each release in Github will also release 2 corresponding Docker images on GCR:
For a long-term plan, we will set up a docker build hook so the release process can be more automated.
- [job-manager-ui](https://console.cloud.google.com/gcr/images/broad-dsp-gcr-public/US/job-manager-ui)
- [job-manager-api-cromwell](https://console.cloud.google.com/gcr/images/broad-dsp-gcr-public/US/job-manager-api-cromwell)
4 changes: 2 additions & 2 deletions ui/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ ADD ./ /job-manager

RUN /bin/bash -c scripts/rebuild_swagger.sh

FROM node:lts
FROM us.gcr.io/broad-dsp-gcr-public/base/nodejs:12-debian

WORKDIR /ui

Expand All @@ -21,7 +21,7 @@ RUN npm install

RUN /ui/node_modules/.bin/ng build --prod

FROM nginx:stable
FROM nginx:mainline-alpine

COPY --from=1 /ui/dist /ui/dist
ADD ./ui/nginx.prod.conf /etc/nginx/nginx.conf

0 comments on commit b713321

Please sign in to comment.