Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move db bootstrapping out of server startup #1897

Closed
lmsurpre opened this issue Feb 2, 2021 · 2 comments
Closed

Move db bootstrapping out of server startup #1897

lmsurpre opened this issue Feb 2, 2021 · 2 comments
Assignees
@lmsurpre @tbieste
Labels
P1 Priority 1 - Must Have persistence
Milestone
Sprint 2021-03

Comments

@lmsurpre
Copy link
Member

lmsurpre commented Feb 2, 2021 •
edited
Loading

Is your feature request related to a problem? Please describe.
We've had a few issues where users report that the default docker container is failing to start due to a timeout that occurs during bootstrapping of the derby databases. See #1554 and #1888

The problem is that the bootstrapping code inherits the global transaction timeout (120s by default) that we use for serving user requests. At the very least, the bootstrapping should have a different transaction timeout.

We've made it easy(ish) to get past the issue by setting the default transaction timeout via an env var (#1666).
However, the only reason we have the bootstrapping code to begin with is to make it dead simple to get our server up and running, so this kind of defeats the purpose.

Describe the solution you'd like

  1. Database bootstrapping should be moved to a bash script
  2. Package this bootstrapping script in our zip installer (and thus our docker image as well)
  3. Write a custom entrypoint for the docker image to make it simple to:

Initially we can just focus on derby, but I think it would make sense to extend that behavior to Db2 and PostgreSQL as well.

Describe alternatives you've considered
Either:
A. never perform bootstrapping during image startup; or
B. keep bootstrapping scoped to derby and force the use of the schema-tool cli and/or image for the 'real' dbs.

Acceptance Criteria
At least one acceptance criteria is included.

GIVEN a global transaction timeout of 60s
AND derby bootstrapping enabled
WHEN a user runs docker run -p9443:9443 ibmcom/ibm-fhir-server
THEN the server starts successfully
AND has a functional embedded derby database

To confirm it:

  1. build the project: mvn clean install -f fhir-parent -DskipTests
  2. build the docker image docker build fhir-install -t ibmcom/ibm-fhir-server
  3. run the docker image, ask it to use the bootstrap DB, and set a lower transaction timeout

docker run -e BOOTSTRAP_DB=true -e FHIR_TRANSACTION_MANAGER_TIMEOUT=60s ibmcom/ibm-fhir-server

Additional context
@lmsurpre lmsurpre mentioned this issue Feb 2, 2021
Closed
@lmsurpre lmsurpre self-assigned this Feb 12, 2021
@lmsurpre lmsurpre mentioned this issue Feb 17, 2021
Closed
@lmsurpre lmsurpre added this to the Sprint 2021-03 milestone Feb 18, 2021
@lmsurpre lmsurpre added the P1 Priority 1 - Must Have label Feb 23, 2021
@tbieste
Copy link
Contributor

tbieste commented Feb 24, 2021

Could this be simplified to just bootstrap an embedded derby database? (With just minimal ENVVARs to enable it.) Then all other database bootstrapping is done via the schema-tool cli. That lets the user give it a quick try via just docker run, but any "real" usage must use the schema tool to set up the DB first.

tbieste added a commit that referenced this issue Feb 24, 2021
@tbieste
Issue #1897 - Add bootstrap script as entrypoint
223346b 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
lmsurpre added a commit that referenced this issue Feb 24, 2021
@lmsurpre
issue #1897 - remove bootstrapping code from fhir-server
c33f62f 
to be replaced by script-based bootstrapping

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
lmsurpre added a commit that referenced this issue Feb 25, 2021
@lmsurpre
issue #1897 - update user's guide and datasource.xml
128d773 
* Remove all references to the fhir-server-config bootstrap properties
* Replace documentation about derby bootstrapping with a reference to
setting DB_BOOTSTRAP on the docker image

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
lmsurpre added a commit that referenced this issue Feb 25, 2021
@lmsurpre
issue #1897 - update user's guide and datasource.xml
0b09039 
* Remove all references to the fhir-server-config bootstrap properties
* Replace documentation about derby bootstrapping with a reference to
setting DB_BOOTSTRAP on the docker image

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>
tbieste added a commit that referenced this issue Feb 25, 2021
@tbieste
Issue #1897 - Add bootstrap script as entrypoint
b64b3b1 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
tbieste added a commit that referenced this issue Feb 25, 2021
@tbieste
Issue #1897 - Optionally bootstrap derby database
38d9963 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
tbieste added a commit that referenced this issue Feb 25, 2021
@tbieste
Issue #1897 - Optionally bootstrap derby database
30a9109 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
lmsurpre pushed a commit that referenced this issue Feb 25, 2021
@tbieste @lmsurpre
Issue #1897 - Optionally bootstrap derby database
5444143 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
tbieste added a commit that referenced this issue Feb 25, 2021
@tbieste
Issue #1897 - Less logging in integration workflow
699ea10 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
tbieste added a commit that referenced this issue Feb 25, 2021
@tbieste
Issue #1897 - Update after review comments
8926cef 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
@tbieste tbieste mentioned this issue Feb 25, 2021
Merged
lmsurpre added a commit that referenced this issue Feb 26, 2021
@lmsurpre
Merge pull request #1987 from IBM/issue-1897
13c3b1f 
issue #1897 - remove bootstrapping code from fhir-server
lmsurpre added a commit that referenced this issue Feb 26, 2021
@lmsurpre
Merge pull request #1987 from IBM/issue-1897
a397598 
issue #1897 - remove bootstrapping code from fhir-server
tbieste added a commit that referenced this issue Feb 26, 2021
@tbieste
Issue #1897 - Update after review comments
c6add49 
Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>
tbieste added a commit that referenced this issue Feb 26, 2021
@lmsurpre @tbieste
issue #1408 - move advanced bootstrapping into CI (#1992)
28a6ad4 
* issue #1897 - remove bootstrapping code from fhir-server

to be replaced by script-based bootstrapping

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>

* issue #1897 - update user's guide and datasource.xml

* Remove all references to the fhir-server-config bootstrap properties
* Replace documentation about derby bootstrapping with a reference to
setting DB_BOOTSTRAP on the docker image

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>

* Issue #1897 - Optionally bootstrap derby database

Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>

* issue #1408 - move advanced bootstrapping into CI

Given the decision to only support bootstrapping of the default tenant's
default datasource, we needed to manually bootstrap the tenant1
datasources for our multi-tenant integration tests.

Additionally, I updated the fhir-persistence-schema Main class to ensure
that the derby schema name is always APP (and that the pool-size is
always 1).

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>

* issue #819 - add undocumented 'resourceTypes' property to
fhir-persistence-schema

We were thinking of adding something like this under #819 anyway, but
for now I'm just trying to reduce the resource burden to pass our CI.

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>

* issue #819 - add undocumented 'resourceTypes' property to
fhir-persistence-schema

We were thinking of adding something like this under #819 anyway, but
for now I'm just trying to reduce the resource burden to pass our CI.

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>

* Update fhir-server/liberty-config/configDropins/defaults/datasource.xml

Signed-off-by: Lee Surprenant <lmsurpre@us.ibm.com>

* Issue #1408 - Copy config override for Windows CI test

Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>

* Issue #1408 - Copy config override for Windows CI test

Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>

* Issue #1408 - Copy config override for Windows CI test

Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>

* Issue #1408 - Copy config override for Windows CI test

Signed-off-by: Troy Biesterfeld <tbieste@us.ibm.com>

Co-authored-by: Troy Biesterfeld <tbieste@us.ibm.com>
@kmbarton423
Copy link
Contributor

Did the following on my Mac machine:

  1. mvn clean install -f fhir-parent -DskipTests
  2. docker build fhir-install -t ibmcom/ibm-fhir-server
  3. docker run -p 9443:9443 -e BOOTSTRAP_DB=true -e FHIR_TRANSACTION_MANAGER_TIMEOUT=60s ibmcom/ibm-fhir-server

Once fhir server was up and running, confirmed derby database usage:

  1. $healthcheck
  2. PUT to create a Patient
  3. GET to search on Patients

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lmsurpre lmsurpre

@tbieste tbieste

Labels
P1 Priority 1 - Must Have persistence
Projects
None yet
Milestone
Sprint 2021-03
Development

No branches or pull requests
4 participants
@prb112 @lmsurpre @tbieste @kmbarton423