Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(upgrade): Update repository on startup #1643

Merged
merged 25 commits into from May 18, 2020

Conversation

@benjamingeer
Copy link
Collaborator

@benjamingeer benjamingeer commented Apr 30, 2020

User story:

https://dasch.myjetbrains.com/youtrack/issue/DSP-165

  • Move the code from the upgrade package into webapi, translating the Bash scripts into Scala, and incorporating the code into ApplicationActor and the new classes RepositoryUpdater and RepositoryUpdatePlan.
  • Implement the necessary HTTP operations (download and upload the whole repository) in HttpTriplestoreConnector.
  • Add an upgrade plugin for PR #1615, to remove the instance of forbiddenResource from the Knora system data. (That PR only removed the ForbiddenResource class, and GraphDB's consistency checker rejects the instance if the class doesn't exist anymore.)
  • In ApplicationActor and its messages, rename the parameter skipLoadingOfOntologies (i.e. don't load ontologies from the triplestore into the cache) to ignoreRepository. When Knora is started for a test, before the test data is loaded into the triplestore, this is true. We can also use it to skip checking whether we should update the repository.
  • Rename some things in ApplicationMessages and TriplestoreMessages to distinguish clearly between the triplestore and the repository.
  • Use a sealed trait instead of an Enumeration for AppState, so the compiler can check that all cases are matched (e.g. in HealthRoute).
  • Update the GitHub action for testing repository update.
    • Add a script webapi/scripts/wait-for-knora.sh that polls Knora's /health route until Knora has finished starting up.
    • Add a Makefile target test-repository-update that loads old repository test data into Knora, restarts Knora, and waits for the update to complete.
  • Update docs.
@benjamingeer
Copy link
Collaborator Author

@benjamingeer benjamingeer commented May 7, 2020

@subotic I've commented out the GitHub CI upgrade-integration-tests workflow, because I removed the scripts that it was using. How do you want to do this now?

@subotic
Copy link
Collaborator

@subotic subotic commented May 7, 2020

We could add a new one based on the new feature, e.g., start the stack without api, load the v7 data, start the api and then check if it starts up.

For most of the steps there is a makefile target. Loading V7 data and checking if knora-api is running could be done with curl.

Would you like to try to do this?

@benjamingeer
Copy link
Collaborator Author

@benjamingeer benjamingeer commented May 7, 2020

@subotic

Would you like to try to do this?

I don't really understand how the Docker containers fit together, but I can try. How do I check whether the API started without errors?

@benjamingeer benjamingeer requested review from nemmK and subotic May 12, 2020
Copy link
Collaborator

@subotic subotic left a comment

Cool, thanks! Was a lot of work at the end. LGTM :-)

@benjamingeer benjamingeer self-assigned this May 13, 2020
@nemmK
nemmK approved these changes May 18, 2020
Copy link
Contributor

@nemmK nemmK left a comment

Maybe improve the documentation a bit as mentioned before otherwise all good :-)

@benjamingeer
Copy link
Collaborator Author

@benjamingeer benjamingeer commented May 18, 2020

@subotic @nemmK Thank you both for the reviews!

@benjamingeer benjamingeer merged commit 0127dca into develop May 18, 2020
8 checks passed
8 checks passed
Build Everything
Details
API Unit Tests
Details
API E2E Tests
Details
API Integration Tests
Details
JS Lib Tests (12.x)
Details
Upgrade Integration Tests
Details
Docs Build Test
Details
Publish to Dockerhub
Details
@benjamingeer benjamingeer deleted the wip/DSP-165-upgrade-at-startup branch May 18, 2020
@benjamingeer benjamingeer mentioned this pull request May 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants