Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VIVO-1443: non destructive vivo home untarring #2

Merged
merged 30 commits into from Jan 28, 2021
Merged

VIVO-1443: non destructive vivo home untarring #2

merged 30 commits into from Jan 28, 2021

Conversation

ghost
Copy link

@ghost ghost commented Jan 27, 2021

This will afford to keep changes made to various VIVO home files if restarting tomcat or redeploying another build.

Approach:
During untar of packed VIVO home, a checksum digest is built from the files in the tar. After untar is complete a checksum digest file is written in VIVO home. The checksum digest is then read before untar and used to check if an existing file in the VIVO home directory has been modified. If the file has not been modified it will be replaced with the file from the tar file. A log will indicate if a file has been preserved to assist in redeploy.

For upgrade from previous versions and there is a desire to maintain the VIVO home directory we can provide a script or instructions on creating the md5 checksum.

If during development and you want all modified files overwritten, the checksum digest file can be deleted.

VIVO home checksum digest can manually be created.

find /vivo/home -type f | cut -c3- | grep -E '^bin/|^config/|^rdf/' | xargs md5sum > /vivo/home/digest.md5

@ghost ghost marked this pull request as draft January 27, 2021 23:52
@ghost ghost marked this pull request as ready for review January 28, 2021 00:23
@ghost
Copy link
Author

ghost commented Jan 28, 2021

digest.md5

ee266969f3df75fb553d156508ad7716 *rdf/applicationMetadata/firsttime/classgroups.rdf
e1260eb94b2b41c74b250bbb43fac1a0 *rdf/display/everytime/vivoQrCodeDataGetter.n3
62c76ef1dcf82fd32be1e195dc5d2636 *rdf/display/everytime/homePageDataGetters.n3
2a47b4cdf4f1e48615733b8096ebee45 *rdf/abox/filegraph/continents.n3
e066ae418f929cb9f03c62bd4a2c23c9 *rdf/abox/filegraph/us-states.rdf
af7d40688c37a7d253f78837841c4b7f *config/default.runtime.properties
6fc2782afee8d0273f1a21dd2827a928 *rdf/display/everytime/vivoListViewConfig.rdf
148bca3fb394b645930912a059f51590 *rdf/display/firsttime/pageList_editableStatements.n3
1cca7b272d3259004c741d867f67aeb7 *rdf/display/firsttime/application.owl
949f5f0b750b43b9fbc6dc363b7e8f37 *rdf/display/firsttime/aboutPage.n3
2f2f6d713f36ea275e2bc82a653f6a23 *rdf/tbox/filegraph/vitro-0.7.owl
8011146949f3c37f535f7d892d2ddd89 *rdf/displayTbox/everytime/displayTBOX.n3
af5fd8b9c6dde832c0406ef88333b693 *rdf/display/everytime/orcidInterfaceDataGetters.n3
5fc38cd358bdcd901a44021f494d5974 *rdf/tbox/filegraph/ontologies.owl
128aff292bfd075904c195d72badba70 *rdf/abox/filegraph/dateTimeValuePrecision.owl
d5d4b651f3e149f800f969ad282dbfdb *rdf/display/firsttime/PropertyConfig.n3
c80fa79631407ad8e3783bbf9a11985b *bin/jena3tools.jar
453e3f5c96c94b719cf0da8ba6c36ed0 *rdf/display/everytime/searchIndexerConfigurationVivo.n3
07daa0e613a48645f354eb146abf972b *rdf/display/everytime/dataGetterLabels.n3
55c0c11ed8050757d62c6cc6711e0170 *rdf/display/firsttime/profilePageType.n3
9f1033a09b456b48f8b142caf4884e76 *rdf/applicationMetadata/firsttime/initialSiteConfig.rdf
19d2197439969aad5616c45dea17d8b4 *rdf/abox/filegraph/README.md
31790d1cb1821ea0d5cddda2abcb018c *rdf/display/everytime/vivoSearchProhibited.n3
ded482569c2c351a1ac2f0ffb7f96e86 *rdf/display/everytime/vivoConceptDataGetters.n3
9fd5824beb54633a8f3b4f573bf770dc *rdf/abox/filegraph/vocabularySource.n3
d9c91dbcc9549db917c8213f4c575ca2 *rdf/abox/filegraph/documentStatus.owl
5b94ab70e6ae856450a65f57cf0d9c1d *rdf/abox/filegraph/academicDegree.rdf
5dc8e1fdaff2188ab925c79f1e3e1435 *rdf/abox/firsttime/geopolitical.ver1.1-11-18-11-individual-labels.rdf
7a5511e918c155d40ce53e9ca273fb8e *bin/jena2tools.jar
0684dc6666cfdca6fcc87f7f6815f62c *rdf/display/everytime/localeSelectionGUI.n3
5ba413ecf3da0efdef1d078fafc4c712 *rdf/display/firsttime/menu.n3
4129c19272ad3048f8f5f0f0a9198fb9 *rdf/tbox/filegraph/vivo.owl
02dce536939065b1a1ba3cb7269e1d97 *rdf/display/everytime/permissions.n3
debdebcc905b8c79089a26fac154cf13 *rdf/displayDisplay/everytime/displayDisplay.n3
cd22f82115e0e035147f4ecb918b038e *rdf/tbox/firsttime/vitroAnnotations.n3
4ddc444463103842d6cd9377f3f1eac5 *rdf/tbox/filegraph/vitroPublic.owl
04ab039199e10ec253a8baf0ec925a6d *rdf/tbox/filegraph/README.md
5acc833f5aef7e85b6e56eec448dee4f *rdf/display/everytime/n3ModelChangePreprocessors.n3
d81f41627ec81fd862f4f948e2f15c03 *rdf/display/everytime/vivoOrganizationDataGetters.n3
66c4569c3a5e2e56aa90419e0e2027fc *rdf/display/everytime/displayModelListViews.rdf
15dad4de37df17de56dbbf059146d606 *rdf/display/everytime/vitroSearchProhibited.n3
8afed03ee1368414a255c84b4713db81 *rdf/display/everytime/pageList.n3
78f675ff1e41213236470c0e554c9807 *rdf/applicationMetadata/firsttime/propertygroups.rdf
5336aaadcdf3de5400b820641924929c *rdf/tbox/firsttime/README.md
a570929b84a1796dc8fac422ca128960 *rdf/auth/everytime/permission_config.n3
4ac4f2d25bc5306b185aff7194a25ef5 *rdf/display/everytime/searchIndexerConfigurationVitro.n3
36d62de3324beae829b813131e5ab407 *config/default.developer.properties
ea71480dbf677e95dfd912078c46479f *rdf/display/firsttime/PropertyConfigSupp.n3
d30a6357deb9d7c20db0ecc21b6f46fd *rdf/abox/filegraph/validation.n3
9894f799e036b3678610249f917393d5 *config/default.applicationSetup.n3
dd24a98d7eaf13a478d76135409e6ae6 *rdf/abox/filegraph/geopolitical.abox.ver1.1-11-18-11.owl

@ghost
Copy link
Author

ghost commented Jan 28, 2021

Applied changes requested. Will this be enough validation of the checksum file?

@ghost ghost requested a review from awoods January 28, 2021 04:24
@awoods awoods merged commit c4ae002 into awoods:vivo-1443 Jan 28, 2021
awoods added a commit that referenced this pull request Feb 5, 2021
* Create vivo/home on application start-up

- Add logic for untarring vivo-home

Related to: https://jira.lyrasis.org/browse/VIVO-1443

* Require runtime.properties and applicationSetup.n3 be in 'config/' dir

Related to: https://jira.lyrasis.org/browse/VIVO-1443

* Support loading config files with or without 'default' prefix
* Prioritize without the prefix

Related to: https://jira.lyrasis.org/browse/VIVO-1443

* Add leading slash to location of vivo-home.tar resource path
* This allows for deployment in both Tomcat and Jetty

Related to: https://jira.lyrasis.org/browse/VIVO-1443

* Ensure VIVO_HOME is populated whether it is empty or not

Related to: https://jira.lyrasis.org/browse/VIVO-1443

* Require common properties to be in JNDI

Properties include:
- vitro/home
- vitro/appName
- vitro/rootUserAddress
- vitro/defaultNamespace

Related to: https://jira.lyrasis.org/browse/VIVO-1443

* VIVO-1443: non destructive vivo home untarring (#2)

* Upgrade Jena version to 3.16.0 (vivo-project#196)

Related to: https://jira.lyrasis.org/browse/VIVO-1943

* remove example-settings.xml
* update home directory untar non-destructive
* checksum digest to retain modified files
* compare checksum from digest with existing file
* overwrite files that have not changed
* add command to manually generate checksum digest
* simplify parsing checksum digest using pattern

Co-authored-by: Andrew Woods <awoods@lyrasis.org>
Co-authored-by: Andrew Woods <awoods@duraspace.org>

* not overwrite existing file if same as tar entry (#3)

* Upgrade Jena version to 3.16.0 (vivo-project#196)

Related to: https://jira.lyrasis.org/browse/VIVO-1943

* remove example-settings.xml
* update home directory untar non-destructive
* checksum digest to retain modified files
* compare checksum from digest with existing file
* overwrite files that have not changed
* add command to manually generate checksum digest
* simplify parsing checksum digest using pattern
* do not overwrite file if same as already exists

Co-authored-by: Andrew Woods <awoods@lyrasis.org>
Co-authored-by: Andrew Woods <awoods@duraspace.org>

* Update log messages for clarity

Co-authored-by: Andrew Woods <awoods@duraspace.org>
Co-authored-by: William Welling <wwelling@library.tamu.edu>
Co-authored-by: Gross, Benjamin <benjamin.gross@clarivate.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants