Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB / Migration / Add step to set sequence value to current. #5003

Conversation

fxprunayre
Copy link
Member

Improvement proposed by @ianwallen in
#4781

With the JPA update in v4, it looks like the sequence were created even with the old
configuration of the domain classes. The migration use the sequenceName or name attribute to set
the new value.

The process can be triggered from the API documentation page with the
step name v3990.UpdateAllSequenceValueToMax.

image

The process is somehow slow due to the fact that there is no generic SQL
statement for updating the current sequence value. Iterating can take
time. In case of proxy timeout, the process can be restarted and it will
continue.

Improvement proposed by @ianwallen in
geonetwork#4781

With the JPA update in v4, it looks like the sequence were created even with the old
configuration of the domain classes. The migration use the `sequenceName` or `name` attribute to set
the new value.

The process can be triggered from the API documentation page with the
step name `v3990.UpdateAllSequenceValueToMax`.

The process is somehow slow due to the fact that there is no generic SQL
statement for updating the current sequence value. Iterating can take
time. In case of proxy timeout, the process can be restarted and it will
continue.
@fxprunayre fxprunayre added this to the 4.0.0-alpha.2 milestone Sep 10, 2020
@fxprunayre
Copy link
Member Author

A Postgresql alternative for the migration

SELECT setval('address_id_seq', (SELECT max(id) + 1 FROM address));
SELECT setval('csw_server_capabilities_info_id_seq', (SELECT max(idfield) FROM cswservercapabilitiesinfo));
SELECT setval('files_id_seq', (SELECT max(id) + 1 FROM files));
SELECT setval('group_id_seq', (SELECT max(id) + 1 FROM groups));
SELECT setval('gufkey_id_seq', (SELECT max(id) + 1 FROM guf_keywords));
SELECT setval('gufrat_id_seq', (SELECT max(id) + 1 FROM guf_rating));
SELECT setval('harvest_history_id_seq', (SELECT max(id) + 1 FROM harvesthistory));
SELECT setval('harvester_setting_id_seq', (SELECT max(id) + 1 FROM harvestersettings));
SELECT setval('inspire_atom_feed_id_seq', (SELECT max(id) + 1 FROM inspireatomfeed));
SELECT setval('iso_language_id_seq', (SELECT max(id) + 1 FROM isolanguages));
SELECT setval('link_id_seq', (SELECT max(id) + 1 FROM links));
SELECT setval('linkstatus_id_seq', (SELECT max(id) + 1 FROM linkstatus));
SELECT setval('mapserver_id_seq', (SELECT max(id) + 1 FROM mapservers));
SELECT setval('metadata_category_id_seq', (SELECT max(id) + 1 FROM categories));
SELECT setval('metadata_filedownload_id_seq', (SELECT max(id) + 1 FROM metadatafiledownloads));
SELECT setval('metadata_fileupload_id_seq', (SELECT max(id) + 1 FROM metadatafileuploads));
SELECT setval('metadata_id_seq', (SELECT max(id) + 1 FROM metadata));
SELECT setval('metadata_identifier_template_id_seq', (SELECT max(id) + 1 FROM metadataidentifiertemplate));
SELECT setval('operation_id_seq', (SELECT max(id) + 1 FROM operations));
SELECT setval('rating_criteria_id_seq', (SELECT max(id) + 1 FROM guf_ratingcriteria));
SELECT setval('schematron_criteria_id_seq', (SELECT max(id) + 1 FROM schematroncriteria));
SELECT setval('schematron_id_seq', (SELECT max(id) + 1 FROM schematron));
SELECT setval('selection_id_seq', (SELECT max(id) + 1 FROM selections));
SELECT setval('status_value_id_seq', (SELECT max(id) + 1 FROM statusvalues));
SELECT setval('user_id_seq', (SELECT max(id) + 1 FROM users));
SELECT setval('user_search_id_seq', (SELECT max(id) + 1 FROM usersearch));

@ianwallen
Copy link
Contributor

@fxprunayre
See comment from #4781
#4781 (comment)

If desired, the migration script can be modified to attempt to accommodate the postgresql query and possibly others however it may be difficult to test some of them.

@fxprunayre fxprunayre merged commit f64640d into geonetwork:4.0.x Sep 14, 2020
ianwallen added a commit to ianwallen/core-geonetwork that referenced this pull request Oct 8, 2020
…sion 4 of geonetwork (see geonetwork#5003)

update migration to support sequence or hybernate_sequence
josegar74 pushed a commit that referenced this pull request Nov 11, 2020
* Restore from history functionality

* Behavior when draft exists

* Fixed migration merge

* Removed MetadataStatusId Serialization and Deserialization and used existing format of /{metadataUuid}/status/{statusId:[0-9]+}.{userId:[0-9]+}.{changeDate}
Removed rollback option as a rollback is the same as a restore.  Also removed logic to delete status of rollback status.
Renamed view to view previous version and changed version and added new showStatusAfter api to support this.
Database
Added missing fre language for insert
Move migration inserts to 3110 and also added missing translation.

* Allow capability to recover deleted records
Changed PK key for metadatastatus so that it is a single key.
Added groupownname to geonet:info to assist with the recovery.
Add metadataStatus entry for deleted objects.
    Added UUID and Titles to the status so that the data is available for deleted records as well.

* removed duplicate status_id which is the same as the statusvalue.

* Fixed improper resolved conflict which accidentally converted file to ascci causing script to fail.

* Updated migration script
attempted to load titles but failed.

* More versioning changes
Changed titles to LinkedHashMap so that the order is static.
Updated status search api so that we can query based on id and uuid.
Updated API results
   Changed RelatedMetadataStatus to only return the id instead of the object.
   Return titles as a json object (applied temp hack until we upgrade jpa)
   renamed *statusEmpty to *stateEmpty for consistency
   Fixed bug with some fields being null when they should have values.
   Removed *statusDetail as we already had the *state fields from the parent.  If we want them named statusDetail then we should rename the fields in MetadataStatus.
Also updated migration script to attempt to fix primary key issue - but may need more work.

* Added @type annotations

* Fixed H2 migration primary key issues and also completed the rest field migrations issues (except for the the titles)
    Removed multikey Unique key on metadata status.  It was confusing H2 as it was using the same index for the PK and unique key making it difficult to remove the primary key.
Added missing commands in the oracle and mysql scripts.

* Revert grouprepository findOne as it was not required.
Fixed indentation
Get first language in case there are multiple for the dc:language.

* Another fix for groupowner - for sample data which does not belong to a group.
Renamed deleteAllByMetadataId back to deleteAllById_MetadataId - There are lots of others that should be renamed and this will be done as separate pr.
Revert api calls using uuid as it could cause issues with drafts.
    Updated api to support metadataUuid being a metadata id or uuid

* Removed sequence name from annotation - issue will be resolved in version 4 of geonetwork (see #5003)
update migration to support sequence or hybernate_sequence

* Refactor - cleanup duplicate code for showing before/after versions.

* Fix encoding issue. - File should be UTF-8

* Titles will be null on migration so allow for null values. Otherwise jpa calls may fail.

* Added some corrections so that the migration works with PostgreSQL

* Added xsl to extra titles for iso19115-3.2018

* Update permission check to use AccessManager.getGroups
Removed comments

Co-authored-by: Antonio C <antonio.cerciello@geocat.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants