1376 make ontohub org database consistent #1377
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a) Hets closes the HTTP connection after a minute, which results in timeouts. If a timeout occurs, we try again to get the provers list twice. b) Sometimes another error occurs (Hets response not usable). In both cases we print the ID of the OntologVersion to know which ones fail. This way, we can process them in another data migration.
| @@ -3,7 +3,7 @@ def up | |||
| obo_ontologies = Ontology.joins(:ontology_version). | |||
| where(ontology_versions: {file_extension: '.obo'}) | |||
| obo_ontologies.each do |ontology| | |||
| ontology.sentences.find_each { |s| s.set_display_text! } | |||
| ontology.sentences.select([:id, :text]).find_each(&:set_display_text!) | |||
There was a problem hiding this comment.
How about a fancy-string: %i(id text)?
| @@ -0,0 +1,14 @@ | |||
| class SetBasepathWhereNil < ActiveRecord::Migration | |||
| def self.up | |||
| OntologyVersion.where(basepath: nil).find_each do |ov| | |||
There was a problem hiding this comment.
You could optimize this query by auto-loading the ontology (includes), and possibly by pre-selecting the fields.
There was a problem hiding this comment.
With select and includes, the columns don't get updated (no matter in what order I use those two methods). So I just use includes in 232f6a3.
|
Apart from my last comment: 👍 |
|
👍 |
eugenk
added a commit
that referenced
this pull request
Jun 22, 2015
…consistent 1376 make ontohub org database consistent
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This shall fix #1376.
Actually, only the first commit (Set basepath where it is nil. 671a2be) restores consistency. The other ones make sure that the data migrations pass. Since failing data migrations were the main reason of the issue, I put them into this PR as well.
Note:
The data migrations should be run manually with
RAILS_ENV=production nohup rake data:migrate &to see the IDs of the failing OntologyVersions (see the third commit's message).This way, 138 + 4350 OntologyVersions don't get a list of provers (timeout + other error).
You can find the file containing the logged failing IDs at
develop.ontohub.org:~ontohub/data_migration_provers_failures