[Vivo 1918] i18n: Reload firsttime files on start-up if changed

[dofeldsc]

JIRA Issue

What does this pull request do?

This PR adds the functionality requested in the JIRA-Ticket
On the firsttime Start-Up a backup of the firsttime files is saved to a new model.
On every Start-Up these backup will be compared to the current state of the firsttime files. If the files have changed, try to apply
these changes to the user model. So if you activate a new language or change stuff in some firsttime files, you will no longer need to do a firsttime Start-Up of your system, with a normal restart the changes will be applied.

How should this be tested?

You need a fresh system (firsttime Start-Up) - for that, you can delete the tdb* directories in vivo_home (if you do not want to set up a whole new system)
Start the system so the new backup models will be created. After this, change some firsttime files and restart the system.
The changes should now be available without a new firsttime Start-Up.

good examples:

• vivo_home_dir/rdf/i18n/en_US/tbox/firsttime/vitroAnnotations_en_US.n3
• vivo_home_dir/rdf/i18n/en_US/display/firsttime/menu_en_US.nt

[awoods]

Nice work, @dofeldsc !
I still need to test the functionality, but here is the code review.

[awoods]

What is the problem with "display" models?

[dofeldsc]

One triple was marked as a change on every startup
<https://some.othernamespace.edu/vivo/individual/portal1> <http://vitro.mannlib.cornell.edu/ns/vitro/0.7#rootTab> [] .
But i should not change.
So i ignore/remove this one in the diff. Maybe you have a better idea, probably isAnon would work here too?!

[awoods]

Why are statements with anonymous subjects or objects being removed for further consideration?

[dofeldsc]

I had a problem with this statement
<http://vitro.mannlib.cornell.edu/ontologies/display/1.1#ProfilePageType> <http://www.w3.org/2002/07/owl#equivalentClass> [ a <http://www.w3.org/2002/07/owl#Class> ; <http://www.w3.org/2002/07/owl#oneOf> [ <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> [ <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> () ] ]
It was marked as a change on every startup.
And Brians suggestion was to ignore any statements where either the subject or the object is a blank node (.isAnon())

[gneissone]

Just taking a look for the first time here, so forgive me if I'm out of the loop.
My understanding is, each time upon startup, the contents of the firsttime files are saved into a new graph, which includes the suffix 'Firsttime.'

Taking a look at the 'Available Jena models' page in the UI, I see the new models, but it's not obvious what their purpose is or how they are organized. Backups of the configuration models, like displayMetadata, are shown on the main store models page while the live graph is on a separate page.

I am also noticing that these backup triples appear in query results, which is confusing.

If we change a value for these triples in the UI (e.g. a class group) the value is updated in the primary graph, but not the backup, as expected. Is there any possibility of conflicts here? Or does the application effectively ignore the backups during operation?

Some suggestions after the first pass review:

• Rename the backup graphs so their purpose is more obvious - perhaps include the word 'backup' in the name.
• Update the manage jena models page to separate the backup models from the functional ones (or exclude them).
• Confirm backup models are only used during the startup process and don't conflict with the 'live' triples anywhere.

[awoods]

Thanks, @gneissone .
Regarding your suggestions,

Rename the backup graphs so their purpose is more obvious - perhaps include the word 'backup' in the name.

This seems like something that could reasonable be changed in this PR.

Update the manage jena models page to separate the backup models from the functional ones (or exclude them).

This may be best for a follow-on ticket.

[awoods]

This PR appears to work; however, given the number of unrelated bugs in the i18n:siteAdmin functionality, it is a big challenging to test. For example, when using multiple languages, most of the "page management" features fail with errors.

The tests that I was able to successfully perform were:

• edit page-title text in UI: it remains after restart - Good
• edit page-title text in firsttime files: it is reloaded and used on restart - Good
• change menu order of page both in UI and in firsttime files: the UI order takes precedence and is used - Good

[awoods]

I was able to locally patch (see: gist) VIVO-i18n in order to perform the following test for this pull-request:

• Build and install this pull-request, while leaving Tomcat stopped
• Delete any existing triplestores (tdbContentModels/ and tdbModels/)
• Start Tomcat
• Log in as site admin
• Navigate to: "Site Admin" -> "Page management"
• Click the pencil icon next to the "Capability Map" row
• Change "Menu Item Name" to "Capability Map-UI1"
• Verify tabs shows new menu item name
• In firsttime file: $VIVO_HOME/rdf/i18n/en_US/display/firsttime/menu_en_US.nt
  • Change two "linkText" entries:
    • "Events" to "Events-file1"
    • "Capability Map" to "Capability Map-file1"
• Restart Tomcat
• Verify that the tabs show:
  • "Events-file1" and
  • "Capability Map-UI1" -- :( I am seeing "Capability Map-file1"

If changes happen via the UI and the firsttime file, the UI should take precedence.

[dofeldsc]

I am also noticing that these backup triples appear in query results, which is confusing.

That really shouldn't be the case, what's the best way to prevent it?

If we change a value for these triples in the UI (e.g. a class group) the value is updated in the primary graph, but not the backup, as expected. Is there any possibility of conflicts here? Or does the application effectively ignore the backups during operation?

They should be ignored, I only use them here to check and update them when changes are made to the firsttime files (after a restart). Or are new models "automatically" accessed somewhere?

• Rename the backup graphs so their purpose is more obvious - perhaps include the word 'backup' in the name.

I added the word 'backup' at the end of the model name.

[dofeldsc]

• "Capability Map-UI1" -- :( I am seeing "Capability Map-file1"
  If changes happen via the UI and the firsttime file, the UI should take precedence.

Right, I can confirm the error :( I'm trying to find a solution for this problem. In the algorithm described at the beginning (see JIRA-ticket) this would not have happened but new language files would not be loaded neither when something in the model was changed in the UI. Maybe for each change compare subject and predicate and only import them if they have not changed between firsttime-backup and UI-model? This would be my first approach

[awoods]

Maybe for each change compare subject and predicate and only import them if they have not changed between firsttime-backup and UI-model?

That sounds right.

[dofeldsc]

I have fixed the bug, so changes in the UI should no longer be overwritten by the changes from the files.
But I'm not sure how to configure the backup-models so that they can't be queried via the SPARQL-API.

As an example, if i now query something like this over the SPARQL-API UI in Vivo

SELECT ?title
WHERE
{
<http://vitro.mannlib.cornell.edu/ontologies/display/1.1#CapabilityMap> <http://vitro.mannlib.cornell.edu/ontologies/display/1.1#title> ?title
}

I will get the value set in the firsttime-files. Before the backup models you did not get a result. But on the actual page everything is displayed correctly.

[awoods]

The updates work as advertised. It is sometimes difficult to test because of the issue you noted in the JIRA:

The second probably has nothing to do with this ticket. Can someone confirm that with two available languages after a firsttime startup, the page
"Site Admin -> Page management -> Click the pencil icon next to the 'Capability Map' row" or any other page on this site, is not fully translated when the language is changed? I see this behavior also in the i18n-sprint branch.

..and because a patch is required, (gist).

One minor comment remaining on this PR.

[awoods]

@gneissone : if you can verify the updates with your tests, we can get this merged.

[gneissone]

I followed the test procedure described by Andrew and things are working as expected. I also tried editing class groups and had success there as well. Weirdness with the behavior in a multi-language environment I believe is unrelated to this PR.