Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade tgis db #306

Merged
merged 10 commits into from Mar 20, 2020
Merged

Upgrade tgis db #306

merged 10 commits into from Mar 20, 2020

Conversation

landam
Copy link
Member

@landam landam commented Jan 28, 2020

This PR introduces ability to upgrade TGIS DB version, only upgrade from version 2 (before introducing band references support, see #63) to 3 (with band ref support) is supported. Please note that this is work-in-progress PR. Main open issues:

  • introduce new module, eg. t.upgrade (*), or
  • add upgrade flag to existing module (and which?) Originally I was planning to add -u flag to t.connect, but it's a C module. For implementation Python is required.
  • 2) how to deal with raster views?

When upgrading db, 2 raster views must be recreated.

Option 1: duplicate raster_views.sql content in upgrade_db_2_to_3.sql (ugly)

Option 2: execute raster_views.sql by upgrade_temporal_database() only when upgrading from 2 to 3 (less ugly, but still not nice)

Option 3: upgrade_temporal_database() could drop and re-create all views regardless version upgrade

Option 4: your suggestions...

(*) Minimalistic t.upgrade.py

#!/usr/bin/env python3

import grass.temporal as tgis

tgis.init(skip_db_version_check=True)
    
dbif = tgis.SQLDatabaseInterfaceConnection()
dbif.connect()

tgis.upgrade_temporal_database(dbif)

Usage

Having TGIS DB created before introducing band reference support.

t.info -d
ERROR: Unsupported temporal database: version mismatch.

Do upgrade:

./t.upgrade.py

Now let's try t.info command again.

t.info -d
 +------------------- Temporal DBMI backend information ----------------------+
...
 | tgis_db_version .......... 3
...

Be aware that band references metadata are not magically introduced to existing datasets.

t.info test76
...
 | Number of registered bands:. None
t.info T33UVR_20181205T101401_B04 type=raster
...
 | Band reference:............. None

You need to create new register file containing band references and register raster maps by t.register again to rewrite existing metadata.

@landam landam mentioned this pull request Jan 28, 2020
2 tasks
@landam landam self-assigned this Jan 28, 2020
@landam landam added the enhancement New feature or request label Jan 28, 2020
@landam
Copy link
Member Author

landam commented Jan 28, 2020

c4349a5 implements "Option 3: upgrade_temporal_database() could drop and re-create all views regardless version upgrade" please test

@landam
Copy link
Member Author

landam commented Jan 28, 2020

Backup how to message improved. Now also t.upgrade command is mentioned.

ERROR: Unsupported temporal database: version mismatch.
The format of your actual temporal database is not supported any more.
From there, create a backup of your temporal database to avoid the loss of
your temporal data.
Recommended solution: Run t.upgrade command installed from GRASS Addons.
Alternative solution: You need to export it by restoring the GRASS GIS
version used for creating this DB.Notes: Use t.rast.export and
t.vect.export to make a backup of your existing space time datasets.To safe
the timestamps of your existing maps and space time datasets, use
t.rast.list, t.vect.list and t.rast3d.list. You can register the existing
time stamped maps easily if you export columns=id,start_time,end_time into
text files and use t.register to register them again in new created space
time datasets (t.create). After the backup remove the existing temporal
database, a new one will be created automatically.
Supported temporal database version is: 2
Current temporal database info:
DBMI interface:..... sqlite3
Temporal database:.. /tmp/test/PERMANENT/tgis/sqlite.db

lib/python/temporal/core.py Outdated Show resolved Hide resolved
lib/python/temporal/core.py Outdated Show resolved Hide resolved
lib/python/temporal/core.py Outdated Show resolved Hide resolved
lib/python/temporal/core.py Outdated Show resolved Hide resolved
lib/python/temporal/core.py Outdated Show resolved Hide resolved
lib/python/temporal/core.py Outdated Show resolved Hide resolved
landam and others added 3 commits January 28, 2020 16:12
Co-Authored-By: Veronica Andreo <veroandreo@gmail.com>
Co-Authored-By: Veronica Andreo <veroandreo@gmail.com>
Co-Authored-By: Veronica Andreo <veroandreo@gmail.com>
@landam landam requested a review from huhabla January 28, 2020 15:15
@landam
Copy link
Member Author

landam commented Feb 27, 2020

Previous version of message is printed only when running very old tgis version (1). For TGIS created with GRASS76+ only t.upgrade solution message is printed.

t.info test
ERROR: Unsupported temporal database: version mismatch.
The format of your actual temporal database is not supported any more.
Please create a backup of your temporal database to avoid the lossing data.
SOLUTION: Run t.upgrade command installed from GRASS Addons in order to
upgrade your temporal database.
Supported temporal database version is: 2
Current temporal database info:
DBMI interface:..... sqlite3
Temporal database:.. /tmp/x/PERMANENT/tgis/sqlite.db

@veroandreo @neteler Please review.

@landam landam merged commit 61b8d40 into OSGeo:master Mar 20, 2020
@landam landam deleted the tgis_upgrade_2_3_new branch March 20, 2020 15:13
petrasovaa added a commit to petrasovaa/grass that referenced this pull request Apr 14, 2020
@petrasovaa
Copy link
Contributor

I would expect you should at least run the existing tests before you merge pretty large change. BTW, consider using some Python static code analysis in your editor, it can very easily catch simple mistakes like this.

@landam
Copy link
Member Author

landam commented Apr 14, 2020

Ops, thanks for the fix.

@neteler neteler changed the title WIP upgrade tgis db Upgrade tgis db Feb 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants