Added a script to dump the hazard outputs needed for the risk #1096
Conversation
|
|
||
|
|
||
| def zipdir(dirpath): | ||
| "zip the contents of a directory in a zipfile and remove the directory" |
There was a problem hiding this comment.
The docstring style is inconsistent within this file, as well as the rest of the codebase.
|
Why is this tool needed? |
|
This was a request by Paul. Please consider the code here as a preview, we have yet to test it in real life use cases. I just opened a pull request since it was the easiest way to show it to Paul. The idea is to perform an expensive hazard calculation on Hope, zip the outputs and give them to a scientist which then can run a fast risk calculation on his laptop. I will open a bug later on. |
| # -*- coding: utf-8 -*- | ||
| # vim: tabstop=4 shiftwidth=4 softtabstop=4 | ||
|
|
||
| # Copyright (c) 2010-2013, GEM foundation |
There was a problem hiding this comment.
Technically, the copyright date should just be 2013.
foundation should be capitalized.
|
Can you please file the bug before this is merged? Any non-trivial pull request should have a Launchpad bug to document the change (including requirements, etc.). Without a description of the use cases, there is no way for a reviewer to know if the code is good or not. |
|
Fair enough. |
|
Small fixes and added the corresponding launchpad issue |
| with the same id. If you think that the hazard calculation on your database | ||
| is not important and can removed together with all of its outputs, then | ||
| remove it by using ``bin/openquake --delete-hazard-calculation`` (which | ||
| must be run by an user with sufficient permissions). Then run again |
There was a problem hiding this comment.
s/by an user/by a user/
|
@micheles The bug you linked doesn't really give any more details than your comments thus far. In fact it looks just like a copy/paste excerpt from one of your comments above. The information I'm interested in is about use cases, for example. Like, what types of calculations do users expect to dump/restore? All calculation modes, or just some of them? Do we expect to export statistical results as well, such mean and quantile hazard curves? That's what I would expect to see in the bug report. Also, I've marked the bug as |
| follow this procedure. | ||
|
|
||
| 1. go in the directory where this README is | ||
| 2. run ``source restore.sh`` |
There was a problem hiding this comment.
Ah, I see that it is generated below. Can you please add a comment explaining that this file is generated, and when/how?
|
Please rebase this branch on master; there are some spurious test failures. |
|
How this change relates to https://bugs.launchpad.net/oq-engine/+bug/1167693 ? It seems that the produced output is not compatible with the Please, also update the launchpad issue by linking this pull request. |
|
This is not directly related to https://bugs.launchpad.net/oq-engine/+bug/1167693. The approached used in that ticket is only for tests, it is not a valid mechanism for importing from a db to another one. This pull request solves the much easier problem of importing data into an empty database, when there are no id conflicts with the auto incremental fields. This is clearly not enough, but since the general problem is difficult it will be faced later on in another ticket. |
|
I'm trying to use the script, and I'm getting this error: $ python openquake/engine/tools/dump_hazards.py 111
Traceback (most recent call last):
File "openquake/engine/tools/dump_hazards.py", line 279, in
arg.user, arg.password)
File "openquake/engine/tools/dump_hazards.py", line 227, in main
dbname=dbname, user=user, password=password)
TypeError: argument 3 must be string, not None
|
| shutil.rmtree(out_dir) | ||
| os.mkdir(out_dir) | ||
|
|
||
| conn = psycopg2.connect(host=None if host == 'localhost' else host, |
There was a problem hiding this comment.
Why is the host set to None is host "localhost"? Why can't we just let it be "localhost"?
There was a problem hiding this comment.
Also, why aren't we leveraging the Django-managed DB connections for this? All other command line interface options (export, delete, etc.) do not require a hostname or credentials.
|
The dump/restore procedures have been completely rewritten; I also rewrote the bug description. |
|
LGTM |
Added a script to dump the hazard outputs needed for the risk
See https://bugs.launchpad.net/openquake/+bug/1156998 for the usage