station map & documentation & upgrade db

cleaned station map & added some larger buffer around stations to prevent basemap from crashing
added the HTML station map to the documentation

!! added the upgrade_db required to switch from float to double in the databases' station tables (X and Y) fields and the corresponding doc.

CONTRIBUTORS.txt

@@ -1,4 +1,5 @@
 Corentin Caudron
+Clare Donaldson
 Raphaël De Plaen
 Thomas Lecocq
 Aurélien Mordret

doc/releasenotes/msnoise-1.4.rst

@@ -16,10 +16,10 @@ Release notes:
 
 Introduction
 ------------
-Almost a year after the last major release (:doc:`msnoise-1.3`) we are proud to
-announce the new :doc:`msnoise-1.4`. It is a **major** release, with a massive
-amount of work since the last one: in `GitHub numbers
-<https://github.com/ROBelgium/MSNoise/graphs/contributors?from=2015-04-01&to=2016-03-20&type=c>`_
+Just over a year after the last major release (:doc:`msnoise-1.3`) we are proud
+to announce the new :doc:`msnoise-1.4`. It is a **major** release, with a
+massive amount of work since the last one: in `GitHub numbers
+<https://github.com/ROBelgium/MSNoise/graphs/contributors?from=2015-04-01&to=2016-04-20&type=c>`_
 , it's over XXX commits and about XXX new lines of code and documentation added!
 
 MSNoise 1.4 introduces **four major new features** : a new ultra-intuitive
@@ -46,12 +46,12 @@ several users/friends:
 Thanks to all for using MSNoise, and please, let us know why/how you use it
 (and please cite it!)!
 
-To date, we found/are aware of 11 publications using MSNoise ! That's the best
+To date, we found/are aware of 12 publications using MSNoise ! That's the best
 validation of our project ever ! See the full list on the
 `MSNoise website <http://www.msnoise.org/they-cite-msnoise/>`_.
 
 
-*Thomas Lecocq & Corentin Caudron*
+*Thomas*
 
 
 ~~~~
@@ -180,6 +180,10 @@ rows show the "REF" stack and its FTAN image. The three rows are:
 * B: PWS stack of  "windows" to "daily" - Linear stack of "daily" to "ref"
 * C: PWS stack of  "windows" to "daily" - PWS stack of "daily" to "ref"
 
+To obtain an LIN-LIN or PWS-PWS stack, simply set ``stack_method`` = 'linear' or
+'pws', respectively when running the ``compute_cc`` and ``stack`` steps.
+And for mixed cases LIN-PWS or PWS-LIN, edit the config between the two steps!
+
 .. image:: ../.static/pws.png
 
 Instrument Response Correction
@@ -262,7 +266,8 @@ Improvements in terms of performances have also been done for MSNoise 1.4:
   XXX not used actually !!!
 * ``compute_cc``: reversed the change done in 1.3, the pre-whitening of the
   traces is now disabled, it led to very high memory usage and needs a fresh
-  rewrite.
+  rewrite. This doens't mean whitening is no longer done, but just some sort
+  of caching of the pre-whitened traces.
 
 
 
@@ -280,6 +285,27 @@ Running the following command will take care of the upgrade from 1.3 to 1.4:
 
 
 
+There was a bug, mainly present in MySQL, with too sharp rounding of station
+coordinates. The bugfix change is done automatically for MySQL databases.
+It is a little different if you are using SQLite as it
+can't be done automatically. This is because SQLite doesn't support "ALTER"
+commands. Ultimately we want the ``station.X`` and ``station.Y`` to be be of
+type ``double``. You will have to do this operation manually:
+
+.. warning:: Do the following at your own risk. It *might* not be needed!
+    From the tests we ran, it seems the coordinates rounding error was present
+    only for MySQL databases!
+
+* Open SQLite database browser (`SQLiteManager <https://addons.mozilla.org/firefox/addon/sqlite-manager/>`_
+  extension for Firefox, for example)
+* Open the msnoise.sqlite file
+* Select the station table
+* Edit the ``X`` field and change its type to ``double``
+* Edit the ``Y`` field and change its type to ``double``
+* Ignore the warnings (it should work, although it could fail!)
+* Close the database
+
+
 A final note about development pace and choices
 -----------------------------------------------
 
@@ -303,7 +329,7 @@ using has been coded by 1 person, and that it's not his full time job. So
 MSNoise is provided "as-is", carefully written and tested, but there will be
 bugs, issues, incompatibility with certain python installations, OS or module
 versions. If you **want or need** developments made and you can afford it,
-contact Thomas via email directly, you can contract with the ROB for
+contact Thomas via email directly, you can contract him for
 paid-developments. If the developments you want are within the focus of the
 developers' research, then a collaboration, i.e. resulting in a co-authored
 peer reviewed publication, can be another option.

msnoise/default.py

@@ -17,8 +17,9 @@
 default['preprocess_highpass'] = ["Preprocessing High-pass value in Hz [0.01]",'0.01']
 
 default['remove_response'] = ["Remove instrument response Y/[N]",'N']
-default['response_format'] = ["Remove instrument format [dataless]/inventory/paz/resp",'dataless']
-default['response_path'] = ["Remove instrument file(s) location (path relative to db.ini)",'inventory']
+default['response_format'] = ["Remove instrument file format [dataless]/inventory/paz/resp",'dataless']
+default['response_path'] = ["Instrument correction file(s) location (path relative to db.ini), defaults to './inventory', i.e."
+                            " a subfolder in the current project folder.<br>All files in that folder will be parsed.",'inventory']
 default['response_prefilt'] = ["Remove instrument correction **pre-filter** (0.005, 0.006, 30.0, 35.0)",'(0.005, 0.006, 30.0, 35.0)']
 
 default['maxlag'] = ["Maximum lag (in seconds) [120.0]",'120.']
@@ -27,7 +28,7 @@
 default['windsorizing'] = ["Windsorizing at N time RMS , 0 disables windsorizing, -1 enables 1-bit normalization [3]",'3']
 
 default['stack_method'] = ["Stack Method: Linear Mean or Phase Weighted Stack: [linear]/pws ",'linear']
-default['pws_timegate'] = ["If stack_method='pws', width of the smoothing : 1.0 ",'1.0']
+default['pws_timegate'] = ["If stack_method='pws', width of the smoothing in seconds : 10.0 ",'10.0']
 default['pws_power'] = ["If stack_method='pws', Power of the Weighting: 2.0 ",'2.0']
 
 default['crondays'] = ["Number of days to monitors with cron [-1]",'-1']

msnoise/plots/station_map.py

@@ -12,9 +12,20 @@
 
 .. image:: .static/station_map.png
 
-"""
 
+It will also generate a HTML file showing the stations on the Leaflet Mapping
+Service:
+
+.. raw:: html
+
+    <iframe src="_static/station_map.html" width=800 height=400></iframe>
+
+
+.. versionadded:: 1.4 | Thanks to A. Mordret!
+
+"""
 
+import traceback
 import folium
 import os
 import numpy as np
@@ -40,6 +51,7 @@ def main(show=True, outfile=None):
 
     sta_map.add_child(folium.LatLngPopup())
     if outfile:
+        tmp = outfile
         if outfile.startswith("?"):
             now = datetime.datetime.now()
             now = now.strftime('station map on %Y-%m-%d %H.%M.%S')
@@ -48,8 +60,8 @@ def main(show=True, outfile=None):
         sta_map.save('%s.html'%tmp)
 
     # plot topography/bathymetry as an image.
-    bufferlat=(np.amax(coords[:,0])-np.amin(coords[:,0]))/10
-    bufferlon=(np.amax(coords[:,1])-np.amin(coords[:,1]))/10
+    bufferlat=(np.amax(coords[:,0])-np.amin(coords[:,0]))+.1
+    bufferlon=(np.amax(coords[:,1])-np.amin(coords[:,1]))+.1
     m = Basemap(projection='mill',llcrnrlat=np.amin(coords[:,0])-bufferlat,urcrnrlat=np.amax(coords[:,0])+bufferlat,\
             llcrnrlon=np.amin(coords[:,1])-bufferlon,urcrnrlon=np.amax(coords[:,1])+bufferlon,resolution='i')
 
@@ -62,11 +74,14 @@ def main(show=True, outfile=None):
     # attach new axes image to existing Basemap instance.
     m.ax = ax
     #im = m.imshow(topodat,cm.GMT_haxby)
-    m.shadedrelief()
+    try:
+        m.shadedrelief()
+    except:
+        traceback.print_exc()
     m.scatter(x,y,50,marker='v',color='r')
     for sta in stations:
         xpt, ypt = m(sta.X,sta.Y)
-        plt.text(xpt+10000,ypt+15000,"%s_%s" % (sta.net, sta.sta),fontsize=9,
+        plt.text(xpt,ypt,"%s_%s" % (sta.net, sta.sta),fontsize=9,
                     ha='center',va='top',color='k',
                     bbox = dict(boxstyle="square",ec='None',fc=(1,1,1,0.5)))
     # draw coastlines and political boundaries.

msnoise/scripts/msnoise.py

@@ -79,7 +79,7 @@ def upgrade_db():
         try:
             e = get_engine()
             e.execute('ALTER TABLE `jobs` CHANGE `type` `jobtype` VARCHAR( 10 )')
-        except OperationalError:
+        except:
             print( "The jobs table seems already up-to-date, exiting.")
     else:
         try:
@@ -89,6 +89,18 @@ def upgrade_db():
             print("You need to edit the `jobs` table manually to match the new"
                   "column naming")
             print ("Please read http://msnoise.org/doc/releasenotes/msnoise-1.3.html")
+    if get_tech() == 2:
+        try:
+            e = get_engine()
+            e.execute("ALTER TABLE stations CHANGE X X REAL NULL DEFAULT NULL")
+            e.execute("ALTER TABLE stations CHANGE Y Y REAL NULL DEFAULT NULL")
+            print("The station table has been updated (floating point bugfix)")
+        except:
+            print("The jobs table seems already up-to-date, exiting.")
+    else:
+        print("You need to edit the `station` table manually to match the new"
+              "column naming")
+        print ("Please read http://msnoise.org/doc/releasenotes/msnoise-1.4.html")
 
 
 @click.command()