Added py script for creating PostGIS queries

[max-zilla]

To streamline process of inserting a set of coordinates as a Polygon to
sites.geometry in BETY

[dlebauer]

Can we either make z optional or query elevation from a public API?

It appears this can be done with gdal http://gis.stackexchange.com/a/29639/3218; @yanliu-chn do have any suggestions?

[max-zilla]

We can certainly make it optional. For querying, the example you linked assumes we have a raster of elevation that we're querying.However, we may be in luck that the USGS National Elevation Dataset has an API:
http://ned.usgs.gov/epqs/

Here is an example query that returns elevation for XY (-76.116081,42.794448):
http://ned.usgs.gov/epqs/pqs.php?x=-76.116081&y=42.794448&units=Meters&output=xml
Elevation can be feet or meters.

[dlebauer]

meters Always use SI by default. NED at 1/3 arc second is ~10m thats great. Outside the domain, throw a warning and that querying elevation outside the US is not yet implemented, with a link to 'request this feature at https://github.com/pecanproject/bety/issues/new' unless there is an even easier alterinative (@yanliu-chn?)

[max-zilla]

I've got a working path to the USGS API implemented, but now I've come across the google elevation API which has global support.
https://developers.google.com/maps/documentation/elevation/intro

We'd need a google maps API key and are subject to traffic quota constraints, but this supports passing a set of points (e.g. a path or line) and returning sampled elevations from along that path. It'll even generate altitude histograms for you if we wanted to get cute.

My guess is that since this is for open research the API key should be no problem and since we're not building some kind of high-frequency mobile app, I doubt traffic volume would be a concern.

[yanliu-chn]

Please try usgs query service http://nationalmap.gov/epqs/ for US locations. It's quite easy to figure out a http get request to this service using code.

[dlebauer]

is the google elevation api implemented as indicated by these comments? it is not necessary, but I don't see it in the script

[max-zilla]

No, not yet. I need to get an API key registered first, will be smooth sailing then.

[max-zilla]

I just committed a version with the google elevation API. However, I am not using it by default - it requires the user to point to a text file with a valid Google Maps API key (they use for traffic quotas).

I can provide you with my API key if you like, or you can generate your own:
https://developers.google.com/maps/documentation/elevation/get-api-key

If that file is present and line 130 points to it, the API should work and we can call it instead of the USGS API on line 233.

[dlebauer]

I have a key.. But to confirm, points in US query USGS by default, correct?
On Thu, Nov 19, 2015 at 4:23 PM Max Burnette notifications@github.com
wrote:

In script/buildSQLGeom.py
#369 (comment):

@@ -0,0 +1,236 @@
+#!/usr/bin/env python
+
+"""
+This module will accept an input text file with (x,y,[z]) coordinates in a CSV-format list
+and generate a SQL string that will add these coordinates as a PostGIS Polygon
+to sites.geometry within a BETYdb instance.
+
+A header row should be included in the input file, although the column names do not matter. If a z column
+for altitude is not included, the script will attempt to get it from USGS NED (US) or Google (global).

I just committed a version with the google elevation API. However, I am
not using it by default - it requires the user to point to a text file with
a valid Google Maps API key (they use for traffic quotas).

I can provide you with my API key if you like, or you can generate your
own:
https://developers.google.com/maps/documentation/elevation/get-api-key

If that file is present and line 130 points to it, the API should work
and we can call it instead of the USGS API on line 233.

—
Reply to this email directly or view it on GitHub
https://github.com/PecanProject/bety/pull/369/files#r45410118.

[max-zilla]

In the current code, everything will go through USGS. My thinking was that if you have a valid key, we could just make everything go through Google. Do you consider USGS the preferred authority in the US?

We can make USGS domestic default and Google international default, but we need a good way to determine whether the point is in the US to begin with, which is likely another API call. Perhaps the anticipated traffic is low enough that this is fine.

Another option would be to query USGS, and if that doesn't find anything, we query Google. Or, we could define a simple rectangle around the US and quickly check whether the lat/lon exceed that boundary, but that falls apart e.g. in northern Mexico. Don't love that idea.

[dlebauer]

USGS is preferred because it is more accurate and consistent. The accuracy
of Google results is inversely proportional to the number of points.

Id go with 1. Query USGS. 2. If no data query Google.
On Thu, Nov 19, 2015 at 4:45 PM Max Burnette notifications@github.com
wrote:

In script/buildSQLGeom.py
#369 (comment):

@@ -0,0 +1,236 @@
+#!/usr/bin/env python
+
+"""
+This module will accept an input text file with (x,y,[z]) coordinates in a CSV-format list
+and generate a SQL string that will add these coordinates as a PostGIS Polygon
+to sites.geometry within a BETYdb instance.
+
+A header row should be included in the input file, although the column names do not matter. If a z column
+for altitude is not included, the script will attempt to get it from USGS NED (US) or Google (global).

In the current code, everything will go through USGS. My thinking was
that if you have a valid key, we could just make everything go through
Google. Do you consider USGS the preferred authority in the US?

We can make USGS domestic default and Google international default, but we
need a good way to determine whether the point is in the US to begin with,
which is likely another API call. Perhaps the anticipated traffic is low
enough that this is fine.

Another option would be to query USGS, and if that doesn't find anything,
we query Google. Or, we could define a simple rectangle around the US and
quickly check whether the lat/lon exceed that boundary, but that falls
apart e.g. in northern Mexico. Don't love that idea.

—
Reply to this email directly or view it on GitHub
https://github.com/PecanProject/bety/pull/369/files#r45412745.

[dlebauer]

@max-zilla I just tried this out and got

[script]$ python buildSQLGeom.py -h
  File "buildSQLGeom.py", line 77
    except Exception as e:
                      ^
SyntaxError: invalid syntax

line 77 has except Exception as e:

[dlebauer]

@max-zilla in this example, no site_id or sitename is provided. Does this create a new site?

[max-zilla]

Currently, if not site_id or sitename is provided, site_id is set to 0. I can modify it to create a new site, but from what I understand of the sites table we would need these non-nullable fields:

• city
• state
• country
• soil
• notes
• soilnotes
• sitename

I'll augment the examples to include one of those arguments.

[max-zilla]

As for the error, I forgot to remove that Exception placeholder as I've not yet tested the SQL execution on a live BETY instance (should be able to do that this week). I'll modify code in the meantime to catch all exceptions so you can at least see what error you're encountering there.