Workaround SQLAlchemy 2.x upgrade issues

This bugfix release pins sqla-raw to > 1.5.1 to avoid incompatibilities with SQLAlchemy 2.x API changes

v0.13.0 - Stream results to CSV

As of this release, results endpoints with /csv format uses db.stream_result_by_name() to connect to database for more efficient memory usage. Without streaming, queries returning large result sets require large amounts of memory to both hold the entire result set and perform format conversion. Streaming handles rows and writes them out in batches. Again, only supported for CSV output at this time, as applying Marshmallow formats to JSON outputs while streaming is a harder problem. It's probably a good idea to avoid JSON output for reports expected to return large numbers of rows.

v0.12.0 - API_KEY option, S3 SQL file reading, consolidate API routes

This is a pretty big and exciting release!

• If API_KEY is set in environment, Nerium will check all requests for an X-API-Key header, and forbid access to all endpoints if the header value is missing or doesn't match
• Adds support for reading SQL files from an S3 bucket instead of local filesystem
• Moves discovery/documentation endpoints from reports/ to /docs
• As /docs endpoints are backwards compatible with v1 API, drops all the v2 endpoints

As usual, see the README for details on usage of these new features!

v0.10.3 - Upgrade dependencies

• Upgraded most library dependencies to latest versions
• Kept Flask and Werkzeug pinned to previous highest working versions (later ones are breaking tests and will have to investigate/fix before upgrading)
• Bumped version number for PyPi

Upgrade sqla-raw to 1.0 RC

• Use raw.db.result_by_name for query submission
• Parse metadata based on YAML delimiters instead of #meta
• Submit entire file as statement instead of splitting out comments first
• Remove pipenv in favor of pip-compile
• Fix missing json args bug in csv result endpoint

Fix <=3.6 compatibility

Query namedtuple object was using a defaults option introduced in Python 3.7. This release supports earlier python3 versions

JSON request body

• Webapp is now able to parse a JSON body sent to v2/results endpoints
• Puts back support for v1/<query_name> style endpoints for results for now (with deprecation note in README)
• Makes /reports discovery endpoints v2-only

Dockerfile fixes

• Bump alpine base image version
• add git executable for gitpython dependency
• pin gunicorn for compatibility with alpine py3-gevent package

Report discovery endpoints and Flask app

Flask

While there was much to love about Kenneth Reitz's 'Responder' framework, it was ultimately pulling in too many dependencies, for reasons that Nerium doesn't really need. I decided to stick with Flask for the foreseeable future, and this release rewrites app.py using Flask.

There is also a significant change to the public web API. This was unrelated to the Flask move, and in fact implemented on Responder first before porting over:

Report discovery

🎉 The release adds "discovery" endpoints:

• list reports available on the server
• get a description of each report by name, including columns returned, parameters expected, and other metadata if provided

See README for more details. Here, note that it was impossible to add these without changing the URL paths to distinguish between discovery endpoints and report results endpoints. Base paths are now:

• /v1/reports/<name> for discovery endpoints
• /v1/results/<name> for reporting results (formerly the whole app)

Query metadata formatting

Instead of using the frontmatter library and making query files a mashup of SQL and YAML, as of this release YAML metadata should be placed in a multiline comment, with a special :meta label introducing the comment. Again, usage is covered in the README

Remove plugin architecture for other query types

The library portion of Nerium had a fair amount of logic—some of it arguably verging on indirection and cruft—devoted to the idea that we might some day support non-SQL query languages, either by adding more nerium.resultset modules, or allowing third parties to provide nerium-* plugins. This was probably hurting performance, was certainly hurting maintainability, and the use case for it was speculative at best. This release drops all that, in favor of a single db module, using SQLAlchemy. Output formats can still be customized, with Marshmallow schemas.

De-munchify

I was always a little embarrassed by using munch for the query object. Replaced that with SimpleNamespace which is StandardLibrary now, and nicer in other respects.