Release 0.8.0

[c0c0n3]

With this PR, we releases QuantumLeap version 0.8.0. This version comes with:

• Experimental support to store NGSI-LD payloads: The notify API endpoint now accepts both NGSI-LD and NGSI-v2 entities! While there are still some limitations to what NGSI-LD payloads QuantumLeap can process, the most common scenarios should be covered.
• Much improved data ingestion (notify API endpoint) performance with a brand new Redis query cache.
• Full support for Timescale: The Timescale backend is now on par, in terms of features, with the CrateDB backend---i.e. all API endpoints work both with Timescale and CrateDB. Running both backends side-by-side is also supported---e.g. routing data to a specific DB depending on tenant.
• Gunicorn support: You can now run QuantumLeap as a WSGI app in your own Gunicorn server or even start QuantumLeap with its own embedded Gunicorn instance.
• Ability to collect telemetry time-series for performance analysis---turn on telemetry then use e.g. Pandas to analyse your data.
• Better logs and documentation as well as a number of critical bug fixes.

See below for the details. We recommend QuantumLeap 0.7.6 users to upgrade to this version.

New features

• Experimental NGSI-LD support
  • Make the notify endpoint accept NGSI-LD payloads, convert them to tabular format and store them in the DB backend (redis cache on insert #373)
  • Retain NGSI-v2 backward compatibility (redis cache on insert #373)
  • Verify basic Orion-LD interoperability (add demo test for orion-ld integration #413)
• Improved performance
  • Make CrateDB async writes the default but allow the setting to be overwritten through configuration (active shards 1 crate #384)
  • Reduce DB queries on insert through a Redis metadata cache (redis cache on insert #373)
  • Pool DB connections (redis cache on insert #373)
• Expose configuration settings to enable/disable caching of geo-queries and metadata queries on insert (cache configuration improvments #429)
• Better logs
  • Adopt Orion log format and improve log messages (redis cache on insert #373)
  • Log FiWare correlation ID to support tracking of requests from agents (through Orion) to QuantumLeap (redis cache on insert #373)
  • Add process and thread ID to log entries (Added PID and TID to logs #367)
  • Make log messages more descriptive and use debug log level (improved logs #355)
  • Timestamp log entries ( Updated Log entry with Timestamp  #352)
• Assign NGSI attribute values a DB type according to their JSON type if no NGSI type is present rather than defaulting to text (redis cache on insert #373)
• Make all API endpoints work with Timescale as a backend (Timescale parity #374)
• Support running QuantumLeap as a WSGI app in Gunicorn (Gunicorn plumbing #357)
• Collect telemetry time-series to analyse performance (Telemetry #411)

Bug fixes

• Honour default DB backend setting in YAML configuration (fix #404 #405)
• Change health status from critical to warning when cache backend is down (changed cache check from critical to not critical (fix #341) #402)
• Explicitly add new columns to CrateDB tables to cater to corner cases where new columns aren't added if using Crate's dynamic column policy (redis cache on insert #373)
• Log a warning if there's a type mismatch between NGSI and DB date-time rather than making queries crash (support mismatch between DateTime (as metadata) and column type #387)
• Use proper ISO 8601 date-times and FiWare service path match operator in CrateDB queries (fix date format to be compatible also with crate 3.x (#353) #371)
• Use proper CrateDB types rather than deprecated aliases (Improve crate 4.x support  #370)
• Assign entities to their respective service paths when a notification contains multiple service paths (fix #208 #363, fix-353 #364, fix 347 #365)
• Return HTTP 500 on DB insert failure (Updated QL to notify when memory not available #346)

Documentation

• Note delay to be expected between an entity insertion and its subsequent availability for querying (add clarifications to docs in relation to #418 #420)
• Update Japanese documentation ((ja) Update Japanese documentation. #414)
• Substantial updates regarding Redis cache, benchmarks and NGSI-LD support (redis cache on insert #373)
• Gunicorn security settings for QuantumLeap (Quantum Leap in Gunicorn docs #380)
• Mention CrateDB lacks support for 3D coordinates (Update document fix for crate.md #340)

Technical debt

• Clean up and refactor translator tests (Clean test translator #403)

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️