2.0 beta2 documentation updates (#564)

* bump beta version

* update docs for 2.0 beta 2

cmd/trickster/main.go

@@ -33,7 +33,7 @@ var (
 
 const (
 	applicationName    = "trickster"
-	applicationVersion = "2.0.0-beta1"
+	applicationVersion = "2.0.0-beta2"
 )
 
 var wg = &sync.WaitGroup{}

docs/influxdb.md

@@ -1,33 +1,11 @@
 # InfluxDB Support
 
-Trickster 1.0 provides support for accelerating InfluxDB queries that return time series data normally visualized on a dashboard. Acceleration works by using the Time Series Delta Proxy Cache to minimize the number and time range of queries to the upstream InfluxDB server.
+Trickster provides support for accelerating InfluxDB queries that return time series data normally visualized on a dashboard. Acceleration works by using the Time Series Delta Proxy Cache to minimize the number and time range of queries to the upstream InfluxDB server.
 
 ## Scope of Support
 
 Trickster is tested with the built-in [InfluxDB DataSource Plugin for Grafana](https://grafana.com/grafana/plugins/influxdb) v5.0.0.
 
-Trickster uses pre-compiled Regular Expression pattern matches on the incoming InfluxDB query to deconstruct its components, determine if it is cacheable and, if so, what elements are factored into the cache key derivation. We also determine what parts of the query are template-able (e.g., `time >= $ts1 AND <= $ts2`) based on the provided absolute values, in order to normalize the query before hashing the cache key.
+Trickster uses InfluxDB-provided packages to parse and normalize queries for caching and acceleration. If you find query or response structures that are not yet supported, or providing inconsistent or unexpected results, we'd love for you to report those so we can further improve our InfluxDB support.
 
-If you find query or response structures that are not yet supported, or providing inconsistent or unexpected results, we'd love for you to report those. We also always welcome any contributions around this functionality. The regular expression patterns we currently use will likely grow in complexity as support for more query patterns is added. Thus, we may need to find a more robust query parsing solution, and welcome any assistance with that as well.
-
-Trickster currently supports the following InfluxDB query patterns (case-insensitive), which align with queries generated by the InfluxDB Data Source Plugin for Grafana:
-
-```sql
-SELECT field1 [, field2, field3...]
-FROM "exampledb"."example_table"
-    WHERE ("some_field" = "some_val")
-    AND $timeExpression
-GROUP BY time($duration) [, group2, group3...]
-```
-
-The `$timeExpression` section must be in the format of `time <operator1> $ts1 [AND <operator2> $ts2]`
-
-Example `$timeExpression` strings:
-
-* `time >= now()`
-* `time >= 1574699000000ms`
-* `time >= 1574699000000ms AND time <= 1574699900000ms`
-
-$duration must be in the format of `<integer>ms` such as `60s`.
-
-The InfluxDB `epoch` HTTP request query parameter is currently required to be set to `ms`.
+Trickster supports integrations with InfluxDB 1.x and 2.0, however, the Flux language is not currently supported.

docs/new-changed-2.0.md

@@ -12,15 +12,27 @@
 - we offer a brand new [Application Load Balancer](./alb.md) feature with unique and powerful options, like merging data from multiple backends into a single response.
 - We've updated to Go 1.16
 - We've re-organized many packages in the codebase to be more easily importable by other projects. Over the course of the beta, we'll be publishing new patterns to the [examples](../examples/) folder for using Trickster packages in your own projects, including caching, acceleration and load balancing.
-- InfluxDB and ClickHouse now support additional formats like csv. More documentation will be provided over the course of the beta
+- InfluxDB and ClickHouse now support additional output formats like CSV. More documentation will be provided over the course of the beta
+
+## New in Beta 2
+
+- For [InfluxDB](./influxdb.md), we've improved our compatibility with InfluxQL and hope you'll see improved cache hit rates. Flux support is not quite ready yet.
+- We've updated our "Backfill Tolerance" feature, which handles volatile data (e.g., real-time data that may not be immutable), to improve cache hit rates while ensuring volatile data is still refreshed. We've also added new options for controlling backfill tolerance thresholds. See the [example config](../examples/conf/example.full.yaml)
+- We've significantly revamped Trickster's compression capabilities. Beta 2 makes the following enhancements:
+  - adds Brotli and Zstd compression support when serving responses (we already supported gzip and deflate), when included in an `Accept-Encoding` header
+  - changes the cache compression storage format from Snappy to Brotli. A future beta will allow customization via configuration
+  - passes through (or injects) an `Accept-Encoding` header to requests between Trickster and Origins, to support end-to-end content encoding. We previously were not accepting encodings from upstreams.
+  - Provides a generic HTTP Handler package, supporting Zstd, Brotli, Gzip and Deflate; which is importable by any golang application
+- We fixed up a memory leak and refined the Prometheus label injection feature, so it plays nicely with the TSMerge ALB
+- Our [Rules Engine](./rule.md) now supports `rmatch` operations to permit regular expression-based routing against any part of the HTTP request.
+- You can now chain a collection [request rewriters](./request_rewriters.md) for more robust possibilities.
 
 ## Still to Come
 
 Trickster 2.0 is not yet feature complete, and we anticipate including the following additional features before the GA Release:
 
-- support for InfluxDB 2.0 and the flux query language, and for queries sourced by Chronograf
+- support for InfluxData's Flux query language, and for queries sourced by Chronograf
 - cache object purge via API
-- brotli compression support (wire and backend cache)
 - additional logging, metrics and tracing spans covering 2.0's new features
 - an up-to-date Grafana dashboard template for monitoring Trickster
 - support for additional Time Series Database providers

examples/conf/example.full.yaml

@@ -313,13 +313,17 @@ backends:
 #     # max_object_size_bytes defines the largest byte size an object may be before it is uncacheable due to size. default is 524288 (512k)
 #     max_object_size_bytes: 524288
 
-#     # These next 6 settings only apply to Time Series backends
+#     # These next 7 settings only apply to Time Series backends
 
-#     # backfill_tolerance_ms prevents new datapoints that fall within the tolerance window (relative to time.Now) from being cached
-#     # Think of it as "never cache the newest N milliseconds of real-time data, because it may be preliminary and subject to updates"
+#     # backfill_tolerance_ms prevents new datapoints that fall within the tolerance window (relative to time.Now) from being permanently
+#     # cached. Think of it as "the newest N milliseconds of real-time data are preliminary and subject to updates, so refresh them periodically"
 #     # default is 0
 #     backfill_tolerance_ms: 0
 
+#     # backfill_tolerance_points works like the _ms version, except the methodology is based on # of intervaled timestamps (points) in the series
+#     # instead of a relative time. You can set both values and the one impacting the most number of elements in the time series takes precedence
+#     backfill_tolerance_points: 0
+
 #     # timeseries_retention_factor defines the maximum number of recent timestamps to cache for a given query. Default is 1024
 #     timeseries_retention_factor: 1024