Adding support for Long-Term Analytics

CHANGELOG.md

@@ -15,6 +15,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - Updated API-Builder version changed from Quezon to Timbuktu
 - API-Builder now always returns an Unknown API object if an API could not be found [#128](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/128)
 
+### Added
+- Added support for Long-Term API-Analytics based on Rollup-Jobs
+- Added support for Transations [#83](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/83)
+
 ## [3.3.1] 2021-08-17
 ### Fixed
 - Index-Templates failed to install without using a region [#124](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/124)
@@ -24,6 +28,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ### Changed
 - API-Builder stops/terminates with an error-message and error-code, if Elasticsearch cannot be configured (e.g. Index-Templates cannot be installed)
+- Events Logstash-Pipeline is now processing Transaction-Events with a customMsgAtts.transactionId into the Traffic-Summary index
 
 
 ## [3.3.0] 2021-07-21

UPDATE.md

@@ -54,7 +54,7 @@ On the other hand, the API builder Docker image, as a central component of the s
 | 3.2.0 | [X](#api-builderlogstashmemcached) | -                                  | -                                  | -             | -               | -               | -               |[X](#elastic-config)| 7.12.1  |            |
 | 3.3.0 | [X](#api-builderlogstashmemcached) | [X](#api-builderlogstashmemcached) | -                                  | -             | -               | -               | [X](#parameters)|-                   | 7.12.1  |            |
 | 3.3.1 | [X](#api-builderlogstashmemcached) | -                                  | -                                  | -             | -               | -               | [X](#parameters)|-                   | 7.12.1  |            |
-| 3.4.0 | [X](#api-builderlogstashmemcached) | -                                  | -                                  | -             | -               | -               | -               |[X](#elastic-config)| 7.14.0  | Unreleased |
+| 3.4.0 | [X](#api-builderlogstashmemcached) | [X](#api-builderlogstashmemcached) | -                                  | -             | -               | [X](#dashboards)| -               |[X](#elastic-config)| 7.14.0  | Unreleased |
 
 ### Update from Version 1.0.0
 

apibuilder4elastic/elasticsearch_config/index_config.json

@@ -12,6 +12,11 @@
         "ilm" : {
             "config": "elasticsearch_config/traffic-summary/ilm_policy.json",
             "name": "apigw-traffic-summary-ilm"
+        },
+        "rollup" : {
+            "config": "elasticsearch_config/traffic-summary/rollup_job.json",
+            "id": "apigw-traffic-summary-rollup",
+            "idSuffix": "1"
         }
     }, 
     "apigw-traffic-details": {

apibuilder4elastic/elasticsearch_config/traffic-summary/rollup_job.json

@@ -1,16 +1,16 @@
 {
     "index_pattern": "apigw-traffic-summary*",
     "rollup_index": "apigw-traffic-sum-hourly-rolledup",
-    "cron": "0 * * * * ?",
+    "cron": "0 45 * * * ?",
     "groups": {
     "date_histogram": {
-        "fixed_interval": "15m",
+        "fixed_interval": "60m",
         "field": "@timestamp",
-        "delay": "5m",
+        "delay": "240m",
         "time_zone": "UTC"
     },
     "histogram": {
-        "interval": 10,
+        "interval": 5,
         "fields": [
         "http.bytesSent",
         "http.bytesReceived"
@@ -37,7 +37,8 @@
         "serviceContext.requestPolicy",
         "serviceContext.responsePolicy",
         "serviceContext.routingPolicy",
-        "serviceContext.service.keyword"
+        "serviceContext.service.keyword",
+        "transactionId"
         ]
     }
     },
@@ -61,6 +62,5 @@
         ]
     }
     ],
-    "timeout": "20s",
     "page_size": 1000
 }