From 8950773a82daf6207ea4c4571a24c2fb44d1b5bf Mon Sep 17 00:00:00 2001 From: Patrick Stephens Date: Thu, 17 Mar 2022 09:44:03 +0000 Subject: [PATCH 1/4] windows: add 1.9 updates Signed-off-by: Patrick Stephens --- installation/windows.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/installation/windows.md b/installation/windows.md index 0932bbf81..32a8ccf6a 100644 --- a/installation/windows.md +++ b/installation/windows.md @@ -164,8 +164,7 @@ The installation wizard will automatically start. ![](<../.gitbook/assets/windows_installer (1) (1).png>) -Click Next and proceed. -By default, Fluent Bit is installed into `C:\Program Files\fluent-bit\`, so you should be able to launch fluent-bit as follows after installation. +Click Next and proceed. By default, Fluent Bit is installed into `C:\Program Files\fluent-bit\`, so you should be able to launch fluent-bit as follow after installation. ``` PS> C:\Program Files\fluent-bit\bin\fluent-bit.exe -i dummy -o stdout From 85629f17ffd62a517d023bc57265588db57d5076 Mon Sep 17 00:00:00 2001 From: Patrick Stephens Date: Thu, 17 Mar 2022 09:47:17 +0000 Subject: [PATCH 2/4] windows: doc tweaks Signed-off-by: Patrick Stephens --- installation/windows.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/installation/windows.md b/installation/windows.md index 32a8ccf6a..0932bbf81 100644 --- a/installation/windows.md +++ b/installation/windows.md @@ -164,7 +164,8 @@ The installation wizard will automatically start. ![](<../.gitbook/assets/windows_installer (1) (1).png>) -Click Next and proceed. By default, Fluent Bit is installed into `C:\Program Files\fluent-bit\`, so you should be able to launch fluent-bit as follow after installation. +Click Next and proceed. +By default, Fluent Bit is installed into `C:\Program Files\fluent-bit\`, so you should be able to launch fluent-bit as follows after installation. ``` PS> C:\Program Files\fluent-bit\bin\fluent-bit.exe -i dummy -o stdout From 6792f06acb3d63b9094148585b77bd382ec2d0cc Mon Sep 17 00:00:00 2001 From: Patrick Stephens Date: Fri, 18 Mar 2022 11:43:39 +0000 Subject: [PATCH 3/4] elasticsearch: add some clarification Signed-off-by: Patrick Stephens --- pipeline/outputs/elasticsearch.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pipeline/outputs/elasticsearch.md b/pipeline/outputs/elasticsearch.md index 73eba4abd..be67d8506 100644 --- a/pipeline/outputs/elasticsearch.md +++ b/pipeline/outputs/elasticsearch.md @@ -20,7 +20,8 @@ The **es** output plugin, allows to ingest your records into an [Elasticsearch]( | AWS\_STS\_Endpoint | Specify the custom sts endpoint to be used with STS API for Amazon OpenSearch Service | | | AWS\_Role\_ARN | AWS IAM Role to assume to put records to your Amazon cluster | | | AWS\_External\_ID | External ID for the AWS IAM Role specified with `aws_role_arn` | | -| Cloud\_ID | If you are using Elastic's Elasticsearch Service you can specify the cloud\_id of the cluster running | | +| Cloud\_ID | If you are using Elastic's Elasticsearch Service you can specify the cloud\_id of the cluster running. The Cloud ID string has the format `:`. Once decoded, the `base64_info` string has the format `$$`. + | | | Cloud\_Auth | Specify the credentials to use to connect to Elastic's Elasticsearch Service running on Elastic Cloud | | | HTTP\_User | Optional username credential for Elastic X-Pack access | | | HTTP\_Passwd | Password for user defined in HTTP\_User | | @@ -193,6 +194,7 @@ Notice that the `Port` is set to `443`, `tls` is enabled, and `AWS_Region` is se ### Fluent Bit + Elastic Cloud Fluent Bit supports connecting to [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) providing just the `cloud_id` and the `cloud_auth` settings. +`cloud_auth` uses the `elastic` user and password provided when the cluster was created, for details refer to the [Cloud ID usage page](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html). Example configuration: From 7644920aad4a37d869d04da4f02b7120669bdd7a Mon Sep 17 00:00:00 2001 From: Patrick Stephens Date: Fri, 18 Mar 2022 12:54:55 +0000 Subject: [PATCH 4/4] elasticsearch: add Suppress_Type_Name info Signed-off-by: Patrick Stephens --- pipeline/outputs/elasticsearch.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/pipeline/outputs/elasticsearch.md b/pipeline/outputs/elasticsearch.md index be67d8506..cfb83f572 100644 --- a/pipeline/outputs/elasticsearch.md +++ b/pipeline/outputs/elasticsearch.md @@ -205,13 +205,15 @@ Example configuration: Tag_Key tags tls On tls.verify Off + Suppress_Type_Name On cloud_id elastic-obs-deployment:ZXVybxxxxxxxxxxxg== cloud_auth elastic:2vxxxxxxxxYV ``` ### Validation Failed: 1: an id must be provided if version type or value are set -Since v1.8.2, Fluent Bit started using `create` method (instead of `index`) for data submission. This makes Flunt Bit compatible with Datastream introduced in Elasticsearch 7.9. +Since v1.8.2, Fluent Bit started using `create` method (instead of `index`) for data submission. +This makes Fluent Bit compatible with Datastream introduced in Elasticsearch 7.9. If you see `action_request_validation_exception` errors on your pipeline with Fluent Bit >= v1.8.2, you can fix it up by turning on `Generate_ID` as follows: @@ -222,3 +224,12 @@ If you see `action_request_validation_exception` errors on your pipeline with Fl Host 192.168.12.1 Generate_ID on ``` + +### Action/metadata contains an unknown parameter type + +Elastic Cloud is now on version 8 so the type option must be removed by setting `Suppress_Type_Name On` as indicated above. + +Without this you will see errors like: +``` +{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400} +```