Fluent Bit Lambda Extensions Example added #42
Conversation
| // Determines listener port, use default 1234 if not provided as Lambda environment variable | ||
| listenerPort := os.Getenv("PORT") | ||
| if os.Getenv("PORT") == "" { | ||
| listenerPort = defaultListenerPort | ||
| } |
There was a problem hiding this comment.
Would it be better to use another variable name rather than the generic PORT. All extensions and the function share ENVVARS so perhaps making it more specific to the extension with something such as FLB_PORT would be preferable
| ) | ||
|
|
||
| const ( | ||
| defaultListenerPort = "1234" // DefaultListenerPort sets the http port to 1234 if not provided by function env variable "HOST" |
There was a problem hiding this comment.
This isn't the variable, should be "PORT" or changed based on other suggestion.
| logger.Fatal("The DNS name sandbox cannot be resolved.") | ||
| } | ||
| host := addr[0] | ||
| err := os.Setenv("HOST", host) |
| defaultListenerPort = "1234" // DefaultListenerPort sets the http port to 1234 if not provided by function env variable "HOST" | ||
| maxItems = 10000 | ||
| maxBytes = 262144 | ||
| timeOut = 1000 |
| @@ -0,0 +1,87 @@ | |||
| # Fluent Bit Lambda Extensions in Go | |||
There was a problem hiding this comment.
nit: Fluent Bit Logs Extension for Lambda
|
|
||
| ## Prerequisites | ||
|
|
||
| 1. You need to compile a Fluent Bit binary. The binary should be placed inside the `fluent-bit/` directory. Here are the steps: |
There was a problem hiding this comment.
Can we include the binary you have been testing with in this repo? That way users can run the example without this step. You can reword it to say that its to update the binary to a new version.
| Listen ${HOST} | ||
| Port ${PORT} |
There was a problem hiding this comment.
this seems to be missing the new tag_key field that I added
| The provided code sample demonstrates how to get Fluent Bit set up for Lambda Extensions. | ||
|
|
||
| Note: | ||
| 1. Supported Fluent Bit: v1.6 |
There was a problem hiding this comment.
This is not true. My code changes have not been merged yet, but once they are, it will mean Lambda is supported in Fluent Bit version 1.7.4 or greater
There was a problem hiding this comment.
Also, link to the official Fluent Bit docs somewhere in the beginning
|
Thanks for this example. Have you planned (or discussed with fluentbit maintainers) to develop directly a input plugin in fluentbit that implement the lambda logs API ? |
|
NOTE: For folks who find this and wonder why we abandoned it, it was because the extension tended to lose logs when the lambda function was invoked repeatedly with a very short runtime. It is possible that this solution can work well if you know that your function will always execute for at least 1 second. @r0mdau We discussed building native support in FB but that would require a lot more work. It's very non-trivial. |
|
Closing based on submitter comments. |
Issue #, if available:
Description of changes:
Added extensions example for Fluent-Bit. Requesting Code Review.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.