x-pack/filebeat/input/http_endpoint: add support for request trace logging

[efd6]

Proposed commit message

See title.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works - tested manually
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

Run go test -log-traces in x-pack/filebeat/input/http_endpoint` and then check the resulting ndjson files in the trace_logs directory.

Related issues

• Closes [Filebeat] http_endpoint - add option for capturing a trace data #36951
• Depends x-pack/filebeat/input/internal/httplog: fix incorrect append source #36956

Use cases

Screenshots

Logs

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 143 min 38 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 36951-http_endpoint upstream/36951-http_endpoint
git merge upstream/main
git push upstream 36951-http_endpoint

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 36951-http_endpoint upstream/36951-http_endpoint
git merge upstream/main
git push upstream 36951-http_endpoint

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 36951-http_endpoint upstream/36951-http_endpoint
git merge upstream/main
git push upstream 36951-http_endpoint

[andrewkroh]

Needs tailoring to this input (mentions CEL).

[andrewkroh]

I would specify that only requests that have passed validation checks (e.g. auth, content-type, method, HMAC) are logged.

Perhaps we should be trace logging unconditionally, and annotating that the request was rejected (like with the http response body and status code). That would be useful for debugging HMAC issues.

[andrewkroh]

I would like to have the source.{ip,port} in the log.

[efd6]

This comment is not strictly speaking true. The current handling of gzip content breaks the invariant that request bodies are closed. This is a bug in the current code, so I won't fix it here, but the change is

diff --git a/x-pack/filebeat/input/http_endpoint/gzip.go b/x-pack/filebeat/input/http_endpoint/gzip.go
index 332b482497..5b3e7111f1 100644
--- a/x-pack/filebeat/input/http_endpoint/gzip.go
+++ b/x-pack/filebeat/input/http_endpoint/gzip.go
@@ -18,15 +18,16 @@ var gzipDecoderPool = sync.Pool{

 type pooledGzipReader struct {
        Reader *gzip.Reader
+       closer io.Closer
 }

-func newPooledGzipReader(r io.Reader) (*pooledGzipReader, error) {
+func newPooledGzipReader(r io.ReadCloser) (*pooledGzipReader, error) {
        gzipReader := gzipDecoderPool.Get().(*gzip.Reader)
        if err := gzipReader.Reset(r); err != nil {
                gzipDecoderPool.Put(gzipReader)
                return nil, err
        }
-       return &pooledGzipReader{Reader: gzipReader}, nil
+       return &pooledGzipReader{Reader: gzipReader, closer: r}, nil
 }

 // Read implements io.Reader, reading uncompressed bytes from its underlying Reader.
@@ -34,13 +35,17 @@ func (r *pooledGzipReader) Read(b []byte) (int, error) {
        return r.Reader.Read(b)
 }

-// Close closes the Reader. It does not close the underlying io.Reader.
+// Close closes the Reader and the underlying source.
 // In order for the GZIP checksum to be verified, the reader must be
 // fully consumed until the io.EOF.
 //
 // After this call the reader should not be reused because it is returned to the pool.
 func (r *pooledGzipReader) Close() error {
        err := r.Reader.Close()
+       _err := r.closer.Close()
+       if err == nil {
+               err = _err
+       }
        gzipDecoderPool.Put(r.Reader)
        r.Reader = nil
        return err

[andrewkroh]

LGTM. Thanks for adding the source.{ip,port} and covering all the request paths with trace logging.