[Filebeat][elasticsearch] fix JSON lines picked up in gc.logs

[kesslerm]

When used with docker autodiscover, the elasticsearch gc module
erroneously picks up JSON formatted line from docker's stdout.

Fix by excluding lines that start with {.

Fixes #13164

[ycombinator]

Hi @kesslerm, thanks again for your contribution.

Could you add a XXXX.log file showing log entries in a Docker environment and a corresponding XXXX.log-expected.json file showing the expected parsed event JSONs (should be only for the GC log entries in XXXX.log)? BTW, the XXXX is meant to be a placeholder — feel free to replace it with a descriptive filename as you see fit.

The easiest way to do this would be to capture a few lines from the Docker container log, containing both GC and non-GC log entries, and pasting them in the XXXX.log file. Then run the following commands to generate the XXXX.log-expected.json file.

cd $GOPATH/src/go/src/github.com/elastic/beats/filebeat

. build/python-env/bin/activate

rm -rf build/system-tests/*

GENERATE=1 INTEGRATION_TESTS=1 TESTING_FILEBEAT_MODULES=elasticsearch TESTING_FILEBEAT_FILESETS=gc nosetests -v --nocapture tests/system/test_modules.py

This will also help ensure that the changes in the PR don't break any other tests — if they do, then other *.expected.json files will also get regenerated for the elasticsearch/gc fileset, which should not happen.

Hope that makes sense and helps. Feel free to ask questions, of course!

[ycombinator]

Hey @kesslerm, is this something you wanted to continue working on? If so, please see my previous comment. Thanks!

[ycombinator]

Hi @kesslerm, I've taken your commit and included it my PR, which also includes test files: #17164. Thanks for your contribution. Closing this PR now in favor of #17164.