Datastreams for zeek should be inside the logs-* naming

[philippkahr]

Hi

I just discovered this great repository. I personally think that when using datastreams the naming convention should be logs-zeek-default. This would follow our datastream recommendations for type-dataset-namespace.

By placing the index with starting logs it will show up automatically within the logs stream and the security pages and all other sorts of places within Kibana.

[keithjjones]

Thanks!

I looked this over and don't understand what I'm losing if I don't name data streams this way, but I think I'd have to remove functionality from zeek2es to match this naming convention.

The way I'm naming things is pretty important to how I'm processing things down the line for my research. I have the index/datastream names this way so Zeek formats can change over time, which happens a lot in my research. If I put all conn logs in logs-zeek-conn I won't be able to change the mapping for two different conn logs with different formats.

[keithjjones]

This functionality should exist already if you use the -d and -i switches to put it in your logs-zeek-conn data stream.

[keithjjones]

I clarified the -i option in the readme and the zeek2es help screen with -h.

[keithjjones]

Here is a new script to automate what I talked about above: https://github.com/corelight/zeek2es/blob/working/process_logs_as_datastream.sh

[keithjjones]

TIL you can make a data stream by naming a regular index log-zeek-conn-something! I did not know that was possible! So for that instance, you don't even need the -d option if you do not want zeek2es to automatically set up your index rollover based on size. Just use -i log-zeek-conn or whatever you want to call your data stream with that log-zeek prefix.

[keithjjones]

I added another helper script and some documentation: https://github.com/corelight/zeek2es/tree/master#helperscripts These should do what you describe above.

[keithjjones]

Since the newest updates addressed this issue, I'm closing it now. Please reopen if you think I missed the mark. Thanks!