Windows events #3246

cyriltovena · 2021-01-27T14:18:37Z

What this PR does / why we need it:

This PR introduces support for reading windows event logs with Promtail and send them to Loki.

It support saving the position of which events has been sent to avoid skipping logs during rollout or service interuption.

Which issue(s) this PR fixes:
Fixes #1395

Special notes for your reviewer:

I have forked telegraf windows event library in our repo to make it fit our scrape/push model. I'm using mostly syscall and event definition there. see https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_eventlog

Checklist

Documentation added
Tests updated

I had to add couple of new go modules and hook the windows log in. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

codecov-io · 2021-01-27T15:55:58Z

Codecov Report

Merging #3246 (5062184) into master (8716e24) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3246      +/-   ##
==========================================
+ Coverage   63.06%   63.08%   +0.02%     
==========================================
  Files         198      198              
  Lines       16796    16796              
==========================================
+ Hits        10592    10596       +4     
+ Misses       5239     5236       -3     
+ Partials      965      964       -1

Impacted Files	Coverage Δ
pkg/promtail/scrapeconfig/scrapeconfig.go	`12.00% <ø> (ø)`
pkg/promtail/targets/file/tailer.go	`75.00% <0.00%> (+1.78%)`	⬆️
pkg/querier/queryrange/downstreamer.go	`97.64% <0.00%> (+2.35%)`	⬆️

rfratto

This is exciting! 🎉

pkg/promtail/targets/windows/targetmanager_windows.go

owen-d

Started review, left a few nits

docs/sources/clients/promtail/configuration.md

go.mod

pkg/promtail/targets/windows/bookmark.go

owen-d

Finished review, there's a few nits and one []byte mis-instantiation, then I'll approve.

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

cyriltovena · 2021-01-28T20:56:00Z

Started review, left a few nits

Will check it out.

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

…_events

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

simnv · 2021-02-02T15:04:51Z

pkg/promtail/targets/windows/bookmark.go

+// The bookmark will be saved at the given path. Use save to save the current position for a given event.
+func newBookMark(path string) (*bookMark, error) {
+	// 4kb buffer for rendering bookmark
+	buf := make([]byte, 4<<10)


4kb will not be enough for complex event filters

You have an example ? what would be a good value ? Thanks for the feedback

By hitting some filters in Custom View of Windows Event Viewer and exporting resulting filter XML, I am easily getting a 8kb file. Bookmark with that filter takes a little bit more space.

Telegraf uses a 16kb buffer, and it's a better default value, enough for most cases.

The best way would be going full winapi: calling _EvtRender with zero as buf pointer value. It will return error, but also put the required length into bufferUsed. So then we can make buffer of required length and call _EvtRender again, this time with the newly created buffer pointer. It is a less performant way, and there can be a headache of memory allocating and freeing, but it is bulletproof.

Thank you very much for this PR btw! I made like 75% of porting Telegraf code to Loki, but got stuck at the polling loop, didn't find the satisfying method to do polls and got carried away by other duties. I like Loki, but extending it is such nontrivial work comparing to Telegraf, so many core files to change with custom targets, it is very unfriendly.

I like Loki, but extending it is such nontrivial work comparing to Telegraf, so many core files to change with custom targets, it is very unfriendly.

Sorry about that ! We'll try to get better at this.

I'll increase the buffer :) before we cut a new release

includes grafana/loki#3284 and grafana/loki#3246

See this discussion grafana#3246 (comment) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

See this discussion #3246 (comment) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

See this discussion grafana#3246 (comment) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

includes grafana/loki#3284 and grafana/loki#3246

cyriltovena added 12 commits December 20, 2020 16:20

First commit for windows event targets.

d2c8112

I had to add couple of new go modules and hook the windows log in. Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Add the ability to save where we are.

1f9cb05

Finished setup tests and comments.

042d22b

nits

70204a3

adding documentation

e639452

go tidy

a2487b7

go.mod fun

f4ceb9c

lint windows

bc1245e

nope

4c8f49c

add manager for non windows

825ce3e

don't lint forked package

afc5869

mod check

5c229be

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

rfratto reviewed Jan 27, 2021

View reviewed changes

pkg/promtail/targets/windows/targetmanager_windows.go Outdated Show resolved Hide resolved

owen-d reviewed Jan 27, 2021

View reviewed changes

owen-d reviewed Jan 28, 2021

View reviewed changes

Update pkg/promtail/targets/windows/bookmark.go

5062184

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

owen-d approved these changes Jan 29, 2021

View reviewed changes

rfratto mentioned this pull request Jan 29, 2021

Promtail: Windows Event Logging grafana/agent#338

Closed

cyriltovena and others added 10 commits February 1, 2021 16:44

Merge remote-tracking branch 'upstream/master' into windows_events

0722ed6

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Use passed-in Prometheus registerer.

83f0708

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Apply suggestions from code review

8ae11f3

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

testing out windows drone.

2b39c7b

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Merge branch 'windows_events' of github.com:grafana/loki into windows…

7dc2063

…_events

type exec.

7b41ba1

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

docker image.

23e1980

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

go/go.exe

24d9ef4

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

build windows on container.

7043e1c

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

trying another image.

5aaf324

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

cyriltovena added 5 commits February 1, 2021 17:47

typo in image

b2496a6

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Fixed local timezone issue.

6a76b8f

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Fix tests with timezone.

2a8b211

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Fixes timezone tests.

68dc467

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

Update logstash since the last image is not working anymore 🤷

c7f782b

Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

cyriltovena merged commit 72b3acb into master Feb 2, 2021

cyriltovena deleted the windows_events branch February 2, 2021 12:37

simnv reviewed Feb 2, 2021

View reviewed changes

rfratto added a commit to rfratto/agent that referenced this pull request Feb 3, 2021

update loki

3e726af

includes grafana/loki#3284 and grafana/loki#3246

rfratto mentioned this pull request Feb 3, 2021

Update Loki to d667dd grafana/agent#377

Merged

3 tasks

rfratto added a commit to grafana/agent that referenced this pull request Feb 3, 2021

update loki (#377)

d32d289

includes grafana/loki#3284 and grafana/loki#3246

cyriltovena added a commit to cyriltovena/loki that referenced this pull request Feb 8, 2021

Increase the windows bookmark buffer.

d2db159

See this discussion grafana#3246 (comment) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

cyriltovena mentioned this pull request Feb 8, 2021

Increase the windows bookmark buffer. #3303

Merged

cyriltovena added a commit that referenced this pull request Feb 9, 2021

Increase the windows bookmark buffer. (#3303)

82cead2

See this discussion #3246 (comment) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

cyriltovena added a commit to cyriltovena/loki that referenced this pull request Feb 15, 2021

Increase the windows bookmark buffer. (grafana#3303)

747ef55

See this discussion grafana#3246 (comment) Signed-off-by: Cyril Tovena <cyril.tovena@gmail.com>

mattdurham pushed a commit to grafana/agent that referenced this pull request Nov 11, 2021

update loki (#377)

9488599

includes grafana/loki#3284 and grafana/loki#3246

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows events #3246

Windows events #3246

cyriltovena commented Jan 27, 2021 •

edited

Loading

codecov-io commented Jan 27, 2021 •

edited

Loading

rfratto left a comment

owen-d left a comment

owen-d left a comment

cyriltovena commented Jan 28, 2021

simnv Feb 2, 2021

cyriltovena Feb 2, 2021 •

edited

Loading

simnv Feb 3, 2021

cyriltovena Feb 8, 2021

cyriltovena Feb 8, 2021

Windows events #3246

Windows events #3246

Conversation

cyriltovena commented Jan 27, 2021 • edited Loading

codecov-io commented Jan 27, 2021 • edited Loading

Codecov Report

rfratto left a comment

Choose a reason for hiding this comment

owen-d left a comment

Choose a reason for hiding this comment

owen-d left a comment

Choose a reason for hiding this comment

cyriltovena commented Jan 28, 2021

simnv Feb 2, 2021

Choose a reason for hiding this comment

cyriltovena Feb 2, 2021 • edited Loading

Choose a reason for hiding this comment

simnv Feb 3, 2021

Choose a reason for hiding this comment

cyriltovena Feb 8, 2021

Choose a reason for hiding this comment

cyriltovena Feb 8, 2021

Choose a reason for hiding this comment

cyriltovena commented Jan 27, 2021 •

edited

Loading

codecov-io commented Jan 27, 2021 •

edited

Loading

cyriltovena Feb 2, 2021 •

edited

Loading