Flaky test in github.com/prometheus/prometheus/discovery/file.go

[elifkus]

I ran make test and the tests for file service discovery failed. But sometimes they don't.

--- FAIL: TestFileSD (0.03s)
file_test.go:105: Unexpected target groups [], we expected a failure here.

I looked into it and found out what makes the tests fail. But I'm not able to fix it (as I'm new to the codebase and to the language) so I'm writing here in the hope someone who knows better might help out. I'll continue in the comments.

System information: MacOs 10.13.1
Darwin 17.2.0 x86_64
Prometheus version:
prometheus, version 2.0.0-rc.3 (branch: master, revision: 09be10a)
build user: mine@my-MacBook.local
build date: 20171107-08:54:35
go version: go1.8

[elifkus]

The test that fails is testFileSD(t, "invalid_nil", ".yml", false)

I found out that the test fails when the content of fixtures/_test_invalid_nil.yml is empty instead of

• targets: ['localhost:9090', 'example.org:443']
  labels:
  foo: bar

• null

I guess that's because sometimes the content is read before the content is written over to the file (which is done in the test testFileSD).

Probably

go fsd.Run(ctx, ch)

at line 53 starts the read before line 74 writes the file content to fixtures/_test_invalid_nil.yml

_, err = io.Copy(newf, f)

When I put a 100ms sleep just before line 74, the test always fails.

[grobie]

Thanks for the report @elifkus! I guess we need some goroutine synchronization here. @krasi-georgiev you were looking into our sd tests already, right?

[krasi-georgiev]

@grobie no I haven't, but It happens almost every time on my local tests and quite annoyed by these so will get it fixed.

@elifkus thanks for the pointers.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.