Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update gonx to support variables with uppercase letters #119

Closed
ngauthier opened this issue Oct 25, 2018 · 1 comment
Closed

Update gonx to support variables with uppercase letters #119

ngauthier opened this issue Oct 25, 2018 · 1 comment

Comments

@ngauthier
Copy link
Contributor

honeycombio/gonx@dc1c4fe allows parsing fields with uppercase characters. This will allow honeytail to parse nginx fields like traceId which will allow traces from nginx logs.

Please bump the dependency on gonx to at least dc1c4fe6e0cc63c38a537b78f9ddaa60e168b957

@ngauthier
Copy link
Contributor Author

Here is an example failing test (I modified the first test in the nginx_test.go file):

func TestProcessLines(t *testing.T) {
	t1, _ := time.ParseInLocation(commonLogFormatTimeLayout, "08/Oct/2015:00:26:26 -0000", time.UTC)
	preReg := &parsers.ExtRegexp{regexp.MustCompile("^.*:..:.. (?P<pre_hostname>[a-zA-Z-.]+): ")}
	tlm := []testLineMaps{
		{
			line:        "Nov 05 10:23:45 myhost: https - 10.252.4.24 - - [08/Oct/2015:00:26:26 +0000] 200 174 0.099 a873d74c-4588-4a25-b3ed-77d23fe6275a 17d22fa7-796e-85f6-3d58-2d6e693da860",
			trimmedLine: "https - 10.252.4.24 - - [08/Oct/2015:00:26:26 +0000] 200 174 0.099 a873d74c-4588-4a25-b3ed-77d23fe6275a 17d22fa7-796e-85f6-3d58-2d6e693da860",
			ev: event.Event{
				Timestamp: t1,
				Data: map[string]interface{}{
					"pre_hostname":           "myhost",
					"body_bytes_sent":        int64(174),
					"http_x_forwarded_proto": "https",
					"remote_addr":            "10.252.4.24",
					"request_time":           0.099,
					"status":                 int64(200),
					"traceId":                "a873d74c-4588-4a25-b3ed-77d23fe6275a",
					"id":                     "17d22fa7-796e-85f6-3d58-2d6e693da860",
				},
			},
		},
	}
	p := &Parser{
		conf: Options{
			NumParsers: 5,
		},
		lineParser: &GonxLineParser{
			parser: gonx.NewParser("$http_x_forwarded_proto - $remote_addr - $remote_user [$time_local] $status $body_bytes_sent $request_time $traceId $id"),
		},
	}
	lines := make(chan string)
	send := make(chan event.Event)
	go func() {
		for _, pair := range tlm {
			lines <- pair.line
		}
		close(lines)
	}()
	go p.ProcessLines(lines, send, preReg)
	for _, pair := range tlm {
		resp := <-send
		if !reflect.DeepEqual(resp, pair.ev) {
			t.Fatalf("line resp didn't match up for %s. Expected: %+v, actual: %+v",
				pair.line, pair.ev, resp)
		}
	}
}

tredman added a commit that referenced this issue Oct 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant