Skip to content

feat: added pattern name and regex #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mayankpande88 merged 3 commits into from
Dec 11, 2024
Merged

feat: added pattern name and regex #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6f30503
fix: fix updated sensitive log rules
mayankpande88 Dec 10, 2024
00f8de0
fix: added regex and pattern name
mayankpande88 Dec 11, 2024
ff89868
Merge branch 'main' into log-parsing
mayankpande88 Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions containers/container.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,8 +370,7 @@ func (c *Container) Collect(ch chan<- prometheus.Metric) {
}
}
for _, c := range p.parser.GetSensitiveCounters() {
log.Printf("sensitive log message: %s", c.Pattern)
ch <- counter(metrics.SensitiveLogMessages, float64(c.Messages), source, c.Pattern, c.Sample)
ch <- counter(metrics.SensitiveLogMessages, float64(c.Messages), source, c.Pattern, c.Sample, c.Regex, c.Name)
}
}

Expand Down
2 changes: 1 addition & 1 deletion containers/metrics.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ var metrics = struct {
NetBytesReceived: metric("container_net_tcp_bytes_received_total", "Total number of bytes received from the peer", "destination", "actual_destination", "src_workload_name", "src_workload_namespace", "src_workload_kind", "destination_workload_name", "destination_workload_namespace", "destination_workload_kind", "actual_destination_workload_name", "actual_destination_workload_namespace", "actual_destination_workload_kind"),

LogMessages: metric("container_log_messages_total", "Number of messages grouped by the automatically extracted repeated pattern", "source", "level", "pattern_hash", "sample"),
SensitiveLogMessages: metric("container_sensitive_log_messages_total", "Number of messages that contain sensitive information", "source", "pattern", "sample"),
SensitiveLogMessages: metric("container_sensitive_log_messages_total", "Number of messages that contain sensitive information", "source", "pattern", "sample", "regex", "name"),

ApplicationType: metric("container_application_type", "Type of the application running in the container (e.g. memcached, postgres, mysql)", "application_type"),

Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ require (
github.com/hashicorp/golang-lru/v2 v2.0.5
github.com/jpillora/backoff v1.0.0
github.com/mdlayher/taskstats v0.0.0-20230712191918-387b3d561d14
github.com/nudgebee/logparser v0.0.0-20241210142322-b2bd9d573f94
github.com/nudgebee/logparser v0.0.0-20241211102337-637d710a1162
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417
github.com/prometheus/client_golang v1.18.0
github.com/prometheus/common v0.46.0
Expand Down
4 changes: 4 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -841,6 +841,10 @@ github.com/ncw/swift v1.0.47/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/nudgebee/logparser v0.0.0-20241210142322-b2bd9d573f94 h1:Cawv/bZ93Oy9pOizk4NpAkSq/pbA6K5HfIK/s2sNs6s=
github.com/nudgebee/logparser v0.0.0-20241210142322-b2bd9d573f94/go.mod h1:ddmUa8en2sToRL+bBvFkuS1AHvA6Msn56v3BpLMbKoc=
github.com/nudgebee/logparser v0.0.0-20241211095040-6f2c19b5d944 h1:xxKNjxg85roYcNLJe0RgOlrdMtNkZNkHv+0dDcHAsIE=
github.com/nudgebee/logparser v0.0.0-20241211095040-6f2c19b5d944/go.mod h1:ddmUa8en2sToRL+bBvFkuS1AHvA6Msn56v3BpLMbKoc=
github.com/nudgebee/logparser v0.0.0-20241211102337-637d710a1162 h1:0NAxxCRfdUfZdXhgiy+PPytEXxGJuYsf5IGzx1DcjFU=
github.com/nudgebee/logparser v0.0.0-20241211102337-637d710a1162/go.mod h1:ddmUa8en2sToRL+bBvFkuS1AHvA6Msn56v3BpLMbKoc=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
Expand Down