[thog-1548] log verification errors

[0x1]

Description:

Update plaintext logging to include verification error

Current Output

Updated Output

Checklist:

• Tests passing (make test-community)?
• Lint passing (make lint this requires golangci-lint)?

[rgmz]

Related to #1924. From my own experiences enabling this, it would be good to release this feature in tandem with an improvement to the list of known false-positives. For instance, localhost/127.0.0.1/placeholders for URI or database connections.

e.g., this should never even be attempted to be verified:

Found unverified result 🐷🔑❓
Detector Type: JDBC
Decoder Type: BASE64
Raw result: jdbc:postgresql://<IP>/<DBNAME>?cloudSqlInstance=<CONNECTION_NAME>;socketFactory=com.google.cloud.sql.postgres.SocketFactory
...
Verification Error: dial tcp: lookup <IP>: no such host
dial tcp: lookup <IP>: no such host
dial tcp 127.0.0.1:5432: connect: connection refused
dial tcp: lookup <IP>: no such host

The SQLServer pattern is also overly permissive and results in a lot of frequent (and large) verification errors.

Found unverified result 🐷🔑❓
Detector Type: SQLServer
Decoder Type: PLAIN
Raw result: Senha
format=Formato
rule_lowerCase=regra

date=Data
direction=Dire\u00e7\u00e3o
validator=Validador
extractor=Extrator
minor=Secund\u00e1rio
major=Principal
url=URL
displayName=Nome de Exibi\u00e7\u00e3o
deployedBaselines=Linhas de Base Implementadas
enabled=Ativado
debug=Depura\u00e7\u00e3o
rulesetProperties=Propriedades do Conjunto de Regras
... (this goes on for hundreds of lines)
Verification Error: unable to open tcp connection with host 'localhost:1433': dial tcp 127.0.0.1:1433: connect: connection refused

[0x1]

Related to #1924. From my own experiences enabling this, it would be good to release this feature in tandem with an improvement to the list of known false-positives. For instance, localhost/127.0.0.1/placeholders for URI or database connections.

i think placeholders make sense to filter out, but localhost/127 seem useful for internal testers, no?

[rgmz]

i think placeholders make sense to filter out, but localhost/127 seem useful for internal testers, no?

That makes sense for 172.x, 192.168.x, and 10.x (private IP ranges). Localhost/0.0.0.0/127.0.0.1 will never succeed unless you are already running the service on your local machine, otherwise you'll get a connection refused error.

It's an exceptionally low signal/noise ratio that drowns out legitimate verification errors. Not to mention that "localhost" is often the default for incomplete or incorrect information.

[0x1]

closed in favor of #2335

[0x1]

Related to #1924. From my own experiences enabling this, it would be good to release this feature in tandem with an improvement to the list of known false-positives. For instance, localhost/127.0.0.1/placeholders for URI or database connections.

e.g., this should never even be attempted to be verified:

Found unverified result 🐷🔑❓
Detector Type: JDBC
Decoder Type: BASE64
Raw result: jdbc:postgresql://<IP>/<DBNAME>?cloudSqlInstance=<CONNECTION_NAME>;socketFactory=com.google.cloud.sql.postgres.SocketFactory
...
Verification Error: dial tcp: lookup <IP>: no such host
dial tcp: lookup <IP>: no such host
dial tcp 127.0.0.1:5432: connect: connection refused
dial tcp: lookup <IP>: no such host

The SQLServer pattern is also overly permissive and results in a lot of frequent (and large) verification errors.

Found unverified result 🐷🔑❓
Detector Type: SQLServer
Decoder Type: PLAIN
Raw result: Senha
format=Formato
rule_lowerCase=regra

date=Data
direction=Dire\u00e7\u00e3o
validator=Validador
extractor=Extrator
minor=Secund\u00e1rio
major=Principal
url=URL
displayName=Nome de Exibi\u00e7\u00e3o
deployedBaselines=Linhas de Base Implementadas
enabled=Ativado
debug=Depura\u00e7\u00e3o
rulesetProperties=Propriedades do Conjunto de Regras
... (this goes on for hundreds of lines)
Verification Error: unable to open tcp connection with host 'localhost:1433': dial tcp 127.0.0.1:1433: connect: connection refused

will make separate tickets for these since they are not blocking for the log output changes