Performant in-process log stripping

[davidmarkclements]

Sometimes we want to avoid logging sensitive information, post processing isn't ideal for this case so it must be done in process.

We can achieve this with a serialiser generator, which should be a separate module

Once created we can add a section to the readme about handling sensitive info

Cc @mcollina @emilyrose

[mcollina]

We need to support this in the web framework modules too!

[XVincentX]

This is the only thing that is preventing us from seriously evaluate this tool!
Looking forward!

[mcollina]

@XVincentX, this is already possible, but probably not easy nor documented :D.

@davidmarkclements we should test this in the context of hapi-pino and pino-http. Probably we need options added there too.

[davidmarkclements]

@XVincentX got a first pass going on here https://github.com/davidmarkclements/pino-noir won't be long coming :)

[davidmarkclements]

@mcollina agreed - maybe pino-noir should be included in pino-http and hapi-pino by default and just pass a filter object through - is that your position?

[davidmarkclements]

Ok, https://github.com/davidmarkclements/pino-noir is fully working, version 0.0.1 publish to npm too

Works for top level keys and nested keys, does not yet support wildcards, we need to run benchmarks & profile but it should be quite fast since we avoid copying objects to redact nested values by piggy backing on the subsequent JSON.stringify loop

@emilyrose would love to get your feedback for pino-noir API

[davidmarkclements]

@mcollina - just to say, since all inputs are strings (or array of strings), we can obv go faster by removing closure scope and compiling a string and passing to new Function, and again apply this same thinking to _.get and _.set - however, I'd like to benchmark the diff if it's not significant enough I'd rather not use new Function

[XVincentX]

That sounds great. Let me marinate the new things and I'll give it a try.

[UnidentifiedContributor]

@davidmarkclements, just now getting back to GitHub.

It looks beautiful! I love it.

[davidmarkclements]

@emilyrose - I've started on wildcard functionality on the plane, it's not quite there - my time is limited next couple of days so - I'll push anyway in case you want to bring it to land (@XVincentX feel free to PR too!) - immediate steps after that are setting up an integration benchmark, we can copy the object logging benchmarks from pino, and put pino-noir in the middle to benchmark general usage, and wildcard usage

[XVincentX]

I'd love to but I am not sure I'm the right person (time/competencies ratio) to do that!

[mcollina]

@davidmarkclements what's the status of this?

[davidmarkclements]

it needs a few hours input to get wildcard done - just need the time

[davidmarkclements]

done :)

http://npm.im/pino-noir

[davidmarkclements]

oh wait, need to add to pino readme - doing

[davidmarkclements]

Oh yah this is done

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.