New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

option to disable HTTP logs appearing in reports on demand #397

Closed
ptrthomas opened this Issue May 17, 2018 · 7 comments

Comments

Projects
None yet
2 participants
@ptrthomas
Contributor

ptrthomas commented May 17, 2018

a bunch of asks coming in:

  • option to hide steps which are just "noise", and only have method, match and print shown
  • option to switch on and off logs from within a running test

@ptrthomas ptrthomas self-assigned this May 17, 2018

@intuit intuit deleted a comment from nesrinezouari Jun 13, 2018

@ptrthomas

This comment has been minimized.

Contributor

ptrthomas commented Jun 13, 2018

good to have:

  • declare list of headers that need to be masked for all requests. else even if you hide the "sign-in" flow, all subsequent "business" flows would still have sensitive data.
  • if we could declare JsonPath and XPath expressions of request and response fields that need to be masked, that would be super
@svarunbe

This comment has been minimized.

svarunbe commented Jun 13, 2018

@ptrthomas

We can hide the steps starting with '*' in the report, so that all the noise will be reduced and user can have control over which step to be displayed in the report and which is not.
In the generated json files, there is a parameter called 'Keyword' and the same can be used to handle this.

But in case of a failure in the hidden step, it should be displayed in the report.

My suggestion is that hide the steps when Keyword is '*' and result is 'Passed'.

Thanks,

@ptrthomas

This comment has been minimized.

Contributor

ptrthomas commented Jun 13, 2018

@svarunbe I like this idea. we can have this behavior switched on via a * configure report = foo setting so the existing behavior can be the default.

thinking aloud, the configure report RHS should be a JSON to be extensible / future proof, this is already a pretty confusing area and I am seeing the following requirements:

  • configure report = true - re-sets any fine-grained config (see below)
  • configure report = false - disables all logging useful for those common / shared features where you don't want to log boring sign-in etc.
  • configure report = { showAllSteps: true } - this is the default. below we talk about only the keys similar to / same level as showAllSteps
  • if showAllSteps is false, any step with the Cucumber keyword / prefix * will be hidden from HTML / JSON reports. in other words only steps that begin with Given, When etc would be logged
  • an exception to the above rule is the print step. I'm thinking even method, match and assert need to be prefixed with When, Then, And etc. should be reasonable !
  • maskRequestHeaders: ['Authorization', 'SomeOtherSecretHeader'] - header will still be shown but the content replaced with ***
  • maskResponseHeaders - like above
  • maskRequestPaths: ['$.password', '$..ssnId'] - this will replace the payload values with ***
  • maskResponsePaths - like above
  • any step that fails would over-ride all rules and be logged / printed
  • of course, mask rules are always honored
@svarunbe

This comment has been minimized.

svarunbe commented Jun 13, 2018

@ptrthomas

It is clean and neat.
If possible, please provide a flag to enable and disable visibility of Doc String in the report.
So that high level report can be shared across the organisation.

Thanks,

@ptrthomas

This comment has been minimized.

Contributor

ptrthomas commented Jun 13, 2018

speaking of DocString - just realized there are 2 distinct concepts:

  1. logs - by default controlled by logback-test.xml- this controls what is logged to the console and the target/karate.log (by default). HTTP traffic is logged to the console / log-file only if the log-level is DEBUG (default)

  2. reports - in current state, the HTTP request and responses are inserted into the cucumber JSON / HTML ONLY IF the log-level is DEBUG. I'm proposing we de-couple this. so what appears in the JSON / HTML should be independent of the console / log-file log-level. and controlled via configure

so in other words configure report = { showRequest: false, showResponse: false } can disable HTTP traffic ending up as DocString in the HTML report.

there is this configure logPrettyRequest and configure logPrettyResponse that will still work unchanged. I don't think many people use this though, since print can do the same on demand.

ptrthomas added a commit that referenced this issue Jun 15, 2018

logging refactoring precursor to #397
log level of custom appender decoupled
response time calc more accurate, based on htto client interceptor

ptrthomas added a commit that referenced this issue Jun 18, 2018

related to #397 logPrettyRequest / Response
now apply to primary http log itself and are not duplicated if set to true
much cleaner

ptrthomas added a commit that referenced this issue Jul 9, 2018

implemented configure report to hide logs on demand ref #397
hiding steps based on whether it starts with a * was not possible
we will wait for the cucumber removal task to complete because we will have a much better reporting life-cycle
will open a new ticket for the remaining reporting changes, one is masking of sensitive values

@ptrthomas ptrthomas added the fixed label Jul 9, 2018

@ptrthomas ptrthomas changed the title from finer grained control over what appears in the cucumber html report to option to disable HTTP logs appearing in reports on demand Jul 9, 2018

@ptrthomas

This comment has been minimized.

Contributor

ptrthomas commented Jul 9, 2018

this ticket ended up only implementing the ability to show / hide logs on demand. see above history for the 2 tickets split-opened for the remaining enhancements

@ptrthomas

This comment has been minimized.

Contributor

ptrthomas commented Jul 16, 2018

released 0.8.0

@ptrthomas ptrthomas closed this Jul 16, 2018

ptrthomas added a commit that referenced this issue Jul 23, 2018

Merge pull request #466 from vmchukky/Issue-464
Issue-464: proper handling of abort and improved KarateJunitFormatter

@vmchukky this looks great I'm merging it now and will take care of removing your comments. you just may have solved why I had trouble trying to make a change to reporting as per #397
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment