Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide smarter Pass/Fail criteria #303

Open
pmouawad opened this Issue Jan 22, 2019 · 3 comments

Comments

Projects
None yet
2 participants
@pmouawad
Copy link
Contributor

pmouawad commented Jan 22, 2019

Starting with JMeter 5.1, a JSON file containing details statistics on load test will be generated on end of test by JMeter.

The proposed feature here will allow defining in pom.xml criterions
<criteria> <criterion>criterion 1 definition</criterion> <criterion>criterion 2 definition</criterion> </criteria>

Where criterion would be:

  • $..[?(@.transaction==TransactionName)].pct1ResTime > thresholdValue

Or easier syntax (which I favor) would be:

  • TransactionName.pct1ResTime > thresholdValue
  • TransactionName.errorPct > 0.01

@pmouawad pmouawad added this to the 2.8.4 milestone Jan 22, 2019

@pmouawad pmouawad modified the milestones: 2.8.4, 2.8.5 Jan 29, 2019

@Ardesco

This comment has been minimized.

Copy link
Contributor

Ardesco commented Jan 29, 2019

Do we have to define this in the POM?

Can it be defined in something like a properties file, or a config block of JSON in a file (We could set a default template that can be overridden)?

It feels like you could make the criteria quite complex and it's the sort of thing you would want to reuse by copying the pass fail config around, or having multiple pass/fail config in files and select the one you want to use for a specific test

@pmouawad

This comment has been minimized.

Copy link
Contributor Author

pmouawad commented Jan 30, 2019

Hi @Ardesco ,
I think this feature will end up in core JMeter as it looks it would be useful there and would not be restricted to people using maven plugin.

Regards

@Ardesco

This comment has been minimized.

Copy link
Contributor

Ardesco commented Jan 30, 2019

Would the config live in an existing properties file, or a new one (i.e. do we need to add support for another config file location and set a default/use a JMeter default)

@pmouawad pmouawad modified the milestones: 2.8.6, 2.8.7 Feb 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.