Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Runner should not stop when an error arrises #3360

Closed
chrisron95 opened this issue Aug 8, 2017 · 24 comments
Closed

Runner should not stop when an error arrises #3360

chrisron95 opened this issue Aug 8, 2017 · 24 comments
Assignees
@vkaegis @malvikach
Labels
collection-runner feature product/runtime
Projects
Runtime Triage and Development

Comments

@chrisron95
Copy link

App Details:

Postman for Windows
Version 5.1.3
win32 10.0.14393 / x64

Issue Report:

  1. Did you encounter this recently, or has this bug always been there:
    always there
  2. Expected behaviour:
    The runner should keep going through tests even when a simple error arrises.
  3. Actual behavior:
    If we make a call that results in an html error page (normal html page, starts with ), the runner just stops dead in it's tracks and logs in the devtools "unexpected token". It's very frustrating when I have a run that's been going for a while and an error comes up that stops the runner before it's finished and I have to start over. This error should not stop the runner. I could understand it logging the error, stating that there was an error, and then skipping that call, but the runner should not stop. I have logic in my runner tests that deal with this error, however it's useless since the runner itself won't let it continue.

This also happens if it's searching for (in my case) "_id" and it's not there. This rarely stops it, but every once in a while it does.

image
@chrisron95
Copy link
Author

This is also not strictly this computer. This happens on all computers I've tried on, including Mac.

@SORC3r3r
Copy link

SORC3r3r commented Aug 9, 2017

Same here.
I edited the tests so they would not quit the runner:

if (responseBody.has("<!DOCTYPE html>") === true){
    test["..."] = true === true;
}
if (responseBody.has("_id") === true){
    test["..."] = true === true;
}

@prashantagarwal
Copy link

@chrisron95 Yeah this makes sense. We will try to enhance runner with this suggestion. I will keep you updated here about this.

@prashantagarwal prashantagarwal mentioned this issue Aug 15, 2017
Closed
@mirsoo
Copy link

mirsoo commented Sep 19, 2018

To edit the tests catching this Postman incorrect behaviour is not the right approach!
Postman should not fail on getting 500 errors.
I can react on 500 error and quit test by myself if i want (with postman.setNextRequest(null))
I'm using Version 6.3.0 and it still failing on 500 error

@vkaegis vkaegis self-assigned this Nov 14, 2018
@21anujashah
Copy link

Is there any option to resolve this issue? Can we add relevant scripts in "Tests" section to handle these type of errors? In my single collection, I have 400+ requests. It stops after 35 requests, sometimes after 40 requests.

@vvs11 vvs11 added this to Pending triage in Runtime Triage and Development Mar 12, 2020
@saswatds saswatds moved this from Pending triage to Later in Runtime Triage and Development Mar 12, 2020
@nttlorand
Copy link

Any news about this issue? It was created in 2017 and several years have passed already. If this is not going to be fixed ever due to the "Low" priority then maybe a professional could give us a pertinent solution. Thanks!

@m3dbedb
Copy link

m3dbedb commented Mar 2, 2021

I would add my 5 cents for option "halt on errors".

@half-eaten-apple
Copy link

Much needed feature in 2021 !
When i run the collection using Newman CLI, then only i am able to run all the API's even if some of them fails.
Postman should have this feature to treat failed test cases as Soft Assertions and keep the collection execution running.

@giridharvc7 giridharvc7 mentioned this issue Nov 2, 2021
Closed
1 task
@Rabadash8820
Copy link

@giridharvc7 If you're going to close #10391 as a duplicate of this one, then could you please provide an update here? These early exits during long collection runs are quite frustrating, and this issue is now over 4 years old...

@xaviergm
Copy link

xaviergm commented Mar 3, 2022

Any updates on this issue?

@Freakishmahesh
Copy link

Awaiting for the resolution. Much needed feature.!

@wnagelx
Copy link

wnagelx commented May 11, 2022

Any updates on this issue?

@LTRichHouse
Copy link

Awaiting resolution. Much needed feature!

@dcstamps
Copy link

dcstamps commented Oct 6, 2022

I see this will likely never be fixed since it's been outstanding for 5 years. I can't imagine having issues that impacts our customers for 5 years and just ignoring their countless pleas for a fix. Bad business.

@malvikach
Copy link
Collaborator

@dcstamps Apologies for your experience and for not getting back on this thread sooner! I can definitely see how an error might force you to restart the run and you might end up losing time over it.

We have taken note of this feature request! I will update this thread as soon as we have some news on this.

If you would like to talk to me 1:1 about the above issue or any other similar issues you are facing- here is a link to my calendar that you can use to schedule time with me.

@malvikach malvikach self-assigned this Oct 6, 2022
@lucianamerege
Copy link

@dcstamps Apologies for your experience and for not getting back on this thread sooner! I can definitely see how an error might force you to restart the run and you might end up losing time over it.

We have taken note of this feature request! I will update this thread as soon as we have some news on this.

If you would like to talk to me 1:1 about the above issue or any other similar issues you are facing- here is a link to my calendar that you can use to schedule time with me.

Any updates?

@mirsoo
Copy link

mirsoo commented Mar 21, 2023 via email

@Rabadash8820
Copy link

I will close this issue soon

@mirsoo You can't do that if you're not a maintainer or the Issue author.

@malvikach
Copy link
Collaborator

malvikach commented Mar 22, 2023 •
edited

Hi All,

I have provided some updates regarding this issue here

Please do take a look and add your thoughts.

@mirsoo Could you drop us a note at help@postman.com regarding the limits. Our goal is to enable your workflows as seamlessly as possible. If you wish to discuss this instead, please use this calendly link to schedule some time with me.

Regards,
Malvika

@mirsoo
Copy link

mirsoo commented Mar 22, 2023 via email

@malvikach
Copy link
Collaborator

Hi All,

Glad to inform everyone that this improvement has been released in the latest Postman release (version 10.13). You can now customize the Collection Runner to continue to run if an error occurs.

ezgif com-video-to-gif (2) (1)

The setting will be checked by default i.e. by default Runner will stop the execution in case of an error. If you wish to continue runner to execute, you can uncheck this setting and Postman will remember it throughout your session for all subsequent runs unless you change it.

Hope this helps your workflows. Once again, thanks a lot for sharing your feedback on this.

Please note: The Postman version update is being rolled out, you will get it in the next 1-2 days if you haven't already.

Cheers,
Malvika

Runtime Triage and Development automation moved this from Later to Ready Apr 19, 2023
@dcstamps
Copy link

dcstamps commented Apr 19, 2023 via email

@xaviergm
Copy link

Awesome! Thank you very much @malvikach !

@LTRichHouse
Copy link

Wonderful - thanks @malvikach . Just used it. Remembers my turning it off too -yay! ☺️

@malvikach malvikach mentioned this issue Aug 7, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vkaegis vkaegis

@malvikach malvikach

Labels
collection-runner feature product/runtime
Projects
Runtime Triage and Development
  
Ready
Milestone
No milestone
Development

No branches or pull requests