Set enablePending to false don't work

[flappyBug]

Software versions

• OS: Mac OSX 11.6
• @pact-foundation/pact: 9.17.0
• @pact-foundation/pact-node: 10.16.1
• X-Pact-Broker-Version: 2.93.0
• Node Version: v14.18.1

Please confirm the following:

• I have upgraded to the latest
• I have the read the FAQs in the Readme
• I have triple checked, that there are no unhandled promises in my code and have read the section on intermittent test failures
• I have set my log level to debug and attached a log file showing the complete request/response cycle
• For bonus points and virtual high fives, I have created a reproduceable git repository (see below) to illustrate the problem

Relevant log files

[2021-12-14 07:13:32.894 +0000] DEBUG (89686 on ***): pact-node@10.13.10: Starting pact binary '***/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.77/pact/bin/pact-provider-verifier', with arguments [--provider-states-setup-url http://localhost:56275/_pactSetup --log-level debug --provider *** --provider-app-version 056f6ac9 --publish-verification-results true --consumer-version-tag *** --pact-broker-base-url *** --provider-base-url http://localhost:56275 --verbose true --enable-pending false]

[2021-12-14 12:50:28.339 +0000] DEBUG (7320 on ***): pact-node@10.16.1: 
15 interactions, 0 failures, 2 pending

Pending interactions: (Failures listed here are expected and do not affect your suite's status)

I've report that and some investigation to @TimothyJones in slack's #pact-js channel. I just put an issue here to help our team to track this.

Also, great thanks to @TimothyJones for quick response.

[mefellows]

Ah, I can see the problem:

From the verifier CLI help:

      [--enable-pending], [--no-enable-pending]                              # Allow pacts which are in pending state to be verified without causing the overall task to fail. For more information, see https://pact.io/pending

It's a flag, not an argument.

[TimothyJones]

Yes, I messed this one up. Unfortunately, the arg generator is general purpose- I think we either:

1. Review the args for all the binaries to see if we can make the arg generator follow the “—no-*” pattern everywhere where there are booleans
2. (more work) replace the arg generator with something more flexible like the native verifier arg generator (we could possibly use that actual code)

[bethesque]

Everywhere there is a boolean argument in the Ruby standalone, it will follow the --foo/--no-foo pattern. That's just how Thor does it.

[TimothyJones]

This should now be fixed in pact-node 10.17.0.

It doesn't require fixing in pact-core / pact-js v10 beta, because the native verifier is used instead.

[flappyBug]

Sad. With updating to pact-node 10.17.0, I still pass with pending interactions. I've saw --no-enable-pending in my spawn binary args.

So I think there's also a bug with broker or pact-provider-verifier 😭

[flappyBug]

Or I make my pacts escape from pending state and try that again

[flappyBug]

Sorry. I still need help. The pending state don't throw exception. Does someone know --no-enable-pending is a client only behavior or we can see that in the request body or somewhere.

[TimothyJones]

@bethesque can you advise on this one?

[bethesque]

The pending state don't throw exception

I don't know what this means, sorry. Can you explain what you expect to see and what you are actually seeing?

[bethesque]

You can set VERBOSE=true and it will give you some extra output showing the HTTP logs. Can you please copy it here?

[bethesque]

Ah, I know what it was. I've put out a new release of the standalone. Can you merge that in @TimothyJones

[flappyBug]

Can you explain what you expect to see and what you are actually seeing?

@bethesque I expect totally disable the pending feature by sending --no-enable-pending to pact-provider-verifier. I don't need the notion of pending. My need my test suite fail if there's any interaction failed to pass.

[flappyBug]

You can set VERBOSE=true and it will give you some extra output showing the HTTP logs. Can you please copy it here?

@bethesque Sorry I can't put all the logs here. But here's some pieces of log (with some text redacted with ***) I think may be helpful:

[2021-12-16 02:14:29.356 +0000] DEBUG (22406 on ***): pact-node@10.17.0: Starting pact binary '***/node_modules/@pact-foundation/pact-node/standalone/darwin-1.88.80/pact/bin/pact-provider-verifier', with arguments [--provider-states-setup-url http://localhost:55853/_pactSetup --log-level debug --provider *** --provider-app-version 0b553ec0 --publish-verification-results true --consumer-version-tag *** --no-enable-pending --pact-broker-base-url *** --provider-base-url http://localhost:55853 --verbose true]

[2021-12-16 02:14:31.463 +0000] DEBUG (22406 on ***): pact-node@10.17.0: opening connection to ***...
opened
<- "GET / HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nHost: ***\r\n\r\n"
-> "HTTP/1.1 200 OK\r\n"
-> "Date: Thu, 16 Dec 2021 02:14:31 GMT\r\n"
-> "Content-Type: application/hal+json;charset=utf-8\r\n"
-> "Content-Length: 5364\r\n"
-> "Connection: keep-alive\r\n"
-> "Status: 200 OK\r\n"
-> "Vary: Accept\r\n"
-> "X-Content-Type-Options: nosniff\r\n"
-> "Server: Webmachine-Ruby/1.6.0 Rack/1.3\r\n"
-> "X-Pact-Broker-Version: 2.93.0\r\n"
-> "X-Powered-By: Phusion Passenger(R) 6.0.12\r\n"
-> "\r\n"
reading 5364 bytes...

INFO: Fetching pacts for *** from *** with the selection criteria: latest for tag ***
opening connection to ***:80...
opened
<- "POST /pacts/provider/***/for-verification HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: application/hal+json\r\nUser-Agent: Ruby\r\nContent-Type: application/json\r\nHost: ***\r\nContent-Length: 81\r\n\r\n"
<- "{\"consumerVersionSelectors\":[{\"tag\":\"***\",\"latest\":true}]}"


[2021-12-16 02:14:31.464 +0000] DEBUG (22406 on ***): pact-node@10.17.0: rification\",\"text\":\"This pact is in pending state for this version of *** because a successful verification result for *** has not yet been published. If this verification fails, it will not cause the overall build to fail. Read more at https://docs.pact.io/go/pending\"},{\"when\":\"after_verification:success_true_published_false\",\"text\":\"This pact is still in pending state for *** as the successful verification results have "

[bethesque]

Once pact js has been released with the latest version of the standalone, this should be fixed.

[TimothyJones]

Pact-node @10.17.1 released with this fix, please let us know how you go.

[mefellows]

Thanks Beth and Tim!

[flappyBug]

Thanks, as I tested, it's fixed with pact-node 10.17.1.

[TimothyJones]

Excellent news! Thanks @bethesque for the fast fix in the standalone, and @flappyBug for the original PR and testing! Much appreciated!