Provider verification succeeds only for the first tag of a consumer

[pkuras]

I have Pact provider verification tests which is verifying two tags of each consumer from the Pact Broker: @PactBroker( host = "...", port = ".", tags = "master,prod" )

The test execution goes fine: for each interaction, there are two tests run (one for the master tag, the other for the prod tag).

As expected, the penultimate state test prints that there's 1 more interaction to verify for both the tags (it's a pity the tag names are not printed, but it's a different issue).

Verifying a pact between my-consumer and my-provider
  Given employee list exists
  list retrieval
DEBUG  [CALL] Completed filtering of request -> GET /my-url
    returns a response which
      has status code 200 (OK)
      has a matching body (OK)

WARN   [DefaultTestResultAccumulator] Not all of the 31 were verified. The following were missing:
WARN   [DefaultTestResultAccumulator]     customer history retrieval


Verifying a pact between my-consumer and my-provider
  Given employee list exists
  list retrieval
DEBUG  [CALL] Completed filtering of request ->  GET /my-url
    returns a response which
      has status code 200 (OK)
      has a matching body (OK)

WARN   [DefaultTestResultAccumulator] Not all of the 31 were verified. The following were missing:
WARN   [DefaultTestResultAccumulator]     customer history retrieval

Things get interesting on the last test. The first pact with the first tag gets their pacts verified (result is not published because I'm running tests locally), but the the pact with the second tag gets its verified interactions count reset and the pact is not verified.

Verifying a pact between my-consumer and my-provider
  Given customer history exist for customer #1
  customer history retrieval
DEBUG  [CALL] Completed filtering of request -> GET /my-url-2
    returns a response which
      has status code 200 (OK)
      has a matching body (OK)

WARN   [DefaultTestResultAccumulator] Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')


Verifying a pact between my-consumer and my-provider
  Given customer history exist for customer #1
  customer history retrieval
DEBUG  [CALL] Completed filtering of request -> GET GET /my-url-2
    returns a response which
      has status code 200 (OK)
      has a matching body (OK)

WARN   [DefaultTestResultAccumulator] Not all of the 31 were verified. The following were missing:
WARN   [DefaultTestResultAccumulator]     get API version
WARN   [DefaultTestResultAccumulator]     login with valid credentials
... The list continues with 28 more unverified interactions

I dug into the code a bit and it seems the problem is that DefaultTestResultAccumulator uses hash of a Pact to store interaction results. The hash is calculated using only consumer and provider name:

    HashCodeBuilder().append(pact.consumer.name).append(pact.provider.name).toHashCode()

This means that differently tagged pacts of the same consumer have the same hash. Then, once one pact is fully verified, pact hash is removed from the testResults map: https://github.com/DiUS/pact-jvm/blob/master/provider/pact-jvm-provider/src/main/kotlin/au/com/dius/pact/provider/TestResultAccumulator.kt#L60

It seems like a solution could be to change the calculatePactHash to also take into account tag or consumer version.

[uglyog]

Ah, right! It uses the contents of the Pact to determine when results are not complete, but not the tags. Yes, the tag should also be included!

[anto-ac]

@pkuras Was this resolved by @uglyog‘s change?

[pkuras]

Unfortunately, I'm now affected by the linked issue #1113. Tests are now run only for the first tag of consumer.

[pkuras]

Tested with version 4.1.4 and the issue has been resolved.