Use matchers when verifying metadata

[mduesterhoeft]

Feature description

Since v10.0.0-beta.45 metaData present in the consumer contract is verified.

It would be great if we could also use matches when verifying metadata.

Something like this should be working.

  "messages": [
    {
      "matchingRules": {
        "body": {},
        "metaData": {
          "$.traceId": { "combine": "AND", "matchers": [{ "match": "type" }] }
        }
      },
      "metaData": {
        "traceId": "8bf558a2-0b0d-457e-b1b2-75800352d637"
      }
    }
  ]

Use case

The metaData object contains a traceId which is change whenever the consumer contract is published. The verification should just make sure that the field exists and carries a string. A match by value is not useful.

[mefellows]

Thanks @mduesterhoeft. I just took a quick look at the source and it seems matchers should be applied.

Could you please set the log level to trace and share an appropriately redacted log? That will help us diagnose what went wrong.

[mefellows]

Actually, on closer inspection that output is a bit confusing - metaData should actually be metadata.

The core knows how to read in both metaData and metadata at the interaction level, but not for the matching rules.

But it shouldn't actually be generating that. This might be a bug in Pact JS (not sure how yet) or in the version of the FFI we are using.

(I'd still love those logs if you could please share)

[TimothyJones]

Can you tell us which pact framework + version produced the pact file?

[mduesterhoeft]

Sorry for the confusion. I played around with the contract a bit - and posted a modified version. The contract the consumer generates has metadata in the matchingRules section - but metaData on interaction level.

So my snippet above is really this one:

"messages": [
    {
      "matchingRules": {
        "body": {},
        "metadata": {
          "$.traceId": { "combine": "AND", "matchers": [{ "match": "type" }] }
        }
      },
      "metaData": {
        "traceId": "8bf558a2-0b0d-457e-b1b2-75800352d637"
      }
    }
  ]

This is the version information from the contract ({ "pact-jvm": { "version": "4.2.3" }, "pactSpecification": { "version": "3.0.0" } })

I tried with 10.0.0-beta.48 on the provider side.

Here are the relevant pieces from the log

[2021-10-11 15:55:35.652 +0000] INFO: pact@10.0.0-beta.48: Verifying message
[2021-10-11 15:55:35.661 +0000] INFO: pact-core@13.1.6: Verifying Pacts.
[2021-10-11 15:55:35.662 +0000] INFO: pact-core@13.1.6: Verifying Pact Files
[2021-10-11 15:55:35.746 +0000] DEBUG: pact-core@13.1.6: sending arguments to FFI:
[2021-10-11 15:55:35.746 +0000] DEBUG: pact-core@13.1.6: --request-timeout
30000
--loglevel
trace
--provider-name
<provider>
--provider-version
1
--file
<local-contract-file>
--port
60293
--hostname
localhost

 RUNS   <testname>  <filename
  Given ...
    WARNING: State Change ignored as there is no state change URL provided
  Given ...
    WARNING: State Change ignored as there is no state change URL provided
  Event is triggered
    generates a message which
      includes metadata
        "traceId" with value "<actual-traceId-from-contract>" (FAILED)
      has a matching body (OK)


Failures:

1) ...
    1.1) has matching metadata
           Expected message metadata 'traceId' to have value '"<actual-traceId-from-contract>"' but was '"<trace-id-generated-by-provider>"'

So it seems to recognize the metaData from the interaction but compares against the real value from metaData instead of applying the matchingRules

[NiklasEi]

Is there an update on this? I am using pact-js to generate a MessageConsumerPact with meta data. The pact ends up containing a messages.metaData section with a regex matcher:

       "metaData": {
        "kafka_topic": {
          "json_class": "Pact::Term",
          "data": {
            "generate": "some-project-production-some-event",
            "matcher": {
              "json_class": "Regexp",
              "o": 0,
              "s": "some-project-(production|validation)-some-event"
            }
          }
        }
      }

On the provider side this fails with:

metadata: Expected metadata key 'kafka_topic' to have value '{json_class=Pact::Term, data={generate=some-project-production-some-event, matcher={json_class=Regexp, o=0, s=some-project-(production|validation)-some-event}}}' (LinkedHashMap) but was 'some-project-production-some-event' (String)

If I change metaData in the pact file to metadata the test is green.
Edit: I realized changing the pact file like this just caused the metadata to not be checked.

[mefellows]

Hi @NiklasEi, that pact file looks broken - it should not have json_class etc. in it. It's being generated by the current major version and not the v3 beta.

There have been issues with that in the past with the Ruby core, but please check you're on the latest version and have run a clean test.

[NiklasEi]

Thank you for the answer. Since this seems to be a different problem, I opened #763.

[mduesterhoeft]

@mefellows do you already have an idea on when you can fit this in. If you explain how you envision a solution I would also be happy to give it a try and come up with a PR.

[mefellows]

Thanks @mduesterhoeft, apologies for the delay in my response.

I don't have the answer right now I'm afraid. I just had a look through the rust source code and it seems any references to "metaData" have been addressed.

With the next release of the beta, I'm hoping this problem should be resolved.

cc: @uglyog as he may be able to provide a little more detail.

[mefellows]

In the meantime, i'd remove it from your consumer test until we fix the validation (which I assume you're probably doing!) or you could potentially post-process the pact file to replace metaData with metadata to see if it gets picked up properly.

[NiklasEi]

In the meantime, i'd remove it from your consumer test until we fix the validation (which I assume you're probably doing!) or you could potentially post-process the pact file to replace metaData with metadata to see if it gets picked up properly.

We tried it with metadata, but then the metadata is ignored. At the moment we just hardcode one value.

[mefellows]

👍

…

On Tue, 16 Nov 2021, 11:06 pm Niklas Eicker, ***@***.***> wrote: In the meantime, i'd remove it from your consumer test until we fix the validation (which I assume you're probably doing!) or you could potentially post-process the pact file to replace metaData with metadata to see if it gets picked up properly. We tried it with metadata, but then the metadata is ignored. At the moment we just hardcode one value. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANFDGBHF3A6PHF6MZXUVTUMJCLBANCNFSM5ERDXNBA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[mduesterhoeft]

With the next release of the beta, I'm hoping this problem should be resolved.

@mefellows Hi there - do you have any update for me on this topic? I am also totally ready to get my hands dirty to fix this issue with a PR from my side. But I would need an initial pointer, solution idea to get started.

[mefellows]

Thanks @mduesterhoeft. I'm not sure if it has been resolved.

You could

1. run a quick test on the most recent beta just to check if it's been fixed
2. if not, take a look around at https://github.com/pact-foundation/pact-reference/tree/master/rust. I haven't been in the rust core for a while, but @uglyog might be able to give you some advice (perhaps worth chatting with us in this channel)

[mduesterhoeft]

Thanks @mefellows

run a quick test on the most recent beta just to check if it's been fixed

I did that before bothering you again. I have the same issue with the latest beta.

If the solution would be in the rust core I am afraid I cannot help. I was hoping that this was in the js world still.

[Kampfmoehre]

Hi, I have a similar problem, I am using matchers in message consumer metadata like this

.withMetadata({
  contentType: "application/json", // Working
  correlationId: Matchers.string("nCB3FYqJIugLvQZ2b1dV") // Not Working
})

but the generated Pact file contains only the example value

{
  "consumer": {},
  "messages": [
    {
      "contents": {
        // ...
      },
      "matchingRules": {
        "body": {
          "$.name": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          }
        }
      },
      "metadata": {
        "contentType": "application/json",
        "correlationId": "nCB3FYqJIugLvQZ2b1dV",
      },
      "providerStates": [
        {
          // ...
        }
      ]
    }
  ]
}

This causes the provider tests to fail since the correlationId is a generated value that changes each time, the test output looks like this

    1.1) has matching metadata
           Expected message metadata 'correlationId' to have value '"nCB3FYqJIugLvQZ2b1dV"' but was '"YUqJ2pQhyYPqS5k1W5-5"'

Both use PactJS 10.0.1. I have tried MatchersV3 instead but that gives my TypeScript errors since the withMetdata method expects Matchers. Is there anything else I can provide to help with this?

[TimothyJones]

my TypeScript errors since the withMetdata method expects Matchers

This probably is a mistake in the pact v10 types - I think it would be more correct to give a V3 matcher there. However, I believe the string matcher is identical in both spec versions, so it's not the source of the problem (if you want to confirm it doesn't fix the problem, you can try as Matcher<string>).

Update: I think I found the issue:

https://github.com/pact-foundation/pact-js/blob/master/src/messageConsumerPact.ts#L178

Could you try patching that line in your node_modules? I'm guessing a bit, but I think it should say:

typeof v === 'string' ? v : JSON.stringify(v)

@mefellows if this does fix it, I think pact-js should just pass the whole thing down to the core, and the core should do the conversion.

[Kampfmoehre]

@TimothyJones I tested your line and Pact json now looks like this

{
        "correlationId": "{\"value\":\"nCB3FYqJIugLvQZ2b1dV\",\"pact:matcher:type\":\"type\"}"
}

[TimothyJones]

Ah 😅 apologies.

I think the next culprit is in the rust core. I’ll leave this for a maintainer to look at.

[rholshausen]

@Kampfmoehre That is wrong, there are only matching rules for the body

"matchingRules": {
        "body": {
          "$.name": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          }
        }
      },

[rholshausen]

The matching rules for the metadata are not being created in the Pact file.

[mefellows]

The matching rules for the metadata are not being created in the Pact file.

Does the rust core support matching rules on the metadata? I think that's the question.

[rieckpil]

Can anyone confirm that matching rules are supported for metadata? I'm struggling to get the matching rules for metadata for an asynchronous message (Pact V4) working. The Pact file properly inculdes matching rules for the metadata, e.g.:

{
  "consumer": {
    "name": "test"
  },
  "interactions": [
    {
      "matchingRules": {
         "metadata": {
          "Insert-Time": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          },
        },
        "metadata": {
          "Insert-Time": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          }
       }
   },
      "metadata": {
        "Correlation-Id": "bc0c3b06-2123-4824-a866-25767a060249",
        "Insert-Time": "2022-05-26T06:34:24.123Z",
        "Tenant-Id": "CIR7nQwtS0rA6t0S6ejd",
        "contentType": "application/json"
      },

But somehow the log states (running it with TRACE as the log level) there is no matching rule defined and it fall back to an equality check:

2022-12-07T14:07:34.829777Z TRACE ThreadId(02) pactffi_verifier_execute{handle=0x128e1d4d0}:verify_interaction{interaction="test"}: pact_models::matchingrules: matcher_is_defined: for category metadata and path ["Insert-Time"] -> false
2022-12-07T14:07:34.829780Z DEBUG ThreadId(02) pactffi_verifier_execute{handle=0x128e1d4d0}:verify_interaction{interaction="test"}: pact_matching::json: JSON -> JSON: Comparing '"2022-05-26T06:34:24.123Z"' to '"2022-12-07T14:07:34.807Z"' using Equality -> Err(Expected '2022-05-26T06:34:24.123Z' to be equal to '2022-12-07T14:07:34.807Z')

The Pact is created with Pact JVM 4.4.1 and the provider is using Pact-JS 10.2.2

[mefellows]

Could you please re-test? The latest version should support this now.

[TezzM0]

This still seems to be happening for me with version 12.1.1.

When I include metadata in the pact like so:

      .withMetadata({
        'message-id': uuid("a0d38a09-f2da-46cf-8267-3cd575056606"),
        'message-type': 'create-product',
      })

It generates a pact contract that looks like this:

{
      "contents": {
        "body": {
          "createdAt": "2015-08-06T16:53:10.123+01:00",
          "id": "6551bfe2a3168487de1773dd",
          "updatedAt": "2015-08-06T16:53:10.123+01:00"
        },
        "headers": {
          "message-id": "5892a89d-c665-4dc2-939a-8a0d1f7744cc",
          "message-type": "create-product"
        }
      },
      "description": "A product created event",
      "matchingRules": {
        "body": {
          "$.body": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          },
          "$.body.createdAt": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^\\d{4}-[01]\\d-[0-3]\\dT[0-2]\\d:[0-5]\\d:[0-5]\\d\\.\\d+([+-][0-2]\\d(:?[0-5]\\d)?|Z)$"
              }
            ]
          },
          "$.body.id": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^[a-fA-F0-9]{24}$"
              }
            ]
          },
          "$.body.updatedAt": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^\\d{4}-[01]\\d-[0-3]\\dT[0-2]\\d:[0-5]\\d:[0-5]\\d\\.\\d+([+-][0-2]\\d(:?[0-5]\\d)?|Z)$"
              }
            ]
          },
          "$.headers": {
            "combine": "AND",
            "matchers": [
              {
                "match": "type"
              }
            ]
          },
          "$.headers['message-id']": {
            "combine": "AND",
            "matchers": [
              {
                "match": "regex",
                "regex": "^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$"
              }
            ]
          }
        }
      },
      "metadata": {
        "contentType": "application/json",
        "message-id": "a0d38a09-f2da-46cf-8267-3cd575056606",
        "message-type": "create-product"
      }
    },

The header matching rules are now included in the pact, which I didn't see happening previously (not sure which version I last tried with, sorry) - but they aren't used by the provider during verification:

A product created event (1s 994ms loading, 3ms verification)
    generates a message which
      includes metadata
        "message-type" with value "create-product" (OK)
        "message-id" with value "a0d38a09-f2da-46cf-8267-3cd575056606" (FAILED)
        "contentType" with value "application/json" (OK)
      has a matching body (OK)

Pending Failures:
1) Verifying a pact between repeat-rewards and product-catalogue - A product created event
    1.1) has matching metadata
           Expected message metadata 'message-id' to have value '"a0d38a09-f2da-46cf-8267-3cd575056606"' but was '"569079cd-611c-47a0-9968-ddf88ac6badc"'

It still requires the UUID to match exactly, rather than using the regex specified in the matchers.

[github-actions]

🤖 Great news! We've labeled this issue as smartbear-supported and created a tracking ticket in PactFlow's Jira (PACT-1495). We'll keep work public and post updates here. Meanwhile, feel free to check out our docs. Thanks for your patience!

[mefellows]

I think this would be resolved via this change: https://github.com/pact-foundation/pact-reference/pull/343/files

[ferrisbuhler]

I checked with pact-js 12.1.2 and 12.2.0. For both versions

• MessageConsumerPact refuses to accept V3 matchers (such as MatchersV3.uuid() in metadata (see (TypeScript) Cannot use V3 matchers within MessagePact. Types are incompatible. #1133)
• no matchers are generated in the "Pact" contract for the metadata.

Any news on this? Any workarounds to be recommended?

[mefellows]

This should be fixed now - thanks all.