Asynchronous/Messages metadata validation fail

[ColdFire87]

Hi,

I have generated a pact for a message based interaction. The gist of it is:

const messagePact = new MessageConsumerPact({
    consumer: process.env.CONSUMER || 'AuditService',
    provider: 'CardService_Async',
    logLevel: 'warn',
    dir: path.resolve(process.cwd(), 'pacts'),
    spec: SpecificationVersion.SPECIFICATION_VERSION_V3,
    pactfileWriteMode: 'update',
});

describe('Kafka Pact test', () => {
    describe('receive an audit log message', () => {
        test('accepts a message', () => {
            messagePact
                .expectsToReceive('a message event update')
                .withMetadata({
                    'content-type': 'application/json',
                    'x-kafka-topic': KAFKA_TOPIC,
                })
                .withContent({
                    viewer: like('John Doe'),
                    card: {
                        id: like('08'),
                        type: regex(/^(CREDIT_CARD|PERSONAL_LOAN)$/, 'CREDIT_CARD'),
                        name: like('MyFlexiPay'),
                        version: like('v1'),
                    },
                })
                .verify(asynchronousBodyHandler(({content}) => streamHandler(content)));
        });
    });
});

Things I noticed when using different specification versions:

• when using SPECIFICATION_VERSION_V1 or SPECIFICATION_VERSION_V2, asynchronousBodyHandler passes the following data to the callback:

{
  card: { id: '08', name: 'MyFlexiPay', type: 'CREDIT_CARD', version: 'v1' },
  viewer: 'John Doe'
}

• when using SPECIFICATION_VERSION_V3, asynchronousBodyHandler passes the following data to the callback:

{
  content: {
    card: {
      id: '08',
      name: 'MyFlexiPay',
      type: 'CREDIT_CARD',
      version: 'v1'
    },
    viewer: 'John Doe'
  },
  contentType: 'application/json',
  encoded: false
}

• SPECIFICATION_VERSION_V4 is not supported by MessageConsumerPact

Regardless of using V1 or V2, the same pact contents is generated with this pact metadata:

"metadata": {
  "pact-js": {
    "version": "10.1.4"
  },
  "pactRust": {
    "ffi": "0.3.12",
    "models": "0.4.5"
  },
  "pactSpecification": {
    "version": "3.0.0"
  }
}

Although I only specify content-type & x-kafka-topic for the message metadata, contentType is automatically added:

"metadata": {
  "content-type": "application/json",
  "contentType": "application/json",
  "x-kafka-topic": "audit-log"
}

AuditService-CardService_Async_V1_V2.json.txt

When using V3, I get this pact metadata:

"metadata": {
  "pact-js": {
    "version": "10.1.4"
  },
  "pactRust": {
    "ffi": "0.3.12",
    "models": "0.4.5"
  },
  "pactSpecification": {
    "version": "4.0"
  }
}

The message metadata seems to be correct now (no additional contentType field added):

"metadata": {
  "content-type": "application/json",
  "x-kafka-topic": "audit-log"
}

AuditService-CardService_Async_V3.json.txt

---

On the provider side, I have this verification code:

const commonVerifierConfig = (commonEnv) => {
    return {
        logLevel: 'warn',
        providerVersion: commonEnv.GIT_COMMIT_SHA,
        providerVersionBranch: commonEnv.GIT_BRANCH,
        publishVerificationResult: commonEnv.PUBLISH_VERIFICATION_RESULTS,

        // Similar to state handlers?
        messageProviders: {
            'a message event update': () => JSON.parse(buildMessage(
                'John Doe',
                {id: '08', type: 'CREDIT_CARD', name: 'MyFlexiPay', version: 'v1'},
            ).value),
        },
    };
};

const verifySinglePact = (commonEnv) => () => {
    it('validates the expectations of CardService', async () => {
        // Initialize the Pact verifier
        const verifier = new MessageProviderPact({
            ...commonVerifierConfig(commonEnv),
            pactUrls: [process.env.PACT_URL],
        });

        // Verify pacts
        const output = await verifier.verify();
        console.log(output);
    });
};

where buildMessage is:

const buildMessage = (viewer, card) => ({
    value: JSON.stringify({viewer, card}),
    headers: {
        'content-type': 'application/json',
        'x-kafka-topic': KAFKA_TOPIC,
    },
});

The validation for the message body passes, but the one for the message metadata fails:

Other things I noticed:

• The documentation shows a metaData field (capital D), whereas all contracts I generated (V1/V2/V3) have metadata
  
• Pactflow also shows metaData:
  

I believe, regardless of specification version, Pactflow expects metaData.

The original thread of discussion on Slack -> https://pact-foundation.slack.com/archives/C9VBGLUM9/p1667408672806869

[ColdFire87]

Also, this makes things pretty confusing, when discussing specification versions :)

export declare enum SpecificationVersion {
    SPECIFICATION_VERSION_V2 = 3,
    SPECIFICATION_VERSION_V3 = 4,
    SPECIFICATION_VERSION_V4 = 5
}

[mefellows]

Thanks again for this detailed bug report!

So there are a few things here:

1. How to use the async interface

• You do need to set the specification to V3 or greater, because message pact is only supported from V3.
• To verify metadata, you need to modify your handlers to return a Promise<MessageFromProviderWithMetadata>, using the providerWithMetadata function (see https://github.com/pact-foundation/pact-js/blob/master/docs/messages.md#provider-producer). This is why the metadata validation is failing.

2. Improvements to the API.

• We shouldn't allow message pacts to be generated for < V3
• There appears to be a bug where the wrong shape is being sent into the message handler

I'll leave this ticket open to track these items.

3. Bug in the Pactflow UI for metaData

• We are aware of a bug in the Pactflow UI that uses the legacy metaData property on an interaction, however it should use the metadata property.

---

Other items

export declare enum SpecificationVersion {
SPECIFICATION_VERSION_V2 = 3,
SPECIFICATION_VERSION_V3 = 4,
SPECIFICATION_VERSION_V4 = 5
}

What's confusing about it? You should use the enum provided by the framework to set the spec version, not the value (implementation detail). Behind the scenes the specification is a zero-indexed enum.

[ColdFire87]

Implemented your feedback and got it to work.

Metadata bug still in UI, but doesn't affect the verification.

Thanks for helping out!

export declare enum SpecificationVersion {
SPECIFICATION_VERSION_V2 = 3,
SPECIFICATION_VERSION_V3 = 4,
SPECIFICATION_VERSION_V4 = 5
}

The confusing bit about the above comes when manually inspecting the pact (JSON), as a 4 actually means V3 for example.

[mefellows]

Implemented your feedback and got it to work.

Great - thanks for confirming! I'll close this off now, as the bug in the UI is a Pactflow issue on the backlog to be resolved.

The confusing bit about the above comes when manually inspecting the pact (JSON), as a 4 actually means V3 for example.

Oh no. In the pact JSON file, 4 == 4. It's just that the backing enums in the TS interface are indexed by 0.