PACT generated files doesn't escape generated strings

[hsyogesh12]

Hello,
I am one of the PACT consumer, our current version of pact is 4.1.41,
It generates random strings (as example) to match the specified pattern and for some reason when it serializes those example string, it generates invalid sequence,
It doesn't escape those strings above 4.0.x version that's the reason we don't see this issue before.
we are getting below error

[ERROR] PUT JSON request failed with status HTTP/1.1 500 Internal Server Error
Failed - Request failed with HTTP/1.1 500 Internal Server Error
{"error":{"message":"source sequence is illegal/malformed utf-8","reference":"HjPEylMxlF"}}

[rholshausen]

Are you able to provide a Pact file or test code that exhibits this behavior?

[rholshausen]

/jira ticket

[github-actions]

👋 Thanks, Jira [PACT-547] ticket created.

[rholshausen]

From the error message PUT JSON request failed with status HTTP/1.1 500 Internal Server Error it looks like it is failing with a PUT request. Is this trying to publish the Pact file? Where are you publishing it?

[hsyogesh12]

We are PACT consumer, trying to publish to Pact production environment

[rholshausen]

Ok, I meant is it a hosted Pact OSS broker, or Pactflow.

Also, can you please provide one of the Pact files that are failing.

[hsyogesh12]

Yes its hosted to OSS broker

[hsyogesh12]

{
"provider": {
"name": "overview-adapter"
},
"consumer": {
"name": "mobile-adapter"
},
"interactions": [
{
"description": "a request to get the alert summary",
"request": {
"method": "GET",
"path": "/api/alertSummary/last24Hours",
"headers": {
"userId": "\uC816\uD53D\u71A8\u0287\u1C37\u232C\u2BD1\uC8D3\u7CA9\u257D\uA593\uED37\u6EB2"
},
"query": {
"filter": [
"\uEBB1"
],
"tid": [
"\u7166\u79E0\uF0E1\u7FFB\uB5F9\u9748\uF8EB"
]
},
"matchingRules": {
"query": {
"tid": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
},
"filter": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
}
},
"header": {
"userId": {
"matchers": [
{
"match": "regex",
"regex": ".+"
}
],
"combine": "AND"
}
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"entries": [
{
"severity": {
"raw": "INFO",
"text": "string"
},
"count": 100
}
],
"entryCount": 100,
"totalAlertsCount": 100
},
"matchingRules": {
"body": {
"$.totalAlertsCount": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entryCount": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entries": {
"matchers": [
{
"match": "type",
"min": 1
}
],
"combine": "AND"
},
"$.entries[].count": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entries[].severity.raw": {
"matchers": [
{
"match": "regex",
"regex": "INFO|WARNING|ERROR|CRITICAL"
}
],
"combine": "AND"
},
"$.entries[].severity.text": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
}
}
},
"generators": {
"body": {
"$.totalAlertsCount": {
"type": "RandomInt",
"min": 0,
"max": 2147483647
},
"$.entryCount": {
"type": "RandomInt",
"min": 0,
"max": 2147483647
},
"$.entries[].count": {
"type": "RandomInt",
"min": 0,
"max": 2147483647
},
"$.entries[].severity.text": {
"type": "RandomString",
"size": 20
}
}
}
},
"providerStates": [
{
"name": "alert data is available"
}
]
},
{
"description": "an internal request to get the list of capacity predictions",
"request": {
"method": "GET",
"path": "/internal/capacityPredictionList",
"headers": {
"userId": "\uA49D\u7F29\u9A28\u5E7C"
},
"query": {
"filter": [
"\u5AAB"
],
"uid": [
"\u24C1\uE946"
],
"perPage": [
"7"
],
"sort": [
"\uBDCD\u089C\u880C"
],
"page": [
"3"
],
"tid": [
"\u5FBE\u7EA7"
]
},
"matchingRules": {
"query": {
"tid": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
},
"uid": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
},
"filter": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
},
"sort": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
},
"page": {
"matchers": [
{
"match": "regex",
"regex": "[0-9]+"
}
],
"combine": "AND"
},
"perPage": {
"matchers": [
{
"match": "regex",
"regex": "[0-9]+"
}
],
"combine": "AND"
}
},
"header": {
"userId": {
"matchers": [
{
"match": "regex",
"regex": ".+"
}
],
"combine": "AND"
}
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"entries": [
{
"id": "string",
"name": "string",
"predictedFullCategory": {
"raw": "WEEK",
"text": "string"
},
"predictedFullDate": {
"raw": 100,
"text": "string"
},
"storageSystemDisplayIdentifier": "string",
"storageSystemId": "string",
"storageSystemIsAaS": true,
"storageSystemModel": "string",
"storageSystemName": "string",
"storageSystemSerialNumber": "string",
"storageSystemType": "UNITY",
"type": {
"raw": "POOL",
"text": "string"
}
}
],
"entryCount": 100
},
"matchingRules": {
"body": {
"$.entryCount": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entries": {
"matchers": [
{
"match": "type",
"min": 1
}
],
"combine": "AND"
},
"$.entries[].id": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].name": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].storageSystemId": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].storageSystemSerialNumber": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].storageSystemDisplayIdentifier": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].storageSystemName": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].storageSystemModel": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].storageSystemType": {
"matchers": [
{
"match": "regex",
"regex": "UNITY|SC|XIO|VMAX|POWERVAULT|ISILON|POWERSTORE"
}
],
"combine": "AND"
},
"$.entries[].storageSystemIsAaS": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].type.raw": {
"matchers": [
{
"match": "regex",
"regex": "FILESYSTEM|POOL|SYSTEM"
}
],
"combine": "AND"
},
"$.entries[].type.text": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].predictedFullCategory.raw": {
"matchers": [
{
"match": "regex",
"regex": "FULL|DAY|WEEK|MONTH|QUARTER"
}
],
"combine": "AND"
},
"$.entries[].predictedFullCategory.text": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].predictedFullDate.raw": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entries[].predictedFullDate.text": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
}
}
},
"generators": {
"body": {
"$.entryCount": {
"type": "RandomInt",
"min": 0,
"max": 2147483647
},
"$.entries[].id": {
"type": "RandomString",
"size": 20
},
"$.entries[].name": {
"type": "RandomString",
"size": 20
},
"$.entries[].storageSystemId": {
"type": "RandomString",
"size": 20
},
"$.entries[].storageSystemSerialNumber": {
"type": "RandomString",
"size": 20
},
"$.entries[].storageSystemDisplayIdentifier": {
"type": "RandomString",
"size": 20
},
"$.entries[].storageSystemName": {
"type": "RandomString",
"size": 20
},
"$.entries[].storageSystemModel": {
"type": "RandomString",
"size": 20
},
"$.entries[].type.text": {
"type": "RandomString",
"size": 20
},
"$.entries[].predictedFullCategory.text": {
"type": "RandomString",
"size": 20
},
"$.entries[].predictedFullDate.raw": {
"type": "RandomInt",
"min": 0,
"max": 2147483647
},
"$.entries[].predictedFullDate.text": {
"type": "RandomString",
"size": 20
}
}
}
},
"providerStates": [
{
"name": "capacity prediction data is available"
}
]
},
{
"description": "a request to get the health score summary",
"request": {
"method": "GET",
"path": "/api/healthScoreSummary",
"headers": {
"userId": "\u60A6\u022D"
},
"query": {
"filter": [
"\u3581\u6888"
],
"tid": [
"\u9A0A\u8B1D"
]
},
"matchingRules": {
"query": {
"tid": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
},
"filter": {
"matchers": [
{
"match": "regex",
"regex": "."
}
],
"combine": "AND"
}
},
"header": {
"userId": {
"matchers": [
{
"match": "regex",
"regex": ".+"
}
],
"combine": "AND"
}
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"entries": [
{
"count": 1,
"state": {
"raw": "GOOD",
"text": "Good"
},
"systems": [
{
"displayIdentifier": "PACT0001",
"healthScore": 90,
"id": "PACT0001",
"name": "PACT0001",
"serialNumber": "PACT0001",
"type": "UNITY"
}
]
}
],
"entryCount": 1,
"totalSystemsCount": 1
},
"matchingRules": {
"body": {
"$.entryCount": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.totalSystemsCount": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entries": {
"matchers": [
{
"match": "type",
"min": 1
}
],
"combine": "AND"
},
"$.entries[].count": {
"matchers": [
{
"match": "integer"
}
],
"combine": "AND"
},
"$.entries[].state.raw": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].state.text": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].systems": {
"matchers": [
{
"match": "type",
"min": 1
}
],
"combine": "AND"
},
"$.entries[].systems[].id": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].systems[].serialNumber": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].systems[].displayIdentifier": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].systems[].name": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
},
"$.entries[].systems[].healthScore": {
"matchers": [
{
"match": "number"
}
],
"combine": "AND"
},
"$.entries[].systems[*].type": {
"matchers": [
{
"match": "type"
}
],
"combine": "AND"
}
}
}
},
"providerStates": [
{
"name": "get health score summary"
}
]
}
],
"metadata": {
"pactSpecification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "4.1.40"
}
}
}

[hsyogesh12]

@Pact(provider = "system-adapter", consumer = "mobile-adapter")
public RequestResponsePact createPact(PactDslWithProvider builder) {
DslPart responseBody = new PactDslJsonBody()
.integerType("entryCount")
.minArrayLike("entries", 1)
// entry start
.stringType("id", "name", "model", "serialNumber", "displayIdentifier", "connectivityStatus",
"healthConnectivityStatus", "healthStatus")
.stringMatcher("type", PactUtils.PRODUCT_TYPES, "UNITY")
.integerType("healthScore")
.booleanType("isHealthUncertain")
.object("usedSize").integerType("raw").stringType("text").closeObject()
.object("usedPercent").numberType("raw").stringType("text").closeObject()
.object("freeSize").integerType("raw").stringType("text").closeObject()
.object("freePercent").numberType("raw").stringType("text").closeObject()
.object("configuredSize").integerType("raw").stringType("text").closeObject()
.object("logicalSize").integerType("raw").stringType("text").closeObject()
.object("thinSavings").numberType("raw").stringType("text").closeObject()
.object("snapsSavings").numberType("raw").stringType("text").closeObject()
.object("compressionSavings").numberType("raw").stringType("text").closeObject()
.object("overallEfficiency").numberType("raw").stringType("text").closeObject()
.object("healthState").stringMatcher("raw", PactUtils.HEALTH_STATES, "GOOD").stringType("text")
.closeObject()
// entry end
.closeArray();

    return builder.given("system data is available")
            .uponReceiving("a request to get the system capacity cards")
            .method("GET")
            .path("/api/system/card/capacity")
            .matchQuery("tid", "[0-9A-Za-z]+")
            .matchQuery("filter", "[0-9A-Za-z]+")
            .matchQuery("sort", "[0-9A-Za-z]+")
            .matchQuery("fields", "[0-9A-Za-z]+")
            .matchQuery("page", "[0-9]+", "1")
            .matchQuery("perPage", "[0-9]+", "25")
            .matchHeader(PactUtils.HEADER_USER_ID, PactUtils.USER_ID)
            .willRespondWith()
            .status(200)
            .headers(Collections.singletonMap("Content-Type", "application/json"))
            .body(responseBody)
            .toPact();
}

/**
 * Verifies the pact. Mandatory for the pact file to generate.
 */
@Test
@PactTestFor(pactMethod = "createPact")
public void testPact(MockServer mockServer) {
    HttpHeaders httpHeaders = new HttpHeaders();
    httpHeaders.add(PactUtils.HEADER_USER_ID, "toto");
    HttpEntity<Object> req = new HttpEntity<>(httpHeaders);
    String url = mockServer.getUrl()
            + "/api/system/card/capacity?tid=toto&filter=toto&sort=toto&fields=toto&page=1&perPage=999";
    ResponseEntity<String> response = restTemplate.exchange(url, HttpMethod.GET, req, String.class);
    assertThat(response.getStatusCode().value(), is(200));
}

[rholshausen]

I am unable to replicate this, using 4.1.41 and the latest Pact broker. Using both the provided Pact file and test code, I can see the following in the logs:

2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice: Created mobile-adapter version 0.0.1
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice:   Next steps:
    Configure the version branch to be the value of your repository branch.
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice: Pact successfully published for mobile-adapter version 0.0.1 and provider overview-adapter.
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice:   View the published pact at http://localhost:9292/pacts/provider/overview-adapter/consumer/mobile-adapter/version/0.0.1
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice:   Events detected: contract_published (pact content is the same as previous versions with tags  and no new tags were applied)
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice:   Next steps:
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice:     * Add Pact verification tests to the overview-adapter build. See https://docs.pact.io/go/provider_verification
2023-02-02T12:14:01.385+1100 [DEBUG] [au.com.dius.pact.core.pactbroker.PactBrokerClient] notice:     * Configure separate overview-adapter pact verification build and webhook to trigger it when the pact content changes. See https://docs.pact.io/go/webhooks
2023-02-02T12:14:01.386+1100 [QUIET] [system.out] OK

Just a note about using matchQuery and matchHeader, we recommend that you use example values (i.e. the third parameter) in your tests and if you want to only use random values, not to use regular expressions that can match any byte sequence (i.e. use examples like \\w+ or [0-9A-Za-z]+ instead of .+, the latter can generate any byte sequence).

For example, .matchQuery("tid", "[0-9A-Za-z]+", "abc123") will write a consistent example to the Pact file but still use random values at runtime.