Pact verifier is throwing Error (Failure/Error: expect(response_body).to match_term expected_response_body, diff_options Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8) #90

ghost · 2018-08-02T03:36:42Z

Hi,
can you please help me on below scenario ?

i am able to create Jason mock properly but at the time json file generation below error is coming but Json file is created successfully
Error: "AssertionError: [{'class': ['HealthCheck'], 'properties':[863 chars]'}]}] != <pact.matchers.EachLike object at 0x06C66AD0>"

using pact verifier to verify the json file but getting below error .

        Failure/Error: expect(response_status).to eql expected_response_status
                expected: 200
                  got: 403

                 (compared using eql?)

      Failure/Error: expect(response_body).to match_term expected_response_body, diff_options
           Encoding::UndefinedConversionError:
            "\xE2" from ASCII-8BIT to UTF-8

i am doing poc and just want to mock only some part of the response to pass the test case. can you please help me what i am doing wrong ?

Consumer.py:

import requests
def callAPI (url):
      return requests.get ( url ).json()

Test_consumer.py :


 import atexit
import unittest

from consumer import callAPI

from pact import Consumer, Provider
from pact import EachLike, SomethingLike, Term, Like

pact = Consumer ( 'consumer' ).has_pact_with ( Provider ( 'provider' ) )
pact.start_service ()
atexit.register ( pact.stop_service )


class MakeItContractTestingWork ( unittest.TestCase ):
    # maxDiff = None

    def test_callAPI_HealthCheck (self):
        url = 'http://localhost:1234'
        expected =EachLike({'class': Like(['HealthCheck']),
                    'properties': {
                        'timestamp':  Term('\d+-\d+-\d+T\d+:\d+:\d+', '2018-07-31T01:38:08+0000'),
                        'systemName': Like("minimal-rx"),
                        "buildInfo": {
                            "buildDate": Term('\d+-\d+-\d+T\d+:\d+:\d+', '2018-07-31T01:38:08+0000'),
                            "appVersion": Like("CS-MRA_develop.25"),
                            "revision": Like("291a6553f509edc68074d31cd7d5a077a98c5add"),
                            "branch": Like("develop")
                        },
                        "health": Like("OK"),
                        "checks": [
                            {"check": Like("DB connection via Instance of class com.zaxxer.hikari.HikariDataSource"),
                             "status": Like("OK"),
                             "assertion": Like("available"),
                             "target": Like("Instance of class com.zaxxer.hikari.HikariDataSource")
                             },
                            {
                                "check": Like("Connection to https://alignapi.alta-sqa1.aligntech.com/protocol/v1/userinfo"),
                                "status": Like("OK"),
                                "config_id": Like("alignapi\\.alta-sqa1\\.aligntech\\.com"),
                                "secure_key": Like("alignapi.alta-sqa1.aligntech.com"),
                                "assertion": Like("access"),
                                "config_type": Like("OAUTH"),
                                "target": Like("https://alignapi.alta-sqa1.aligntech.com/protocol/v1/userinfo")
                            }
                        ]
                    },
                    "links": [
                        {
                            "rel": Like(["self"]), "href": Like("/minimal-rx/v1/admin/healthcheck")
                        }
                    ]
                    }, minimum=1
)
        pact.given (
            'the health of the system is OK'
        ).upon_receiving (
            'a request for the health check'
        ).with_request (
            'get',
            '/'
        ).will_respond_with ( 200, headers={'Content-Type': 'application/json;charset=utf-8'}, body= expected)

        with pact:
            result = callAPI ( url )
        self.assertEqual ( result, expected )

mockobject = MakeItContractTestingWork ( 'test_callAPI_HealthCheck' )
mockobject.test_callAPI_HealthCheck ()




# Pact verfier syntax : i am running below command from terminal and getting error

#   pact-verifier --provider-base-url=https://alignapi.alta-sqa1.aligntech.com/minimal-rx/v1/admin/healthcheck  .\consumer-provider.json

can you please help me what i am doing wrong ?

The text was updated successfully, but these errors were encountered:

mefellows · 2018-08-02T05:41:03Z

Thanks. Can you please provide the output pact (JSON) file, and ideally, a minimal code base (in GitHub) that we can use to replicate your problem?

ghost · 2018-08-02T07:41:02Z

Thanks Matt for helping out , here is JSON file

{
  "consumer": {
    "name": "consumer"
  },
  "provider": {
    "name": "provider"
  },
  "interactions": [
    {
      "description": "fetch all the associated protocol ids, versions & Operations available",
      "providerState": "Given there is a valid existing rx form",
      "request": {
        "method": "get",
        "path": "/"
      },
      "response": {
        "status": 200,
        "headers": {
        },
        "body": [
          {
            "class": [
              "RxForm"
            ],
            "properties": {
              "lastModified": "2018-07-16T13:54:37.433Z",
              "clinId": "mbricker",
              "dateCreated": "2018-07-16T13:54:37.433Z",
              "id": "83359274-7ad6-4724-89c4-e3648b7e2f2f"
            },
            "actions": [
              {
                "name": "update",
                "method": "PUT",
                "href": "/minimal-rx/v1/rx-forms/83359274-7ad6-4724-89c4-e3648b7e2f2f",
                "title": "Update rx form",
                "type": "application/json",
                "fields": [
                  {
                    "name": "protocolId",
                    "title": "Protocol id",
                    "type": "TEXT",
                    "required": true
                  },
                  {
                    "name": "protocolVersion",
                    "title": "Protocol version",
                    "type": "TEXT",
                    "required": true
                  },
                  {
                    "name": "content",
                    "title": "Rx form content",
                    "type": "TEXT",
                    "required": true
                  }
                ]
              },
              {
                "name": "delete",
                "method": "DELETE",
                "href": "/minimal-rx/v1/rx-forms/83359274-7ad6-4724-89c4-e3648b7e2f2f",
                "title": "Delete rx form",
                "type": "application/x-www-form-urlencoded"
              }
            ],
            "links": [
              {
                "rel": [
                  "self"
                ],
                "href": "/minimal-rx/v1/rx-forms/83359274-7ad6-4724-89c4-e3648b7e2f2f"
              }
            ]
          }
        ],
        "matchingRules": {
          "$.body": {
            "min": 1
          },
          "$.body[*].*": {
            "match": "type"
          },
          "$.body[*].class": {
            "match": "type"
          },
          "$.body[*].properties.lastModified": {
            "match": "type"
          },
          "$.body[*].properties.clinId": {
            "match": "type"
          },
          "$.body[*].properties.dateCreated": {
            "match": "type"
          },
          "$.body[*].properties.id": {
            "match": "type"
          },
          "$.body[*].actions[0].name": {
            "match": "type"
          },
          "$.body[*].actions[0].method": {
            "match": "type"
          },
          "$.body[*].actions[0].href": {
            "match": "type"
          },
          "$.body[*].actions[0].title": {
            "match": "type"
          },
          "$.body[*].actions[0].type": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[0].name": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[0].title": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[0].type": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[0].required": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[1].name": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[1].title": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[1].type": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[1].required": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[2].name": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[2].title": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[2].type": {
            "match": "type"
          },
          "$.body[*].actions[0].fields[2].required": {
            "match": "type"
          },
          "$.body[*].actions[1].name": {
            "match": "type"
          },
          "$.body[*].actions[1].method": {
            "match": "type"
          },
          "$.body[*].actions[1].href": {
            "match": "type"
          },
          "$.body[*].actions[1].title": {
            "match": "type"
          },
          "$.body[*].actions[1].type": {
            "match": "type"
          },
          "$.body[*].links[0].rel": {
            "match": "type"
          },
          "$.body[*].links[0].href": {
            "match": "type"
          }
        }
      }
    }
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}

ghost · 2018-08-02T08:01:19Z

hi matt,
i have created repository . here is the link
https://github.com/thiruHp/PACT-PYTHON-Issue

bethesque · 2018-08-02T23:14:44Z

@thiruHp as I asked repeatedly on slack, can you please identify what is in the response body by looking at the verification logs. Yes, the encoding error could be handled better by the code, however, it is being caused by the wrong response being returned by the provider. You need to fix the 403 error first.

bethesque · 2018-08-02T23:20:34Z

In response to your question on the mailing list, the authorization headers will be sent through with all the other headers. You must set up the right data on the provider using the provider states feature so that when the token is sent through, a matching user exists, so that the code returns a 200. I see no evidence that you have used any provider states in this code.

ghost · 2018-08-03T06:32:52Z

@bethesque - Sorry , i have looked at the log file . i am able to see below error log when i verify pact using pact verfier . are you referring different log ? i have verified the real service response through browser and i can see data properly .

F[ailure/Error: expect(response_status).to eql expected_response_status
expected: 200
got: 403

             (compared using eql?)

  Failure/Error: expect(response_body).to match_term expected_response_body, diff_options
       Encoding::UndefinedConversionError:
        "\xE2" from ASCII-8BIT to UTF-8](url)

Regarding Provider States :
my understanding is i can use pact verfifer with PACT JSON with out setting up Provider states so i didn't explored much about provider states since just i have to do quick POC to convince my team to adopt pact.sure , i will go through the links you have refereed, Thank you for sharing the info.

Regarding Authorization Headers
since i am unable to run pact verifier succesfully fior for current code , i am trying to do POC on another service but that service requires authorization so i have requested on how to send below authorization details . can i send below authorization details as part of pact file or can i keep this information in --custom-provider-header pact verifier side. Matthew Fellows provided links on this info , i am following his links to understand it and implement the same. please kindly send me real example , if you have any where it shows how to send authentication info pact Json or to keep in --custom-provider-header . unable to see the real time example . Thank you in advance.

[GET https: http....com/ /v1/ forms/83359274-7ad6-4
Accept-Encoding: gzip,deflate
Authorization: Bearer 58771381-333e-334f-9604-ebf977ed7784
Content-Length: 0
Host: company.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_162)

OAUTH2.0:
CSClientUser=username
CSClientPassword=pwd
CSClientIdendification=xxxx
CSClientSecret=fffff
CSAccessTokenURI=company.com/oauth2/token](url)

mefellows · 2018-08-03T08:36:46Z

OK I've taken a look. Here are a few pieces of feedback:

For future bug reports (not just for Pact), please provide instructions (e.g. a README) on how to run the reproducible code, it wasn't immediately obvious to me (a non-native python programmer), although I'm handy enough
You should set the path in your test to /minimal-rx/v1/admin/healthcheck instead of /
You should update the verification path base url, otherwise you won't be able to properly test other API endpoints on different paths
As a general rule, you should test your provider against a local instance of a provider e.g. src/provider.py instead of a live endpoint on a real test environment. Otherwise you'll run into other issues with this sort of testing down the line (e.g. managing provider states, or stubbing downstream dependecies)

The problem, from what I can gather, is that you are pointing to a domain with a self-signed certificate (https://alignapi.alta-sqa1.aligntech.com/) in your provider verification, and the process is failing because of it. I could make your tests work locally by extracting the expected response and putting into your test provider, and running the verification against it.

See the "pact-provider-verifier" section at https://github.com/pact-foundation/pact-ruby-standalone/releases on how to specify a custom SSL for the verifier.

You will probably need to talk to a Developer/Ops person in your org to assist you in finding this certificate and setting up your environment.

That being said, my suggestion is that you don't hit the live environment and hit a local one instead, thus side stepping the SSL issue.

mefellows · 2018-08-03T08:41:35Z

In answer to your authorisation details, the process would look like this, assuming you're running pact-verifier as per your example repo:

Generate an OAuth token for a user
1. you will probably need to write a script/python function to do this and make it available
Store that token in a shell environment e.g. TOKEN=$(-> run your script here to generate token <-)
Run your verifier step:
pact-verifier --provider-base-url=http://localhost:5000/ --custom-provider-header="Authorization: Bearer ${TOKEN}" ./consumer-provider.json

ghost · 2018-08-04T03:57:55Z

Hi Matt,
Thank you for your help.

First Time, i have created GitHUb repository so forgot to add ReadMe file. sincere apologies
Added the correct path /minimal-rx/v1/admin/healthcheck instead of /
3)sorry, i am unable to understand
my understanding is in pact verifier , we have to use real service provider so that it will check the real service response with pact file data. is my understanding is incorrect . please correct me

if you dont mind can you add the code how you were able to run locally in test provider . i am still unable to solve the issue.

Regarding SSL Certifcate:

i have contacted team members and as per the team, for this SSL is not required since we were able to browse the service in browser to see the data .
here is th url :https://alignapi.alta-sqa1.aligntech.com/minimal-rx/v1/admin/healthcheck

Regarding Authorization Details :

i have followed your steps on other service where it is taking OAUTH Token. thank you for this, now no issue with sending token to verifier . but unfortunaley after taking token same error is coming which i have faced in current git hub code

 Failure/Error: expect(response_status).to eql expected_response_status

       expected: 200
            got: 404

       (compared using eql?)


Failure/Error: expect(response_body).to match_term expected_response_body, diff_options, example

     Encoding::UndefinedConversionError:
       "\xE2" from ASCII-8BIT to UTF-8

Other question :
i am just wondering , is my error is related to content type ? if i call the service through Fiddler , i am seeing response related content type is Content-Type: application/vnd.siren+json

if i am doing negative scenario test(service response says url is wrong..etc), pact verfictaion is passing since Content-Type: charset=UTF-8 in service response . when ever i am doing happy path scenario(where data will be displayed )so my guess is my issue is related to Content-Type: application/vnd.siren+json. do you have any thoughts on this ?

ghost · 2018-08-05T02:33:42Z

Hi Matt,
issue is resolved . in pact verifier, i am giving url like below which is causing the issue
https://alignapi.alta-sqa1.aligntech.com/minimal-rx/v1/rx-forms/3964d0f1-52cd-476e-90f2-07b60888747e

instead if i give like below it is working

https://alignapi.alta-sqa1.aligntech.com

Thank you for your time

mefellows · 2018-08-08T12:36:34Z

Sorry for my delay I've been on leave for a few days. Glad to hear it's all working! 👍

ghost closed this as completed Aug 5, 2018

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pact verifier is throwing Error (Failure/Error: expect(response_body).to match_term expected_response_body, diff_options Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8) #90

Pact verifier is throwing Error (Failure/Error: expect(response_body).to match_term expected_response_body, diff_options Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8) #90

ghost commented Aug 2, 2018

mefellows commented Aug 2, 2018

ghost commented Aug 2, 2018 •

edited by ghost

ghost commented Aug 2, 2018

bethesque commented Aug 2, 2018

bethesque commented Aug 2, 2018

ghost commented Aug 3, 2018

mefellows commented Aug 3, 2018

mefellows commented Aug 3, 2018

ghost commented Aug 4, 2018

ghost commented Aug 5, 2018

mefellows commented Aug 8, 2018

Pact verifier is throwing Error (Failure/Error: expect(response_body).to match_term expected_response_body, diff_options Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8) #90

Pact verifier is throwing Error (Failure/Error: expect(response_body).to match_term expected_response_body, diff_options Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8) #90

Comments

ghost commented Aug 2, 2018

mefellows commented Aug 2, 2018

ghost commented Aug 2, 2018 • edited by ghost

ghost commented Aug 2, 2018

bethesque commented Aug 2, 2018

bethesque commented Aug 2, 2018

ghost commented Aug 3, 2018

mefellows commented Aug 3, 2018

mefellows commented Aug 3, 2018

ghost commented Aug 4, 2018

ghost commented Aug 5, 2018

mefellows commented Aug 8, 2018

ghost commented Aug 2, 2018 •

edited by ghost