Xpath matcher cannot match on XML with namespace prefix

[brijeshps]

I have a sample xml file, for which I am trying to match request body with xpathMatch. I have validated the xpath in an online tool "https://www.freeformatter.com/xpath-tester.html" and I was able to get a correct match . However when I put the same xpath in hovefly it's throwing an error.
"There was an error when matching
Got error: Could not find a match for request, create or record a valid matcher first! "

Please find attached the sample xml & xpath used.
Sample_xml_xpath.txt
let me know if I am missing any thing.

[tommysitu]

Thanks for reporting this issue. It looks like the XPath library hoverfly is using (https://github.com/ChrisTrenkamp/goxpath) fails to parse your XML even though it is valid. We could try to see if there is a better XPath library for Go. I think adding some error logs when a matching is failed would help to debug for the time being.

[josdeweger-zz]

Any update on this issue? I'm facing the same problem, through an online tool like https://www.freeformatter.com/xpath-tester.html the xpath query is returning the expected result, but Hoverfly can not make the match.

[JohnFDavenport]

The problem is in the underlying XML library we use as @tommysitu said on 4 Jan. We've looked to see if there are any better alternatives and haven't found any yet. We have been looking to see if there could be alternatives. Another language may have better implementation of xpath than go, so we're looking to see if we could introduce that somehow. I'm not saying it would help in every use-case though.

Just to add - I have got around issues like this a long way back by composing searches using regexes, but I also never coded anything without good examples I could use in my unit-tests, so please post them here.

[josdeweger-zz]

First of all, thanks for the quick reply, really appreciated!

What I'm looking for is a solution that is easy to read and maintain, for SOAP requests xpath looks promising, regex not so much. I'm from the .NET side of the universe ;), which has xpath support in the System.Xml namespace.

Example of what I would like to do:

Request Body:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
      <soap:Body>
            <someRequest xmlns="http://www.somenamespace.com">
                  <unit>
                           <someidentifier>123</someidentifier
                           <someotheridentifier>ABC</someotheridentifier>
                           <somedate>2018-09-21</somedate>
                           <someint>7</someint>
                   </unit>
             </someRequest>
      </soap:Body>
</soap:Envelope>

I would like to be able to make matches like
//someRequest/unit/someidentifier/text()=123 | //someRequest/unit/someotheridentifier/text()=abc

[Eynorey]

The matcher starts working as soon as you add the xml declaration <?xml version="1.0" encoding="UTF-8"?> to your request body...
Using the current version from master seems to work though!

[tommysitu]

It still doesn't seem to support xml with namespace prefix, eg. <soapenv:Envelope>, I can't get the xpath to work with the sample request in the original issue.

[tommysitu]

Lacking of support for XML namespace prefix seems to be a general Golang issue: golang/go#9519

[carsnbikes]

Hi Tommy/Benjih, Pls confirm when can we expect fix for this issue ? Thanks.

[benjih]

Hoverfly is dependent, as is a number of other tools on the underlying golang xml package. There are a number of bugs open for Namespace issues see golang/go issue #7113 and some chance of a fix.

There are several projects that use cgo to wrap libxml. This is something we have considered but we think it could be very annoying to use as it would require libxml installed on the Hoverfly host machine.

Ideally, we would like to work with someone in the Hoverfly community to help us with this.

[Rahtgaz]

Could the libxml alternative be implemented? Would it introduce breaking changes if removed later when the Go community fixed #7113?

I am currently developing a proof-of-concept for our company, and this seriously impacted the ability of Hoverfly to virtualize SOAP services, which constitutes about 50% of our use case.

[JohnFDavenport]

@Rahtgaz There seems to be little impetus to fix that issue from the Go Community. Maybe that reflects the interests of the developer’s employers and sponsors as most Go service development is focussed on API/RPC services according to this survey. Hoverfly clearly has a potential role in SOAP services as well so there is a bit of a gap to be bridged.

I’ve used libxml in the past with Python and although it is one of the best XML implementations in any language but it was a challenge to integrate and distribute. To add it to the existing Hoverfly distribution would increase this project’s support overheads as Hoverfly is cross-platform. I have some ideas but there's no point in us doing anything unless we can make it a viable project. We need to gauge interest and need. Contact me please to discuss. (See my profile page.).

[v1bh0r]

Hi, it's been a while since this issue was reported. I noticed that the library, goxpath, has been deprecated. https://github.com/ChrisTrenkamp/xsel is the new recommendation. Xsel supports namespaces. I tried the feature using the CLI utility in order to confirm it.

[tommysitu]

thanks @v1bh0r. will try doing an upgrade

[tommysitu]

this PR should fix it #994

[v1bh0r]

@tommysitu you are a rock star! I would never have imagined such a quick turnaround on this one. I'll give the change a try.