Match Contains for objects changes to equals when running recursively

[Gokuroro]

When testing for match actual contains expected and the structure is more complex (objects containing objects) the deeper tests will not be "contains", but "equals" instead.

Working Example:

Scenario: Test 1 - Shallow contains - Works
	Given def original = { a: 1, b: 2, c: 3, d: { a: 1, b: 2 } }
	And def expected = { a: 1, c: 3, }
	Then match original contains expected

Not Working Example:

Scenario: Test 2 - Deep contains - Does not work
	Given def original = { a: 1, b: 2, c: 3, d: { a: 1, b: 2 } }
	And def expected = { a: 1, c: 3, d: { b: 2 } }
	Then match original contains expected

The second example fails with the message:
path: $.d, actual: {a=1, b=2}, expected: {b=2}, reason: actual value has 1 more key(s) than expected: {a=1}

Given it is a "contains" test, the test should not fail due to the actual having more keys than the expected (same way as it works for the first example).

[ptrthomas]

This is a "wontfix" for me since you have the following reasonable, easier to grok approach:

Given def original = { a: 1, b: 2, c: 3, d: { a: 1, b: 2 } }
And def chunk = { b: 2 }
And def expected = { a: 1, c: 3, d: '#(^chunk)' }
Then match original contains expected

feel free to challenge, or better still contribute a fix, then maybe you will understand why :P

[Gokuroro]

The challenging on my part is that when I send an input for an update method and need to check if the result is correct (and it sends extra data). THAT SAID, I'll take a look into the code and try to understand the reasoning behind it.
Thank you for your input.

[ptrthomas]

I send an input for an update method and need to check if the result is correct (and it sends extra data)

sorry that's hard to understand. can you explain that with an example and I can either advise how you can handle that with existing karate features - or be convinced that karate needs to change

[Gokuroro]

Example given would be an update/create API method which receives something like:

{
   "name": "UserName",
   "address": {
      "line1": "address line",
      "city": "My City"
   }
}

And it (or another endpoint) returns something like:

{
   "name": "UserName",
   "id": 1,
   "address": {
      "line1": "address line",
      "line2": "",
      "city": "My City"
   }
}

(line2 being an optional field for create/update, so sending it is not needed, but the GET/response will always return it)
This is a hypothetical example, in the real world there may be deeper levels of small differences like that which just manipulating the input for comparison may not be ideal.

[ptrthomas]

@Gokuroro okay, that does make sense ! I think I have fixed this in the develop branch can you check. dev instructions are in the wiki: https://github.com/intuit/karate/wiki/Developer-Guide

[Gokuroro]

Looking into it to try and do some tests around it.

Edit: Sorry, I cannot seem to make it work. I cloned and checked out the develop branch, tried running both mvn clean install -P pre-release and mvn clean install -P pre-release -DskipTests, both failing at the karate-ui tests.
If this makes any difference, I'm trying this in Linux.

Edit 2: It seems to indeed be related to Linux and the latest versions of the UI

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:compile (default-compile) on project karate-ui: Compilation failure
[ERROR] /home/gokuroro/Project/karate/karate-ui/src/main/java/com/intuit/karate/ui/ConsolePanel.java:[9,23] cannot access javafx.geometry.Insets
[ERROR]   bad class file: /home/gokuroro/.m2/repository/org/openjfx/javafx-graphics/12-ea+9/javafx-graphics-12-ea+9-linux.jar(javafx/geometry/Insets.class)
[ERROR]     class file has wrong version 55.0, should be 52.0
[ERROR]     Please remove or make sure it appears in the correct subdirectory of the classpath.

[ptrthomas]

@Gokuroro it would help if you figure out a solution else you can wait for the 0.9.5 release I guess - if anyone else is reading this thread please help. FYI the build works in CI: https://travis-ci.org/intuit/karate

[Gokuroro]

@ptrthomas I did some more tests and unfortunately I could not find a way to fix that compilation issue. I don't think I have enough knowledge to go further as of now, but if it helps I can try to think some more scenarios to add to your tests and send a pull request or something.
Thank you very much for the enhancement!

[ptrthomas]

@Gokuroro I made an edit to the developer guide wiki - so you can choose to comment out the karate-ui module without affecting karate-core would be great to see if that works for you, and it will help others

[Gokuroro]

@ptrthomas sorry for taking so long. Here is what I have currently:

• wiki was completely right, I was using Open JDK 8;
• could not install Oracle JDK 8 from Arch Linux;
• installed Oracle JDK 9 and 12;
• Oracle JDK 12 failed at karate-netty;
• Oracle JDK 9 failed at karate-demo, karate-mock-servlet and karate-jersey. Commenting them out builds it successfully;
• during the build, it seems like there were some test failures (from the quick info that went through the compilation messages) but even then I got SUCCESS for karate-parent, karate-core, karate-apache, karate-ui, karate-junit4, karate-junit5, karate-netty, karate-gatling and karate-archetype.
  I will see if I can get JDK8 working in my machine and try again. In the meantime is there anything I can contribute with?

[ptrthomas]

@Gokuroro this helps. I propose we focus on Java 9. one thing I need to do is make the karate-ui a "Java module". anyway if you can see how to get the JDK 9 build passing, even if you feed me some diagnosis - that would be a great start

[Gokuroro]

Sorry for the late reply on this one. Between 70h/week on 2 projects and a presentation for a talk I'm working on, this ended up being forgotten. I did try to work on this for a bit, but I have to admit my lack of knowledge in Java (and maven in general) do not help much. I'll try to work on this, along with testing the files open during report issues this week.

[Gokuroro]

News: I was able to have a "BUILD SUCCESS" by @ignoreing the following feature files for karate-demo:

• cats.feature
• loop.feature
• cats-xml.feature
• create-xml.feature

It looks like the service being tested for xml is returning 406 instead of 200 in some cases (which may be the sole reason for the rest of the issues, though I'm not sure).

If I change the expected status from 200 to 406, I end up getting type mismatches (but this may be expected, if the issue is actually on the service side).

I'll try adding a few positive scenarios for testing this (deep contains) in the karate-demo. How would you recommend to add scenarios where the expected is for the scenario to fail (to add some negative cases for this)?

[ptrthomas]

@Gokuroro good news, the not-contains thing is fixed: baf0cf9

[ptrthomas]

released 0.9.5

[msharma555]

What would be the assertion if I want to validate if $.oType value should either be 'A' or 'B' in response

[ptrthomas]

@msharma555 stack overflow pls: https://stackoverflow.com/a/50350442/143475