use MatchJSON in tests wherever needed #3523

girishramnani · 2020-07-08T23:29:39Z

What type of PR is this?

What does does this PR do / why we need it:
Uses the standard way to compare json output

How to test changes / Special notes to the reviewer:
Tests should plass

openshift-ci-robot · 2020-07-08T23:29:41Z

@girishramnani: The label(s) kind/area/windows cannot be applied, because the repository doesn't have them

In response to this:

What type of PR is this?

Uncomment only one /kind line, and delete the rest.
For example, > /kind bug would simply become: /kind bug

/kind failing-test
/kind area/Windows

What does does this PR do / why we need it:

Which issue(s) this PR fixes:

Fixes ##3499

How to test changes / Special notes to the reviewer:
Failing tests should pass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

girishramnani · 2020-07-09T01:52:08Z

/retest

codecov · 2020-07-09T04:16:48Z

Codecov Report

Merging #3523 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3523   +/-   ##
=======================================
  Coverage   46.18%   46.18%           
=======================================
  Files         112      112           
  Lines       11333    11333           
=======================================
  Hits         5234     5234           
  Misses       5593     5593           
  Partials      506      506

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80affdd...1f2da1d. Read the comment docs.

amitkrout

@prietyc123 Please verify it on windows. I am holding it for now, you can lift the hold after successful verification

/hold

prietyc123 · 2020-07-09T06:15:59Z

@girishramnani IMO issue #3499 is not because of different json struct order, there is some missing attribute which I verified using FIt to a particular spec that is being changed in your pr and still facing the same error.

 ginkgo .\tests\integration\
Running Suite: Integration Suite
================================
Random Seed: 1594274892
Will run 2 of 177 specs

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSCreated dir: C:\Users\Admin\AppData\Local\Temp\831810071
Creating a new project: mjmbuxbujz
Running odo.exe with args [odo project create mjmbuxbujz -w -v4]
[odo]  -  Waiting for project to come up  ...
[odo] I0709 06:08:24.188171    9444 occlient.go:543] Status of creation of project mjmbuxbujz is Active
[odo] I0709 06:08:24.269209    9444 occlient.go:548] Project mjmbuxbujz now exists
[odo] I0709 06:08:24.274117    9444 occlient.go:583] Status of creation of service account &ServiceAccount{ObjectMeta:{default  mjmbuxbujz /api/v1/namespaces/mjmbuxbujz/serviceaccounts/default cf43b679-c1a9-11ea-a594-fa163e9b6334 19192482 0 2020-07-09 06:02:45 +0000 GMT <nil> <nil> map[] map[] [] []  []},Secrets:[]ObjectReference{ObjectReference{Kind:,Namespace:,Name:default-token-nnqlx,UID:,APIVersion:,ResourceVersion:,FieldPath:,},ObjectReference{Kind:,Namespace:,Name:default-dockercfg-ddnms,UID:,APIVersion:,ResourceVersion:,FieldPath:,},},ImagePullSecrets:[]LocalObjectReference{LocalObjectReference{Name:default-dockercfg-ddnms,},},AutomountServiceAccountToken:nil,} is ready
 V  Waiting for project to come up [933ms]
[odo]  V  Project 'mjmbuxbujz' is ready for use
[odo]  V  New project created and now using project: mjmbuxbujz
[odo] I0709 06:08:24.311216    9444 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
Current working dir: C:\Users\Admin\go\src\github.com\openshift\odo\tests\integration
Setting current dir to: C:\Users\Admin\AppData\Local\Temp\831810071
Running odo.exe with args [odo create nodejs cmp-git --project mjmbuxbujz --git https://github.com/openshift/nodejs-ex --context C:\Users\Admin\AppData\Local\Temp\831810071 --app testing]
[odo] Validation
 V  Validating component [11ms]
[odo]
[odo] Please use `odo push` command to create the component with source deployed
Running odo.exe with args [odo url create url-1 --context C:\Users\Admin\AppData\Local\Temp\831810071]
[odo]  V  URL url-1 created for component: cmp-git
[odo]
[odo] To apply the URL configuration changes, please use `odo push`
Running odo.exe with args [odo url create url-2 --context C:\Users\Admin\AppData\Local\Temp\831810071]
[odo]  V  URL url-2 created for component: cmp-git
[odo]
[odo] To apply the URL configuration changes, please use `odo push`
Running odo.exe with args [odo storage create storage-1 --size 1Gi --path /data1 --context C:\Users\Admin\AppData\Local\Temp\831810071]
[odo]  V  Added storage storage-1 to cmp-git
[odo]
[odo] Please use `odo push` command to make the storage accessible to the component
Running odo.exe with args [odo describe --context C:\Users\Admin\AppData\Local\Temp\831810071]
[odo] Component Name: cmp-git
[odo] Type: nodejs
[odo] Source: https://github.com/openshift/nodejs-ex
[odo] Storage:
[odo]  · storage-1 of size 1Gi mounted to /data1
[odo] URLs:
[odo]  · URL named url-1 will be exposed via 8080/TCP
[odo]  · URL named url-2 will be exposed via 8080/TCP
Running odo.exe with args [odo describe -o json --context C:\Users\Admin\AppData\Local\Temp\831810071]
[odo] {
[odo]     "kind": "Component",
[odo]     "apiVersion": "odo.dev/v1alpha1",
[odo]     "metadata": {
[odo]         "name": "cmp-git",
[odo]         "namespace": "mjmbuxbujz",
[odo]         "creationTimestamp": null
[odo]     },
[odo]     "spec": {
[odo]         "app": "testing",
[odo]         "type": "nodejs",
[odo]         "source": "https://github.com/openshift/nodejs-ex",
[odo]         "sourceType": "git",
[odo]         "urls": {
[odo]             "kind": "List",
[odo]             "apiVersion": "odo.dev/v1alpha1",
[odo]             "metadata": {},
[odo]             "items": [
[odo]                 {
[odo]                     "kind": "url",
[odo]                     "apiVersion": "odo.dev/v1alpha1",
[odo]                     "metadata": {
[odo]                         "name": "url-1",
[odo]                         "creationTimestamp": null
[odo]                     },
[odo]                     "spec": {
[odo]                         "port": 8080,
[odo]                         "secure": false
[odo]                     },
[odo]                     "status": {
[odo]                         "state": "Not Pushed"
[odo]                     }
[odo]                 },
[odo]                 {
[odo]                     "kind": "url",
[odo]                     "apiVersion": "odo.dev/v1alpha1",
[odo]                     "metadata": {
[odo]                         "name": "url-2",
[odo]                         "creationTimestamp": null
[odo]                     },
[odo]                     "spec": {
[odo]                         "port": 8080,
[odo]                         "secure": false
[odo]                     },
[odo]                     "status": {
[odo]                         "state": "Not Pushed"
[odo]                     }
[odo]                 }
[odo]             ]
[odo]         },
[odo]         "storages": {
[odo]             "kind": "List",
[odo]             "apiVersion": "odo.dev/v1alpha1",
[odo]             "metadata": {},
[odo]             "items": [
[odo]                 {
[odo]                     "kind": "storage",
[odo]                     "apiVersion": "odo.dev/v1alpha1",
[odo]                     "metadata": {
[odo]                         "name": "storage-1",
[odo]                         "creationTimestamp": null
[odo]                     },
[odo]                     "spec": {
[odo]                         "size": "1Gi",
[odo]                         "path": "/data1"
[odo]                     }
[odo]                 }
[odo]             ]
[odo]         },
[odo]         "ports": [
[odo]             "8080/TCP"
[odo]         ]
[odo]     },
[odo]     "status": {
[odo]         "state": "Not Pushed"
[odo]     }
[odo] }
Deleting project: mjmbuxbujz
Running odo.exe with args [odo project delete mjmbuxbujz -f]
[odo]  V  Deleted project : mjmbuxbujz
[odo]  !  Warning! Projects are deleted from the cluster asynchronously. Odo does its best to delete the project. Due to multi-tenant clusters, the project 
may still exist on a different node.
Setting current dir to: C:\Users\Admin\go\src\github.com\openshift\odo\tests\integration
Deleting dir: C:\Users\Admin\AppData\Local\Temp\831810071

------------------------------
+ Failure [3.781 seconds]
odo component command tests
C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_cmp_test.go:13
  Creating component
  C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/component.go:52
    should describe the component when it is not pushed [It]
    C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/component.go:233

    Expected
        <string>: {
          "apiVersion": "odo.dev/v1alpha1",
          "kind": "Component",
          "metadata": {
            "creationTimestamp": null,
            "name": "cmp-git",
            "namespace": "mjmbuxbujz"
          },
          "spec": {
            "app": "testing",
            "ports": [
              "8080/TCP"
            ],
            "source": "https://github.com/openshift/nodejs-ex",
            "sourceType": "git",
            "storages": {
              "apiVersion": "odo.dev/v1alpha1",
              "items": [
                {
                  "apiVersion": "odo.dev/v1alpha1",
                  "kind": "storage",
                  "metadata": {
                    "creationTimestamp": null,
                    "name": "storage-1"
                  },
                  "spec": {
                    "path": "/data1",
                    "size": "1Gi"
                  }
                }
              ],
              "kind": "List",
              "metadata": {}
            },
            "type": "nodejs",
            "urls": {
              "apiVersion": "odo.dev/v1alpha1",
              "items": [
                {
                  "apiVersion": "odo.dev/v1alpha1",
                  "kind": "url",
                  "metadata": {
                    "creationTimestamp": null,
                    "name": "url-1"
                  },
                  "spec": {
                    "port": 8080,
                    "secure": false
                  },
                  "status": {
                    "state": "Not Pushed"
                  }
                },
                {
                  "apiVersion": "odo.dev/v1alpha1",
                  "kind": "url",
                  "metadata": {
                    "creationTimestamp": null,
                    "name": "url-2"
                  },
                  "spec": {
                    "port": 8080,
                    "secure": false
                  },
                  "status": {
                    "state": "Not Pushed"
                  }
                }
              ],
              "kind": "List",
              "metadata": {}
            }
          },
          "status": {
            "state": "Not Pushed"
          }
        }
    to match JSON of
        <string>: {
          "apiVersion": "odo.dev/v1alpha1",
          "kind": "Component",
          "metadata": {
            "creationTimestamp": null,
            "name": "cmp-git",
            "namespace": "mjmbuxbujz"
          },
          "spec": {
            "app": "testing",
            "ports": [
              "8080/TCP"
            ],
            "source": "https://github.com/openshift/nodejs-ex",
            "sourceType": "git",
            "storages": {
              "apiVersion": "odo.dev/v1alpha1",
              "items": [
                {
                  "apiVersion": "odo.dev/v1alpha1",
                  "kind": "storage",
                  "metadata": {
                    "creationTimestamp": null,
                    "name": "storage-1"
                  },
                  "spec": {
                    "path": "/data1",
                    "size": "1Gi"
                  }
                }
              ],
              "kind": "List",
              "metadata": {}
            },
            "type": "nodejs",
            "urls": {
              "apiVersion": "odo.dev/v1alpha1",
              "items": [
                {
                  "apiVersion": "odo.dev/v1alpha1",
                  "kind": "url",
                  "metadata": {
                    "creationTimestamp": null,
                    "name": "url-1"
                  },
                  "spec": {
                    "kind": "route",
                    "port": 8080,
                    "secure": false
                  },
                  "status": {
                    "state": "Not Pushed"
                  }
                },
                {
                  "apiVersion": "odo.dev/v1alpha1",
                  "kind": "url",
                  "metadata": {
                    "creationTimestamp": null,
                    "name": "url-2"
                  },
                  "spec": {
                    "kind": "route",
                    "port": 8080,
                    "secure": false
                  },
                  "status": {
                    "state": "Not Pushed"
                  }
                }
              ],
              "kind": "List",
              "metadata": {}
            }
          },
          "status": {
            "state": "Not Pushed"
          }
        }
    
    first mismatched key: "spec"."urls"."items"[0]."spec"

    C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/component.go:254

[Fail] odo component command tests Creating component [It] should describe the component when it is not pushed 
C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/component.go:254

Running on windows 3.11 cluster PSI.

kadel · 2020-07-09T07:59:04Z

Even if this doesn't fix #3499 using jsonmarch it is still the right thing to do.

/approve

openshift-ci-robot · 2020-07-09T07:59:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kadel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~tests/OWNERS~~ [kadel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

prietyc123 · 2020-07-09T08:15:02Z

Even if this doesn't fix #3499 using jsonmarch it is still the right thing to do.

/approve

Ok, then its better to change all the occurence in our test suite. @girishramnani Can you please update the description field accordingly so that it won't close the actual issue.

girishramnani · 2020-07-09T08:19:14Z

@prietyc123 yes I have changed all the necessary occurrences

girishramnani · 2020-07-09T08:21:09Z

/hold cancel

prietyc123 · 2020-07-09T08:44:27Z

/kind failing-test
/kind area/Windows

@girishramnani It seems a code refactoring pr. Can you please update this also.

girishramnani · 2020-07-09T08:48:06Z

Done

prietyc123 · 2020-07-09T08:54:45Z

/lgtm

openshift-bot · 2020-07-09T11:34:13Z

/retest