deck diff shows different plugin configuration than sync creates

[odsock]

Using decK v1.8.2 (b538f70).
When updating a request-transformer plugin configuration to change list of headers to remove, the diff output is incorrect. I'm changing the config from removing one header, to removing two different headers. Syncing the change has the correct result, only the diff output is wrong (output in both diff and sync).

Original state yaml:

      remove:
        body: []
        headers:
        - x-posting-ip
        querystring: []

The change:

      remove:
        body: []
        headers:
        - accept-encoding
        - x-forwarded-for
        querystring: []

The diff output shows the original single entry being removed, and then the first of the two new entries twice:

       "headers": [
-        "x-posting-ip"
+        "accept-encoding"
+        "accept-encoding"
       ],

It should be this:

       "headers": [
-        "x-posting-ip"
+        "accept-encoding"
+        "x-forwarded-for"
       ],

Example configuration and output attached.
deck-test.zip

[GGabriele]

Hi @odsock,

thanks for highlighting this!

I dug this a bit and apparently this is related to the third-party library deck uses to compute JSON diffs, and I was able to reproduce this entirely in gojsondiff:

$ jd before.json after.json

               "headers": [
-                0: "x-posting-ip"
+                0: "accept-encoding"
+                0: "accept-encoding"
               ],

Not only that, the library seems to be pretty much unreliable when dealing with arrays, as changing the content of the initial one affects the final output as well:

$ cat before.json
...
...
                      "headers": [
                         "blabla" // instead of "x-posting-ip"
                      ],
...
...

$ jd before.json after.json

               "headers": [
+                0: "accept-encoding"
               ],

There are already some GH issues related to this (yudai/gojsondiff#30 and yudai/gojsondiff#24), but the repo has not been updated in the last 4 years, so maybe we should evaluate something else.

[GGabriele]

This looks like a good candidate:

	opts := jsondiff.DefaultConsoleOptions()
	opts.PrintTypes = false
	opts.Added = jsondiff.Tag{Begin: "+ "}
	opts.Removed = jsondiff.Tag{Begin: "- "}
	opts.Changed = jsondiff.Tag{Begin: "+ "}
	_, diffString := jsondiff.Compare(aJSON, bJSON, &opts)

updating plugin request-transformer for service 9972eec7-2aab-42c3-bc45-af1dcdfc50ef {
    "config": {
        "add": {
            "body": [],
            "headers": [],
            "querystring": []
        },
        "append": {
            "body": [],
            "headers": [],
            "querystring": []
        },
        "http_method": null,
        "remove": {
            "body": [],
            "headers": [
                + "x-posting-ip" => "accept-encoding",
                + "x-forwarded-for"
            ],
            "querystring": []
        },
        "rename": {
            "body": [],
            "headers": [],
            "querystring": []
        },
        "replace": {
            "body": [],
            "headers": [],
            "querystring": [],
            "uri": null
        }
    },
    "enabled": true,
    "id": "b5904c75-8494-420a-9865-9a93d04b5d76",
    "name": "request-transformer",
    "protocols": [
        "grpc",
        "grpcs",
        "http",
        "https"
    ],
    "service": {
        "id": "9972eec7-2aab-42c3-bc45-af1dcdfc50ef"
    },
    "tags": [
        "deck-test"
    ]
}
Summary:
  Created: 0
  Updated: 1
  Deleted: 0

Also this, although it doesn't return the matched lines:

	before, _ := jd.ReadJsonString(string(aJSON))
	after, _ := jd.ReadJsonString(string(bJSON))
	diffString := before.Diff(after).Render()

updating plugin request-transformer for service 9972eec7-2aab-42c3-bc45-af1dcdfc50ef @ ["config","remove","headers",0]
- "x-posting-ip"
+ "accept-encoding"
@ ["config","remove","headers",1]
+ "x-forwarded-for"

Summary:
  Created: 0
  Updated: 1
  Deleted: 0

[hbagdi]

@GGabriele Can we try fixing the bug in the third-party library that we are currently using? That path maybe simpler.
Changing libraries could have significant differences and break our users.

[GGabriele]

#635 has been merged. Closing this. Please feel free to comment back or open a new issue if something weird pops out!