added feature for validating arrays regardless of the order of the eleme... #1

odobroiu · 2014-10-31T09:13:35Z

...nts

…ements

andreineculau · 2014-10-31T19:13:13Z

Thanks for the PR, @odobroiu despite the commit message :)

I only remember that this topic has been discussed, and that we decided for KISS - test only types and values - but not the for&against details. Do you @isakb?

Personally, I have to reflect on this change and the alternatives. I am not so fond of the PR as it is since this adds array-specific semantics that are not related to the values themselves, but is a flag for the comparison algorithm.

IFF more semantics are needed, then I'd prefer a bit more general approach. From the top of my had, create a new syntax like

GET /any_order_array
< 200
< Content-Type: application/json
{
  "list_of_objects": {
    "{{compare}}": "as_set", ~~"unordered_array"~~
    "items": [3,2,1]
  }
}

and basically have a callback compare(as_set, ParentKey, Expected, Actual, ...) to do the actual work, which opens up a different extension mechanism.

Ping @dmitriid - I guess this hasn't come to your attention begging for a fix?

EDITED: unordered_array is a set

isakb · 2014-11-01T11:22:52Z

I think the main argument was simplicity, but if there were any other reasons I have forgotten. I also am not a huge fan of the syntax here, and I think we were discussing something similar and came to the conclusion that it was not pretty enough to keep and we cursed JSON/JavaScript for not having sets.

andreineculau · 2014-11-01T17:52:29Z

@isakb thoughts on the extension proposal {{compare}}: "as_set"?

isakb · 2014-11-02T17:01:48Z

I don't really like that either, and am thinking about a totally different approach. But I guess if there is an immediate need for the functionality it would be acceptable.

andreineculau · 2014-11-02T18:58:01Z

Different approach?

On 02 Nov 2014, at 18:01, Isak Bakken notifications@github.com wrote:

I don't really like that either, and am thinking about a totally different approach. But I guess if there is an immediate need for the functionality it would be acceptable.

—
Reply to this email directly or view it on GitHub.

odobroiu · 2014-11-03T09:28:24Z

@andreineculau, sorry for the PR message... I thought the commit message and the readme was enough... but yes, I agree it would have been nice to write a few words there as well :)

About the change, I did it that way because thought it would be similar to how the {{unexpected}} tag works, in the sense that it also implies that the comparison is done either by exactly matching the 2 arrays, or by testing that the expected array is included in the actual array.

Either way, I do not have strong opinions about the sintax, but I do think that this feature should be supported( one way or the other).

About the possible options, I guess all the options so far are:

As is in PR

{
  "list_of_objects": [ {{unordered}}, 3,2,1]
}

Andrei's proposal:

{
  "list_of_objects": {
    "{{compare}}": "as_set", ~~"unordered_array"~~
    "items": [3,2,1]
  }
}

Similar, but maybe simpler

{
  "list_of_objects":  { "{{unordered_array}}":  [3,2,1] }
}

Not sure how I feel about this one, but added it here

{
  "{{unordered}}:list_of_objects": [3,2,1]
}

Also, maybe the set is not quite a good word, since that would imply elements are unique.

What do you think?

andreineculau · 2014-11-03T09:33:22Z

good Monday morning input, @odobroiu :) thanks

i sort of like the last proposal, but i need to sleep on it

On Mon, Nov 3, 2014 at 10:28 AM, odobroiu notifications@github.com wrote:

@andreineculau https://github.com/andreineculau, sorry for the PR
message... I thought the commit message and the readme was enough... but
yes, I agree it would have been nice to write a few words there as well :)

About the change, I did it that way because thought it would be similar to
how the {{unexpected}} tag works, in the sense that it also implies that
the comparison is done either by exactly matching the 2 arrays, or by
testing that the expected array is included in the actual array.

Either way, I do not have strong opinions about the sintax, but I do think
that this feature should be supported( one way or the other).

About the possible options, I guess all the options so far are:

As is in PR

{
"list_of_objects": [ {{unordered}}, 3,2,1]
}

Andrei's proposal:

{
"list_of_objects": {
"{{compare}}": "as_set", ~~"unordered_array"~~
"items": [3,2,1]
}
}

Similar, but maybe simpler

{
"list_of_objects": { "{{unordered_array}}": [3,2,1] }
}

Not sure how I feel about this one, but added it here

{
"{{unordered}}:list_of_objects": [3,2,1]
}

Also, maybe the set is not quite a good word, since that would imply
elements are unique.

What do you think?

—
Reply to this email directly or view it on GitHub
#1 (comment).

andreineculau.com http://www.andreineculau.com

odobroiu · 2014-11-04T08:34:36Z

Realized that there is some more work to do for this, as when sorting the array the initial order is lost and therefore the error message might be confusing.

Will address this after we agree on the syntax.

andreineculau · 2014-11-04T10:57:12Z

pinging @sstrigler to take a stand

On Tue, Nov 4, 2014 at 9:34 AM, odobroiu notifications@github.com wrote:

Realized that there is some more work to do for this, as when sorting the
array the initial order is lost and therefore the error message might be
confusing.

Will address this after we agree on the syntax.

—
Reply to this email directly or view it on GitHub
#1 (comment).

andreineculau.com http://www.andreineculau.com

sstrigler · 2014-11-04T12:00:05Z

Sorry, somehow I wasn't in the loop here.

As for the approach

{
  "list_of_objects": {
    "{{compare}}": "as_set", ~~"unordered_array"~~
    "items": [3,2,1]
  }
}

I wonder about alternatives for {{compare}}, don't we compare always? Then it sounds a bit redundant.

What if we'd allow are more json-schema like approach (in addition to what we already have)?

{
  "list_of_objects": {
    "type": "set",
    "value": [1,2,3]
  }
}

sstrigler · 2014-11-04T13:11:09Z

Ok, that was stupid. Next try:

{
  "list_of_objects": {
    "{{type}}": "set",
    "value": [1,2,3]
  }
}

which is pretty close to what Andrei proposed.

isakb · 2014-11-09T22:50:59Z

I was thinking that we need a JSON parser with some extensions. Then the syntax could be more readable. Something like:

GET /cache
▶ Accept: application/json
▶ User-Agent: KATT
◀ 200
◀ Content-Type: application/json
    check_exact_keys({
        "url": "http://httpbin.org/cache",
        "headers": *,
        "args": *,
        "origin": store(value, 'origin', 'string')
    })

A fifth request to show that whitespace in JSON body is insignificant.
POST /post
▶ Accept: application/json
▶ Content-Type: application/json
▶ User-Agent: KATT
    {"origin": recall('origin'),"whoarewe":"{{ recall('your_name') }}_and_{{ recall('my_name') }}", "date":"{{ recall('some_date') }}"}
◀ 200
◀ Content-Type: application/json
    {
        "args": _,
        "data": store('raw_data', 'object'),
        "files": must_have([1, 2, 3]),
        "form": must_not_have(['password']),
        "headers": _,
        "origin": recall('origin'),
        "url": "http://httpbin.org/post",
        "json": {
            "origin": recall('origin'),
            "date": recall('some_date'),
            "whoarewe": recall('whoarewe')
        }
    }

(Unicode arrows = Github flavored markdown workaround)

andreineculau · 2014-11-20T09:04:00Z

@isakb I don't know if the others have given any thought so far, I did. Similar to Apiary, love at first sight and ease of use happens because of the familiar syntax - HTTP meta + payload (JSON). Then come the assertions/extensions.

Apiary's new syntax, markdown based, and your suggestion is 180 degrees from their first grammar (which KATT is based on). The same way I was vocal about Apiary's change, I vote against this radical change. But above all, I see in my proposal the chance for more dynamic and readable extensions e.g.

{"{{type}}": "set", "value": [1,2,"{{unexpected}}"]} to match [1,2] or [2,1]
{"{{type}}": "set", "value": [1,2], "unexpected": [3]} to match [1,2] or [2,1] or [4,2,1] or ... but not [3,2,1].

But it would be good with more input :) @sstrigler @dmitriid ?

Short-term, it would be good to see a PR with say @sstrigler 's last comment, that implements support for comparison extensions, implements a "set" comparison extension, and a few tests. Do you have time @odobroiu ? If not, I'll put it as a Xmas todo for myself :)

sstrigler · 2014-11-20T13:39:33Z

@andreineculau I'm not so sure if you'll have time around Xmas as I'll be in town 💃

sstrigler · 2014-11-20T15:01:42Z

As requested my 2c.

I also think that @isakb's approach is too far away from the initial idea of using that version of apiary's blueprint file format. At least I can't see how this relates to what we have now.
I think one big strength of katt lies in it's simple use, i.e. how to describe scenarios in a intuitive and simple way.

Another thing that comes to my mind: What if we'd build in support for validating schemas (as in json-schemas for now) which then could contain those constraints?

katt:run could get a config option for that which tells it where to find schemas used in a scenario and when it sees a content-type within a response it would check the body accordingly. (just brainstorming, sorry if I don't make sense)

sstrigler · 2014-11-20T15:06:20Z

Maybe forget my last comment, it's derailing the discussion.

isakb · 2014-11-20T18:32:30Z

The proposed syntax additions are far from familiar to my mind, and seeing that only some of the special keys have the double bracket "escaping" it also violates the principle of least surprise. And since we have already broken away from APiary blueprint compatibility (or they have...) there is little reason to try to keep the original syntax unless it has some inherent benefit.

The original goal of keeping it simple was already a lost cause due to the constraint that we needed to have the KATT symbols be a part of the expected response, instead of being able to escape from it in a good way. That makes for a terse syntax with complicated workarounds that are difficult to remember. That is the reason why I was thinking of taking a different approach, but like I said if you have a need for the new syntax then I really don't mind, as I am not actually using KATT at all these days, so this change will not affect me.

Sent from my iPad

On 20 nov 2014, at 10:04, Andrei Neculau notifications@github.com wrote:

@isakb I don't know if the others have given any thought so far, I did. Similar to Apiary, love at first sight and ease of use happens because of the familiar syntax - HTTP meta + payload (JSON). Then come the assertions/extensions.

Apiary's new syntax, markdown based, and your suggestion is 180 degrees from their first grammar (which KATT is based on). The same way I was vocal about Apiary's change, I vote against this radical change. But above all, I see in my proposal the chance for more dynamic and readable extensions e.g. {"{{type}}": "set", "must_have": [1,2], "must_not_have": [3]}.

But it would be good with more input :) @sstrigler @dmitriid ?

Short-term, it would be good to see a PR with say @sstrigler 's last comment, that implements support for comparison extensions, implements a "set" comparison extension, and a few tests. Do you have time @odobroiu ? If not, I'll put it as a Xmas todo for myself :)

—
Reply to this email directly or view it on GitHub.

odobroiu · 2014-12-24T14:47:59Z

Created a new PR about this. Should probably close this one.

andreineculau · 2014-12-28T21:46:40Z

Closing on the basis of

Set matching #5 was opened instead
validate_type extension was introduced in 6a7b2b1 . Feedback welcome!

added feature for validating arrays regardless of the order of the el…

ec35747

…ements

andreineculau closed this Dec 28, 2014

andreineculau added the enhancement label Feb 15, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added feature for validating arrays regardless of the order of the eleme... #1

added feature for validating arrays regardless of the order of the eleme... #1

odobroiu commented Oct 31, 2014

andreineculau commented Oct 31, 2014

isakb commented Nov 1, 2014

andreineculau commented Nov 1, 2014

isakb commented Nov 2, 2014

andreineculau commented Nov 2, 2014

odobroiu commented Nov 3, 2014

andreineculau commented Nov 3, 2014

odobroiu commented Nov 4, 2014

andreineculau commented Nov 4, 2014

sstrigler commented Nov 4, 2014

sstrigler commented Nov 4, 2014

isakb commented Nov 9, 2014

andreineculau commented Nov 20, 2014

sstrigler commented Nov 20, 2014

sstrigler commented Nov 20, 2014

sstrigler commented Nov 20, 2014

isakb commented Nov 20, 2014

odobroiu commented Dec 24, 2014

andreineculau commented Dec 28, 2014

added feature for validating arrays regardless of the order of the eleme... #1

added feature for validating arrays regardless of the order of the eleme... #1

Conversation

odobroiu commented Oct 31, 2014

andreineculau commented Oct 31, 2014

isakb commented Nov 1, 2014

andreineculau commented Nov 1, 2014

isakb commented Nov 2, 2014

andreineculau commented Nov 2, 2014

odobroiu commented Nov 3, 2014

andreineculau commented Nov 3, 2014

odobroiu commented Nov 4, 2014

andreineculau commented Nov 4, 2014

sstrigler commented Nov 4, 2014

sstrigler commented Nov 4, 2014

isakb commented Nov 9, 2014

andreineculau commented Nov 20, 2014

sstrigler commented Nov 20, 2014

sstrigler commented Nov 20, 2014

sstrigler commented Nov 20, 2014

isakb commented Nov 20, 2014

odobroiu commented Dec 24, 2014

andreineculau commented Dec 28, 2014