assert: Add support for Map and Set in deepEqual #12142

josephg · 2017-03-31T04:00:51Z

assert.deepEqual and assert.deepStrictEqual currently return true for
any pair of Maps and Sets regardless of content. This patch adds
support in deepEqual and deepStrictEqual to verify the contents of Maps
and Sets.

Unfortunately because there's no way to pairwise fetch set values or map
values which are equivalent but not reference-equal, this change
currently only supports reference equality checking in set values and
map keys. Equivalence checking could be done, but it would be an
O(n^2) operation, and worse, it would get slower exponentially if maps
and sets were nested.

Update: Based on conversation below, the implementation now incurs an O(n^2) cost if the sets or maps are not equal, because to be correct we need to check all pairs of values in sets, and all pairs of keys in a map. This may become exponentially more expensive with input which does deep equality checking on nested maps. But given these checks are almost always only done during testing, and on small objects its probably the right approach.

Update 2: Based on a clever suggestion by @TimothyGu the O(n^2) cost is now only paid when you're either in non-strict mode or when you're using set values & map keys which have typeof object.

Note that this change breaks compatibility with previous versions of
deepEqual and deepStrictEqual if consumers were depending on all maps
and sets to be seen as equivalent. The old behaviour was never
documented, but nevertheless there are certainly some tests out there somewhere
which depend on it.

Support had been stalled because the assert API was frozen, but assert was recently
unfrozen in CTC#63

Fixes: #2309
Refs: tape-testing/tape#342
Refs: #2315
Refs: nodejs/CTC#63

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

Assert

assert.deepEqual and assert.deepStrictEqual currently return true for any pair of Maps and Sets regardless of content. This patch adds support in deepEqual and deepStrictEqual to verify the contents of Maps and Sets. Unfortunately because there's no way to pairwise fetch set values or map values which are equivalent but not reference-equal, this change currently only supports reference equality checking in set values and map key values. Equivalence checking could be done, but it would be an O(n^2) operation, and worse, it would get slower exponentially if maps and sets were nested. Note that this change breaks compatibility with previous versions of deepEqual and deepStrictEqual if consumers were depending on all maps and sets to be seen as equivalent. The old behaviour was never documented, but nevertheless there are certainly some tests out there which depend on it. Support has stalled because the assert API was frozen, but was recently unfrozen in CTC#63 Fixes: nodejs#2309 Refs: tape-testing/tape#342 Refs: nodejs#2315 Refs: nodejs/CTC#63

Trott · 2017-03-31T04:09:37Z

CI: https://ci.nodejs.org/job/node-test-pull-request/7134/

addaleax · 2017-03-31T04:06:11Z

lib/assert.js

+
+function isSet(object) {
+  return object.constructor === Set;
+}


Can you use isMap and isSet from process.binding('util') instead? These checks won’t work for subclasses of Map/Set or Maps/Sets from other VM contexts.

addaleax · 2017-03-31T04:09:29Z

lib/assert.js

+  // it would require scanning each pairwise set of not strict-equal items,
+  // which is O(n^2), and would get exponentially worse if sets are nested. So
+  // for now we simply won't support deep equality checking set items or map
+  // keys.


I think I would actually prefer the awful performance over always using strict equality… @nodejs/collaborators thoughts?

In this case, I think I'd value correct over performant, so yeah, I agree with @addaleax.

For tests, I agree. I'm nervous people might be using assert.deepEqual inside runtime code, although I just checked the node_modules directory of a project I'm working on with ~400 transitive dependancies and only jsprim calls assert.deepEqual outside of a test, and thats a in a very minor way. I'll wait for some more feedback, but unless there's any strong objections I'll change the behaviour to be correct and slow.

... Another strong argument in favor of making it correct > fast is that it'll be more compatible with the current implementation. If someone currently has a test that reads assert.deepStrictEqual(new Set([{x:5}]), new Set([{x:5}])) that test will currently pass because the sets won't be compared. With the PR in its current state, that test will start to fail.

@josephg Maybe just update this PR and see if somebody objects? If we need to go back, the current HEAD is at f051840.

#14258 just landed and reduces the complexity to O(n log n). Therefore the performance should not be of much concern anymore.

Updated PR based on comments Ref: nodejs#12142

Trott

LGTM if CI comes back green and with or without addression @addaleax's comments (but preferably with!)

This change updates the checks for deep equality checking on Map and Set to check all set values / all map keys to see if any of them match the expected result. This change is much slower, but based on the conversation in the pull request its probably the right approach. Ref: nodejs#12142

mcollina

LGTM

josephg · 2017-03-31T08:21:39Z

I've updated the PR to do the quadratic pairwise checking based on the conversation above.

addaleax

Awesome PR. 👍

addaleax · 2017-03-31T08:19:39Z

lib/assert.js

+  // deepEqual. The implementation currently returns false, which is a simpler
+  // and faster implementation.
+  if (a.size !== b.size)
+    return false;


👍 I think deepEqual should be correct to reject sets of different sizes.

addaleax · 2017-03-31T08:20:32Z

lib/assert.js

+
+    // This check is not strictly necessary, but its here to improve
+    // performance of the common case when reference-equal keys exist (which
+    // includes all primitive valued keys).


nit: primitive-valued

addaleax · 2017-03-31T08:24:58Z

@nodejs/ctc semver-major or semver-minor or semver-minor but all dont-land labels?

targos · 2017-03-31T08:45:21Z

lib/assert.js

@@ -307,6 +387,22 @@ function objEquiv(a, b, strict, actualVisitedObjects) {
      return false;
  }

+  // Sets and maps don't have their entries accessable via normal object


s/accessable/accessible/

targos · 2017-03-31T08:49:00Z

I think it's semver-major. This can and probably will break something.

vsemozhetbyt · 2017-03-31T09:43:24Z

@josephg A nit: traditionally, commit title should use lower case imperative verb after subsystem prefix. Buth this can be fixed by anybody who will land the commits.

evanlucas · 2017-03-31T14:42:03Z

lib/assert.js

+    return false;
+
+  var val1, val2;
+  outer: for (val1 of a) {


Can you use const here?

evanlucas · 2017-03-31T14:42:13Z

lib/assert.js

+      // hunting for something thats deep-equal to it. Note that this is O(n^2)
+      // complexity, and will get slower if large, very similar sets / maps are
+      // nested inside. Unfortunately there's no real way around this.
+      for (val2 of b) {


Can you use const here?

evanlucas · 2017-03-31T14:43:11Z

lib/assert.js

+    return false;
+
+  var key1, key2, item1, item2;
+  outer: for ([key1, item1] of a) {


can you use const here as well?

evanlucas · 2017-03-31T14:44:05Z

lib/assert.js

+
+    // Hunt for keys which are deep-equal to key1 in b. Just like setEquiv
+    // above, this hunt makes this function O(n^2).
+    for ([key2, item2] of b) {


can you use const here as well?

evanlucas · 2017-03-31T14:45:58Z

test/parallel/test-assert-deep.js

+  const m2 = new Map();
+  m1.x = 5;
+  assertNotDeepOrStrict(m1, m2);
+}


Maybe add a case where there is a circular reference?

Cleaned up as per comments in issue Ref: nodejs#6416

TimothyGu · 2017-03-31T22:56:45Z

lib/assert.js

+      // The value doesn't exist in the second set by reference, so we'll go
+      // hunting for something thats deep-equal to it. Note that this is O(n^2)
+      // complexity, and will get slower if large, very similar sets / maps are
+      // nested inside. Unfortunately there's no real way around this.


Is it possible to optimize this by only starting the additional search if !strict || typeof val1 === 'object'?

Yes. Thats really clever, and very obvious now that you've pointed it out. Fixed.

TimothyGu · 2017-03-31T22:57:18Z

lib/assert.js

+    }
+
+    // Hunt for keys which are deep-equal to key1 in b. Just like setEquiv
+    // above, this hunt makes this function O(n^2).


Based on comments in the PR, this change restricts an O(n^2) to only happen when your set contains object-like objects, your map contains object-like keys or you're not in strict mode. ref: nodejs#12142 (review)

addaleax · 2017-03-31T23:46:25Z

lib/assert.js

-      for (const val2 of b) {
-        if (_deepEqual(val1, val2, strict, actualVisitedObjects)) {
-          continue outer;
+      if (!strict || typeof val1 === 'object') {


(!strict || (typeof val1 === 'object' && val1 !== null) || typeof val1 === 'function')?

We don't need to search for functions because the only way deepEqual will consider functions equivalent is through reference equality. The null check makes some sense, although I think it'll make a very small difference in practice because putting null in a set and using null as a key are rare. (In my code, null popping up in either of those places is usually an indication of a bug)

Its a minor point, but do you think its still worth putting in the null check? I'm indifferent either way, but happy to add it if you think the marginal speed improvement is worth the marginal extra complexity of that check.

@josephg Yes, I’d say it’s worth it… there might be cases where the difference is not so marginal, and I think it’s okay for Node core to sacrifice a tiny bit of readability for a small performance gain.

ref: https://github.com/nodejs/node/pull/12142/files/8fb6ebf9745badfb2f3f6a72cef211ff8bfbf6ba

TimothyGu · 2017-04-01T00:23:24Z

Not sure how important this is to the discussion. There are also some subtle differences in SameValueZero (the set of comparison semantics .has() uses), abstract equality (==, the operator assert.equal uses), and strict equality (===, the operator assert.strictEqual uses). Concretely, this means code such as the following may yield surprising results:

assert.equal(NaN, NaN);
    // AssertionError: NaN === NaN
assert.strictEqual(NaN, NaN);
    // AssertionError: NaN === NaN

const map1 = new Map([[NaN, 0]]);
const map2 = new Map([[NaN, 0]]);
assert.deepEqual(map1, map2);
     // No error
assert.strictDeepEqual(map1, map2);
     // No error

josephg · 2017-04-01T14:27:04Z

@joyeecheung I've cleaned up the documentation. Left the little inline YAML changelog alone for now.

joyeecheung · 2017-04-01T15:34:14Z

lib/assert.js

@@ -280,9 +282,89 @@ function _deepEqual(actual, expected, strict, memos) {
  memos.actual.push(actual);
  memos.expected.push(expected);

+


Just noticed..unrelated white space change?

joyeecheung · 2017-04-01T15:47:19Z

@josephg I believe the format for a not-yet-released changelog is...

changes:
  - version: REPLACEME
    pr-url: https://github.com/nodejs/node/pull/12142
    description: ....

@joyeecheung

- Added changes: entries in assert API documentation - Refactored setEquiv based on @joyeecheung's comments ref: nodejs#12142

joyeecheung · 2017-04-01T15:57:50Z

doc/api/assert.md

@@ -148,6 +157,21 @@ assert.deepStrictEqual(date, fakeDate);
 // Different type tags
 ```

+An exception is made to the strict equality comparison rule for [`Map`][] keys


This should probably go to the item 1 above...also the NaN case can be demonstrated by linking to the existing caveats section, something like

1. For Sets and Maps, primitive values are compared using the [SameValueZero][] comparison (which means they are free of the [caveats][]). For other objects, primitive values are compared using the [Strict Equality Comparison][] ( `===` )

joyeecheung · 2017-04-01T16:00:36Z

doc/api/assert.md

@@ -18,6 +18,9 @@ An alias of [`assert.ok()`][].
 <!-- YAML
 added: v0.1.21
 changes:
+  - version: REPLACEME
+    pr-url: https://github.com/nodejs/node/pull/12142
+    description: Sets and Maps are handled correctly.


"handled correctly" is a bit vague..maybe "elements in Sets and Maps are compared during deep comparison" or something like that?

joyeecheung

minus some nits, I think this can go ahead now...thanks for bearing with me and for the contribution, your hard work is defnitely appreciated :D

ref: nodejs#12142

josephg · 2017-04-01T16:11:11Z

Cheers :)

@joyeecheung

Refactored setEquiv and mapEquiv based on @joyeecheung's stylistic preference to avoid labels.

josephg · 2017-04-01T17:01:36Z

@joyeecheung I've rewritten mapEquiv and setEquiv based on our conversation above. The tight if() statement logic we were talking about turned out to be completely unreadable. As proof that it was a bad idea neither of us was able to produce working code of that form. Even my final attempt had a small mistake and I completely confused myself trying to fix it, so I abandoned that style completely. The current code passed all tests on its first run.

I think its all very readable now - although using ancillary functions makes it more wordy than it was using loop labels. I'm not sure the code is any better for it. But, we're enough opinions deep that I'm getting exhausted from all this churn on such a minor patch. If anyone wants to make more purely stylistic changes I'm happy to add you as a contributor to josephg/node. You're welcome to modify the PR directly.

Otherwise I don't know the process, but I'm happy for it to be merged.

joyeecheung · 2017-04-01T17:53:56Z

@josephg Sorry if my reviews are disheartening and thanks for sticking with this PR!

CI: https://ci.nodejs.org/job/node-test-pull-request/7153/
CITGM since it's semver-major: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/683/

Oudated reviews..

addaleax · 2017-04-03T08:19:53Z

Landed in 6481c93, and again thanks for the PR!

assert.deepEqual and assert.deepStrictEqual currently return true for any pair of Maps and Sets regardless of content. This patch adds support in deepEqual and deepStrictEqual to verify the contents of Maps and Sets. Deeo equivalence checking is currently an O(n^2) operation, and worse, it gets slower exponentially if maps and sets were nested. Note that this change breaks compatibility with previous versions of deepEqual and deepStrictEqual if consumers were depending on all maps and sets to be seen as equivalent. The old behaviour was never documented, but nevertheless there are certainly some tests out there which depend on it. Support has stalled because the assert API was frozen, but was recently unfrozen in CTC#63. --- Later squashed in: This change updates the checks for deep equality checking on Map and Set to check all set values / all map keys to see if any of them match the expected result. This change is much slower, but based on the conversation in the pull request its probably the right approach. Fixes: #2309 Refs: tape-testing/tape#342 Refs: #2315 Refs: nodejs/CTC#63 PR-URL: #12142 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

josephg · 2017-04-04T11:20:58Z

Yay! Thanks for helping me get my first PR in node everyone!

This fixes a bug where deepEqual and deepStrictEqual would have incorrect behaviour in sets and maps containing multiple equivalent keys. Fixes: nodejs#13347 Refs: nodejs#12142

This fixes a bug where deepEqual and deepStrictEqual would have incorrect behaviour in sets and maps containing multiple equivalent keys. PR-URL: nodejs#13426 Fixes: nodejs#13347 Refs: nodejs#12142 Reviewed-By: Refael Ackermann <refack@gmail.com>

This fixes a bug where deepEqual and deepStrictEqual would have incorrect behaviour in sets and maps containing multiple equivalent keys. PR-URL: #13426 Fixes: #13347 Refs: #12142 Reviewed-By: Refael Ackermann <refack@gmail.com>

nodejs-github-bot added the assert Issues and PRs related to the assert subsystem. label Mar 31, 2017

Trott added the semver-minor PRs that contain new features and should be released in the next minor version. label Mar 31, 2017

addaleax approved these changes Mar 31, 2017

View reviewed changes

addaleax requested review from evanlucas, Trott and joyeecheung March 31, 2017 04:10

assert: Use isSet and isMap from process.binding

f051840

Updated PR based on comments Ref: nodejs#12142

Trott approved these changes Mar 31, 2017

View reviewed changes

mcollina approved these changes Mar 31, 2017

View reviewed changes

addaleax approved these changes Mar 31, 2017

View reviewed changes

assert: Fixed nit in code review

800ae46

targos reviewed Mar 31, 2017

View reviewed changes

assert: Fixed typo in comment

031f6f3

evanlucas previously requested changes Mar 31, 2017

View reviewed changes

assert: var -> const and added tests

d6baaee

Cleaned up as per comments in issue Ref: nodejs#6416

TimothyGu reviewed Mar 31, 2017

View reviewed changes

assert: restrict O(n^2) search

8fb6ebf

Based on comments in the PR, this change restricts an O(n^2) to only happen when your set contains object-like objects, your map contains object-like keys or you're not in strict mode. ref: nodejs#12142 (review)

addaleax reviewed Mar 31, 2017

View reviewed changes

addaleax added semver-major PRs that contain breaking changes and should be released in the next major version. and removed semver-minor PRs that contain new features and should be released in the next minor version. labels Mar 31, 2017

addaleax added this to the 8.0.0 milestone Mar 31, 2017

assert: added null check in set/map short circuit

acef701

ref: https://github.com/nodejs/node/pull/12142/files/8fb6ebf9745badfb2f3f6a72cef211ff8bfbf6ba

joyeecheung reviewed Apr 1, 2017

View reviewed changes

assert: Updated based on PR

7bc29b0

- Added changes: entries in assert API documentation - Refactored setEquiv based on @joyeecheung's comments ref: nodejs#12142

joyeecheung reviewed Apr 1, 2017

View reviewed changes

joyeecheung approved these changes Apr 1, 2017

View reviewed changes

josephg added 2 commits April 2, 2017 02:07

assert: updated docs based on PR

6bdfcaf

ref: nodejs#12142

assert: fixed docs # link

fc5196a

assert: refactored based on PR

7f9d4d8

Refactored setEquiv and mapEquiv based on @joyeecheung's stylistic preference to avoid labels.

addaleax closed this Apr 3, 2017

jasnell mentioned this pull request Apr 4, 2017

8.0.0 Release Proposal #12220

Closed

Trott mentioned this pull request Apr 11, 2017

test: complete coverage for lib/assert.js #12239

Closed

3 tasks

TimothyGu mentioned this pull request May 5, 2017

assert: improve deepEqual perf for large input #12849

Closed

4 tasks

vsemozhetbyt mentioned this pull request May 31, 2017

assert: deepEqual of two Sets with different content passes #13347

Closed

josephg deleted the assert-sets-and-maps branch June 2, 2017 23:53

josephg mentioned this pull request Jun 3, 2017

assert: fix deepEqual similar sets and maps bug #13426

Closed

3 tasks

		@@ -280,9 +282,89 @@ function _deepEqual(actual, expected, strict, memos) {
		memos.actual.push(actual);
		memos.expected.push(expected);

assert: Add support for Map and Set in deepEqual #12142

assert: Add support for Map and Set in deepEqual #12142

Conversation

josephg commented Mar 31, 2017 • edited Loading

Checklist

Affected core subsystem(s)

Trott commented Mar 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josephg Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

josephg Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Trott left a comment • edited Loading

Choose a reason for hiding this comment

mcollina left a comment

Choose a reason for hiding this comment

josephg commented Mar 31, 2017

addaleax left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Mar 31, 2017

Choose a reason for hiding this comment

targos commented Mar 31, 2017

vsemozhetbyt commented Mar 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimothyGu Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josephg Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

josephg Apr 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimothyGu commented Apr 1, 2017 • edited Loading

josephg commented Apr 1, 2017 • edited Loading

Choose a reason for hiding this comment

joyeecheung commented Apr 1, 2017

joyeecheung Apr 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joyeecheung left a comment

Choose a reason for hiding this comment

josephg commented Apr 1, 2017

josephg commented Apr 1, 2017 • edited Loading

joyeecheung commented Apr 1, 2017

addaleax commented Apr 3, 2017

josephg commented Apr 4, 2017

josephg commented Mar 31, 2017 •

edited

Loading

josephg Mar 31, 2017 •

edited

Loading

josephg Mar 31, 2017 •

edited

Loading

Trott left a comment •

edited

Loading

TimothyGu Mar 31, 2017 •

edited

Loading

josephg Mar 31, 2017 •

edited

Loading

josephg Apr 1, 2017 •

edited

Loading

TimothyGu commented Apr 1, 2017 •

edited

Loading

josephg commented Apr 1, 2017 •

edited

Loading

joyeecheung Apr 1, 2017 •

edited

Loading

josephg commented Apr 1, 2017 •

edited

Loading