Map & Set support (v5) #354

aleclarson · 2019-04-18T14:19:19Z

Try v5.0.0 today!

npm install immer@next

Tasks

mweststrate · 2019-04-18T14:56:58Z

Might be nice to have the additional tests from #353 in this branch, in case we missed something?

aigoncharov · 2019-04-18T15:48:34Z

@mweststrate I wish you had told me you wouldn't let the PR for ES6 only maps to pass through in that thread. Though, yeah, it all happened really fast :)
P.S. I have a working POC for es6 Sets. Will raise a PR with it for this branch soon.

aigoncharov · 2019-04-18T16:11:59Z

Do we want to have ES5 support or do we want to have IE11 support? I believe it's not the same thing. IE11 doesn't support iterators and lots of other methods according to MDN.
This would still require to run some tests conditionally. It still would be an issue if a project, successfully using Maps, all of a sudden has to start supporting IE11. That kind of project still would have to be refactored.
@mweststrate are you firm on not letting ES6 only implementation pass through? Since it's gonna be a major release anyway, I guess it's ok to break things. Though IE11 folks wouldn't even be broken, they'll just have to stick to the old way of working with Maps manually. What we could do in this case is to add tests that the old way still works as expected.
P.S. New MobX works only in Proxy-enabled environments, which makes me wondering if new Immer should follow the same path.

mweststrate · 2019-04-22T11:51:29Z

quick update: I'll hope to get back to it next week, travelling this week, so might not have time to look into it / answer the questions :)

…

On Thu, Apr 18, 2019 at 6:12 PM Andrey Goncharov ***@***.***> wrote: Do we want to have ES5 support or do we want to have IE11 support? I believe it's not the same thing. IE11 doesn't support iterators and lots of other methods according to MDN <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map#Browser_compatibility> . This would still require to run some tests conditionally. It still would be an issue if a project, successfully using Maps, all of a sudden has to start supporting IE11. That kind of project still would have to be refactored. @mweststrate <https://github.com/mweststrate> are you firm on not letting ES6 only implementation pass through? Since it's gonna be a major release anyway, I guess it's ok to break things. Though IE11 folks wouldn't even be broken, they'll just have to stick to the old way of working with Maps manually. What we could do in this case is to add tests that the old way still works as expected. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#354 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAN4NBHAAUNYDKWVL5PVWTTPRCMVDANCNFSM4HG5JYMQ> .

Includes support for: - mutable drafts in `produce` callbacks - patch listeners BREAKING CHANGE: `Map` objects are now drafted

Support for patch listeners included BREAKING CHANGE: `Set` objects are now drafted

These traps ensure the underlying state is never touched by the user.

aleclarson · 2019-04-23T02:00:31Z

Added patch-related functionality for ES6 Sets.

Items to discuss:

Patches for Maps will work only for string and numeric keys. Is it ok? Should we add a section to README only or have an exception thrown?
I'm not sure how compatible with RFC-6902 JSON patch standard is this record for ES6 Set patches. Currently, these patches cannot be applied in a different order.
Do we really-really-really want to support ES5? :)

TODOs:

Typings
Add tests (and most probably add missing implementation) for onDelete and onAssign hooks for cases with ES6 Maps and ES6 Sets

@aleclarson if you merge it to v4 branch, I'd be happy to carry on this discussion there, since it really does belong there.

Originally posted by @keenondrums in #358 (comment)

aleclarson · 2019-04-23T02:00:48Z

Patches for Maps will work only for string and numeric keys. Is it ok? Should we add a section to README only or have an exception thrown?

No, we should make sure changes to object keys have patches created for them, since Immer patches can be useful even when not serialized to JSON.

Note: Immer prefers working with tree-like data structures, and I'm not sure what the bugs are when relational data is used.

2. I'm not sure how compatible with RFC-6902 JSON patch standard is this record for ES6 Set patches. Currently, these patches cannot be applied in a different order.

To be clear, Immer patches aren't strictly JSON patches, but you must use JSON-only keys/values if you ever need to serialize the patches.

3. Do we really-really-really want to support ES5? :)

To not support ES5 would add unwanted complexity to the API for ES5 users. :(

Originally posted by @aleclarson in #358 (comment)

aigoncharov · 2019-04-23T13:09:08Z

@aleclarson do we want to support ES5 (read it as environments that lack Proxy support) or IE11? It's not the same thing. As MDN states IE11 doesn't support new Map([iterable]) which means we'll have to come up with a different mechanism to shallow copy Maps in IE11.
Moreover, we do not run tests specifically for IE 11 using that very IE11. It means we can not really guarantee we support it.

mweststrate · 2019-04-30T18:21:56Z

Some random thoughts:

I think it is fine to limit maps to only allow strings / numbers as keys. I think having objects as keys, which can then be updated in a producer as well, will get confusing very quickly, and just bailing out is probably better until we encounter a good RL use case

The goal is to make sure devs can use immer safely inside consumer facing applications, without sudden suprises. In practice that means that we need to support ES5, or at least, things that can be polyfilled to ES 5. So in this specific case, whether Map([iterable]) isn't that important, the important question is, does the implementation correctly after being processed by babel? If babel polyfills the map constructor correctly, it is fine. In MobX there are two major versions still being maintained, which is also not ideal. But at least, practically all new functionality in 5 can be expressed in mobx 4, without big semantic changes.

The risky semantic difference here is that sets and maps would be treated totally different as soon as they are considered draftable. So it could be an option to only support sets and maps in a next major, but I think it would be a significant maintenance burden. Mostly being helping people out that, ignorantly, npm installed the latest version, work a while with that, use maps / sets (probably without checking the docs on that at all seemed to works ootb), and then discover that they have to rewrite a lot of code after a first round of QA on IE11...

omninonsense · 2019-05-01T10:52:42Z

Patches for Maps will work only for string and numeric keys. Is it ok? Should we add a section to README only or have an exception thrown?

I don't think there's much valid use-cases outside of strings/numbers anyway. I think the main benefit of using a Map over an object is convenience (ie numbers not being coerced into strings when the keys are numeric IDs, easier/safer preservation of type information in Flow/TS).

shabbyrobe · 2019-05-15T01:14:13Z

I don't think there's much valid use-cases outside of strings/numbers anyway.

It's actually a very important use-case for Sets and Maps. Most of the time I reach for them it's for this property.

aigoncharov · 2019-05-15T10:33:54Z

@shabbyrobe I have to correct my statement. I think any valid key (including objects, symbols and etc.) is going to work with Maps. I just doubt if it's going to work with patches enabled.

mweststrate · 2019-10-23T09:05:47Z

I resolved the merge conflicts. There might be some mistakes in there, as it was quite tricky. Some tests related to patches are still failing at least, most notable is this error (does it ring a bell @runnez)?

TypeError: Method Map.prototype.forEach called on incompatible receiver #<Map>
        at Map.forEach (<anonymous>)

      113 | export function each(obj, iter) {
      114 | 	if (Array.isArray(obj) || isMap(obj) || isSet(obj)) {
    > 115 | 		obj.forEach((entry, index) => iter(index, entry, obj))
          | 		    ^
      116 | 	} else {
      117 | 		ownKeys(obj).forEach(key => iter(key, obj[key], obj))
      118 | 	}

      at forEach (src/common.js:115:7)

Coming days I won't have time to further investigate, so I'll continue after that, or if anyone beats me to it, welcome :).

Sorry, this got a lot more complicated than it ought to be as there were significant changes on master as well, but it is tricky to find them one by one due to the whitespace changes.

I think it should be possible to build the traps up a bit more efficiently, but let's get the tests passing first.

aleclarson · 2019-10-23T13:44:18Z

Sorry, this got a lot more complicated than it ought to be as there were significant changes on master as well, but it is tricky to find them one by one due to the whitespace changes.

@mweststrate Git has options for merging without whitespace conflicts (see here). Maybe you should rebase and keep the tabs in place?

mweststrate · 2019-10-23T14:05:19Z

Good one, gonna try that as well.

…

On Wed, Oct 23, 2019 at 2:44 PM Alec Larson ***@***.***> wrote: @mweststrate <https://github.com/mweststrate> Git has options for merging without whitespace conflicts (see here <https://stackoverflow.com/questions/9776527/merging-without-whitespace-conflicts>). Maybe you should rebase and keep the tabs in place? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#354?email_source=notifications&email_token=AAN4NBBM7MKV2ICWGNQ32OTQQBILJA5CNFSM4HG5JYM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECBOWTY#issuecomment-545450831>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN4NBHZHHYDJ2QU7DLXFA3QQBILJANCNFSM4HG5JYMQ> .

runnez · 2019-10-23T18:05:35Z

@mweststrate I've found a place where logic is broken. If we move back an old function, it will fail but only one test. I'll try to fix it tomorrow

immer/src/patches.js

Line 141 in ae80361

export function applyPatches(draft, patches) {

  ● applyPatches › applied patches cannot be modified

    TypeError: Cannot add/remove sealed array elements
        at Array.splice (<anonymous>)

      154 | 			const add = (key, value) =>
      155 | 				Array.isArray(base)
    > 156 | 					? base.splice(key, 0, value)
          | 					       ^
      157 | 					: isMap(base)
      158 | 					? base.set(key, value)
      159 | 					: isSet(base)

      at splice (src/patches.js:156:13)
      at add (src/patches.js:176:6)
      at src/immer.js:166:4
      at Immer.recipe (src/immer.js:74:14)
      at Immer.produce [as applyPatches] (src/immer.js:165:15)
      at Object.<anonymous> (__tests__/patch.js:105:3)

 PASS  __tests__/base.js

Test Suites: 1 failed, 10 passed, 11 total
Tests:       1 failed, 1730 passed, 1731 total
Snapshots:   143 passed, 143 total
Time:        4.014s

runnez · 2019-10-25T10:51:34Z

@mweststrate I've fixed tests, but I've got doubts about expected behaviour on this line ff25a83#diff-b40ab5eb20dfada012e789db3e9780bbR142

runnez · 2019-10-25T14:16:14Z

@mweststrate I've fixed errors in ie11, but you may revert it if need ffd8769

src/proxy.js

mweststrate · 2019-10-28T12:16:14Z

@runnez thanks for investigating! Sorry for messing up that if-else logic while merging 😅

mweststrate · 2019-10-28T14:15:53Z

No I mean it is actually safe and a good idea to set is to a property called "@@iterator" as fallback, as there are a couple of utility libraries that actually look for that in the absence of a proper Symbol.iterator

…

On Mon, Oct 28, 2019 at 1:41 PM Andrey Boldyrev ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/proxy.js <#354 (comment)>: > @@ -258,7 +259,7 @@ const mapTraps = makeTrapsForGetters({ keys: state => () => latest(state).keys(), values: iterateMapValues, entries: iterateMapValues, - [Symbol.iterator]: iterateMapValues + [hasSymbol && Symbol.iterator]: iterateMapValues I agree, it looks cleaner. But if env doesn't have Symbol, these code won't fire. Or Did you mean only readability? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#354?email_source=notifications&email_token=AAN4NBDOA65PVS5WSLY735LQQ3TYRA5CNFSM4HG5JYM2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJMXPIA#discussion_r339567389>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN4NBFP4V3YJYBC7VGI7RDQQ3TYRANCNFSM4HG5JYMQ> .

runnez · 2019-10-28T18:24:07Z

I think the condition is incorrect; even if the base is a set, and the op is add, the patch should be defended against accidental mutation, as theoretically a next patch could change that value inside that set (and otherwise, if we are going to error anyway, it doesn't matter that we didn't optimize this clone anymore)

@mweststrate did I understand you correctly? 24658bc

mweststrate

@runnez cool, thanks for the changes, that was what I was looking for indeed!

Did an in-depth review / test, and I think this is starting to look really solid now! Left a bunch of comments, but I think all the hard stuff is done :) Just pushed a few commits as well, so feel free to review them.

mweststrate · 2019-10-30T10:14:31Z

__tests__/patch.js

+			obj2.val = "you"
+		},
+		[
+			{op: "remove", path: [0], value: {id: 1, val: "We"}},


I'd expect patches here that capture that the modification was only the object inside the sets, rather than patches on the set itself, like:
op: "replace", path: [1, "val"], value: "will"

Is there a special reason the entire set entry gets reconstructed? (I agree that the concept of indices in a set can be challenged, and maybe your approach is more correct, but sets guarantee insertion order, so I think it should be ok rely on paths in sets?)

I think this quite an edge case anyway, so we could also open it as a fresh follow up issue, and see if anybody runs into the difference, before doing anything.

Ambiguity of indices in Sets was the primary reason. As you said, you can iterate over them, however, you cannot get an item at index.
Love the idea of leaving this edge case out of the PR.

src/es5.js

mweststrate · 2019-10-30T10:28:48Z

src/proxy.js

+ * Map drafts
+ */
+
+const mapTraps = makeTrapsForGetters({


Random thought: could we use the ES5 implementation, and not use Proxies for maps and sets at all? If that would simplify the implementation that would be cool. If "ES5" administration doesn't play nicely inside a proxy draft tree, than lets leave it as is. But proxies themself don't add much for maps and sets, since all object interactions are through functions anyway.

Could also be opened as follow up optimization issue, to reduce the package size

I think it's possible to use one codebase for Map and Set and use both Proxy and ES5 mechanics. Let's do it as a follow up issue

mweststrate · 2019-10-30T16:17:24Z

Perfect, I'll try to wrap up remaining things and cut a release tonight :)

…

On Wed, Oct 30, 2019 at 3:35 PM Andrey Boldyrev ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/proxy.js <#354 (comment)>: > + "ownKeys", + "has", + "set", + "deleteProperty", + "defineProperty", + "getOwnPropertyDescriptor", + "preventExtensions", + "isExtensible", + "getPrototypeOf" +]) + +/** + * Map drafts + */ + +const mapTraps = makeTrapsForGetters({ I think it's possible to use one codebase for Map and Set and use both Proxy and ES5 mechanics. Let's do it as a follow up issue — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#354?email_source=notifications&email_token=AAN4NBCTZ5EMAOAT3D65XNTQRGSVTA5CNFSM4HG5JYM2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCJXZP6Q#discussion_r340691973>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN4NBC3PVUD372EOBGKH4LQRGSVTANCNFSM4HG5JYMQ> .

aleclarson · 2019-10-30T20:38:20Z

🎉 This PR is included in version 5.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

aleclarson mentioned this pull request Apr 18, 2019

Maps are broken #353

Closed

keenondrums and others added 14 commits April 22, 2019 19:55

feat: Map drafts and patches

30db0f3

Includes support for: - mutable drafts in `produce` callbacks - patch listeners BREAKING CHANGE: `Map` objects are now drafted

chore: reorganize src/proxy.js

3163a7d

refactor: simplify mapGetters.keys

3298a77

fix: remove "function*" syntax

08d88b8

fix: add Map getter for DRAFT_STATE, make isEnumerable more resilient

7bbec0a

fix: Map.prototype.set should return a draft

7001201

nit: ensure isEnumerable returns a boolean

53d0cdf

nit: argument names

a674a78

feat: Set drafts

4355b63

Support for patch listeners included BREAKING CHANGE: `Set` objects are now drafted

refactor: traps for Maps and Sets

dd82d89

docs: es6 map support

6255a58

fix: add more traps for Maps and Sets

930b4ab

These traps ensure the underlying state is never touched by the user.

refactor: markChanged function

5857d63

nit: merge isMap and isSet checks

4164f0d

aleclarson force-pushed the v4 branch 2 times, most recently from 5e6dcf8 to c90cd57 Compare April 23, 2019 01:10

refactor: makeTrapsForGetters function

5093fd0

aleclarson force-pushed the v4 branch from c90cd57 to 5093fd0 Compare April 23, 2019 01:19

fix specs

ff25a83

fix ie11

ffd8769

mweststrate reviewed Oct 28, 2019

View reviewed changes

src/proxy.js Outdated Show resolved Hide resolved

mweststrate reviewed Oct 28, 2019

View reviewed changes

src/proxy.js Outdated Show resolved Hide resolved

runnez added 2 commits October 28, 2019 20:53

avoid creating null keys in proxies

f90b586

ensure set patch is not modified

24658bc

mweststrate added 2 commits October 30, 2019 09:41

More tests, also fixed binding of produceWithPatches

7d32b7c

Made sure maps and sets become truly immutable

82db631

mweststrate reviewed Oct 30, 2019

View reviewed changes

mweststrate added 3 commits October 30, 2019 16:21

Travis fix(?)

e819e79

Updated docs. Also fixes #445

7335671

Updated typings

f8c65f0

This was referenced Oct 30, 2019

Reduce Immer bundle size by eliminating the Proxy based implementations of Set and Map #449

Closed

Updates to objects deep inside a Set should not result in patches for the Set itself #450

Closed

Sanity fix

0f4fe3f

mweststrate approved these changes Oct 30, 2019

View reviewed changes

mweststrate merged commit b6f1d40 into master Oct 30, 2019

aleclarson added the released label Oct 30, 2019

phryneas mentioned this pull request Oct 30, 2019

update immer to v5 reduxjs/redux-toolkit#240

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Map & Set support (v5) #354

Map & Set support (v5) #354

aleclarson commented Apr 18, 2019 •

edited by mweststrate

Loading

mweststrate commented Apr 18, 2019

aigoncharov commented Apr 18, 2019 •

edited

Loading

aigoncharov commented Apr 18, 2019 •

edited

Loading

mweststrate commented Apr 22, 2019 via email

aleclarson commented Apr 23, 2019

aleclarson commented Apr 23, 2019

aigoncharov commented Apr 23, 2019 •

edited

Loading

mweststrate commented Apr 30, 2019

omninonsense commented May 1, 2019 •

edited

Loading

shabbyrobe commented May 15, 2019

aigoncharov commented May 15, 2019

mweststrate commented Oct 23, 2019

aleclarson commented Oct 23, 2019 •

edited

Loading

mweststrate commented Oct 23, 2019 via email

runnez commented Oct 23, 2019

runnez commented Oct 25, 2019

runnez commented Oct 25, 2019

mweststrate commented Oct 28, 2019

mweststrate commented Oct 28, 2019 via email

runnez commented Oct 28, 2019

mweststrate left a comment

mweststrate Oct 30, 2019

mweststrate Oct 30, 2019

aigoncharov Oct 30, 2019

mweststrate Oct 30, 2019

mweststrate Oct 30, 2019

mweststrate Oct 30, 2019

runnez Oct 30, 2019

mweststrate Oct 30, 2019

mweststrate commented Oct 30, 2019 via email

aleclarson commented Oct 30, 2019

Map & Set support (v5) #354

Map & Set support (v5) #354

Conversation

aleclarson commented Apr 18, 2019 • edited by mweststrate Loading

Try v5.0.0 today!

Tasks

mweststrate commented Apr 18, 2019

aigoncharov commented Apr 18, 2019 • edited Loading

aigoncharov commented Apr 18, 2019 • edited Loading

mweststrate commented Apr 22, 2019 via email

aleclarson commented Apr 23, 2019

aleclarson commented Apr 23, 2019

aigoncharov commented Apr 23, 2019 • edited Loading

mweststrate commented Apr 30, 2019

omninonsense commented May 1, 2019 • edited Loading

shabbyrobe commented May 15, 2019

aigoncharov commented May 15, 2019

mweststrate commented Oct 23, 2019

aleclarson commented Oct 23, 2019 • edited Loading

mweststrate commented Oct 23, 2019 via email

runnez commented Oct 23, 2019

runnez commented Oct 25, 2019

runnez commented Oct 25, 2019

mweststrate commented Oct 28, 2019

mweststrate commented Oct 28, 2019 via email

runnez commented Oct 28, 2019

mweststrate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweststrate commented Oct 30, 2019 via email

aleclarson commented Oct 30, 2019

aleclarson commented Apr 18, 2019 •

edited by mweststrate

Loading

aigoncharov commented Apr 18, 2019 •

edited

Loading

aigoncharov commented Apr 18, 2019 •

edited

Loading

aigoncharov commented Apr 23, 2019 •

edited

Loading

omninonsense commented May 1, 2019 •

edited

Loading

aleclarson commented Oct 23, 2019 •

edited

Loading