Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(model): don't alter original scopes when combining them #10722

Merged
merged 2 commits into from Jul 6, 2019

Conversation

jorgelf
Copy link
Contributor

@jorgelf jorgelf commented Apr 8, 2019

Pull Request check-list

Please make sure to review and check all of these items:

  • Does npm run test or npm run test-DIALECT pass with this change (including linting)?
  • Does the description below contain a link to an existing issue (Closes #[issue]) or a description of the issue you are solving?
  • Have you added new tests to prevent regressions?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Did you follow the commit message conventions explained in CONTRIBUTING.md?

Description of change

Model was previously altering the original scopes when merging them for queries.
This happened because Model._mergeFunction was in some cases returning references to attributes of the original scope objects instead of cloning them (see line 811).

Also added a new test to prevent this problem from happening again.

Closes #10696

Additional notes

Tried to clone the object with both Utils.cloneDeep and Lodash's _.cloneDeep, but both result in JavaScript heap out of memory errors when running the tests. My guess is there are some cases where the scopes can include references to complex objects that should not be cloned (maybe references to Sequelize itself?). Because of this, it's cloned with a call to Lodash's _.cloneDeepWith specifying a function that results in cloning only plain objects. If this function for deep cloning only plain objects can be useful in other places, maybe it could be moved to Utils.

Model was previously altering the original scopes when merging them for queries.
This happened because _mergeFunction was in some cases returning references to
attributes of the original scope objects instead of cloning them.

Also added a new test to prevent this problem from happening again.

Closes sequelize#10696
@papb
Copy link
Member

papb commented Apr 8, 2019

@jorgelf - Nice. Thanks also for digging into the clone stuff. Looks like we have three different types of cloning around Sequelize now, though. I've seen direct usage of _.clone and also Utils.cloneDeep, and now we have this "third" way of cloning which you introduced. I'm not saying that you shouldn't have done that, on the contrary, you've investigated and chosen what seemed to be the best solution. Nice work. What I'd like to ask, since you've already digged into this, is: could you investigate the other usages of _.clone and Utils.cloneDeep around Sequelize to see if you could refactor those to have a standard way to clone things, making the code easier to understand? This would be work for another PR, of course, this PR is great as it is 😬

@jorgelf
Copy link
Contributor Author

jorgelf commented Apr 8, 2019

You are probably right, too many flavours of cloning. Maybe this new cloning method can be merged with the already existing Utils.cloneDeep by adding a new optional parameter?

function cloneDeep(obj, onlyPlain) {
  obj = obj || {};
  return _.cloneDeepWith(obj, elem => {
    // Do not try to customize cloning of arrays or POJOs
    if (Array.isArray(elem) || _.isPlainObject(elem)) {
      return undefined;
    }

    // If we specified to clone only plain objects & arrays, we ignore everyhing else
    // In any case, don't clone stuff that's an object, but not a plain one - fx example sequelize models and instances
    if (onlyPlain || typeof elem === 'object') {
      return elem;
    }

    // Preserve special data-types like `fn` across clones. _.get() is used for checking up the prototype chain
    if (elem && typeof elem.clone === 'function') {
      return elem.clone();
    }
  });
}

Tests are still passing after this change, so nothing seems to break with it. If it looks fine to you, I can just commit those changes to this branch, or make a new PR if you prefer it that way.

I probably wouldn't do any changes regarding usage of Lodash's _.clone. It's a simple, straightforward way of making shallow copies of any element. Merging it with the deep cloning method may lead to confusion. Maybe it could have a wrapper inside Utils, but I'm not sure it's actually worth it.

And about the Travis build fail, those tests don't give me any trouble in my machine, so not sure why they failed.

@papb
Copy link
Member

papb commented May 19, 2019

@jorgelf - Sorry to take long. I fully agree with you. Merging with the already existing cloneDeep is excellent, and leaving _.clone untouched is good too.

And don't worry about the travis build fail, it happens.

Right now it's only used in one place, but it might
prove useful in the future, so we now provide a
simple way for deep cloning only plain objects.

See sequelize#10722 & sequelize#10696
@codecov
Copy link

codecov bot commented May 21, 2019

Codecov Report

Merging #10722 into master will increase coverage by 21.81%.
The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master   #10722       +/-   ##
===========================================
+ Coverage   74.54%   96.35%   +21.81%     
===========================================
  Files          85       94        +9     
  Lines        8186     9010      +824     
===========================================
+ Hits         6102     8682     +2580     
+ Misses       2084      328     -1756
Impacted Files Coverage Δ
lib/model.js 96.69% <ø> (+0.86%) ⬆️
lib/utils.js 98.35% <100%> (+4.52%) ⬆️
lib/dialects/mariadb/connection-manager.js 100% <0%> (ø)
lib/dialects/mariadb/index.js 100% <0%> (ø)
lib/dialects/sqlite/index.js 100% <0%> (ø)
lib/dialects/mariadb/query.js 87.69% <0%> (ø)
lib/dialects/postgres/connection-manager.js 95.77% <0%> (ø)
lib/dialects/postgres/index.js 100% <0%> (ø)
lib/dialects/sqlite/query.js 98.64% <0%> (ø)
lib/dialects/sqlite/connection-manager.js 100% <0%> (ø)
... and 31 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30c5ca5...9b6e92f. Read the comment docs.

@jorgelf
Copy link
Contributor Author

jorgelf commented May 21, 2019

Great, thanks for the comment, @papb.

A new commit has been added doing that small refactoring. If something doesn't look good to you, just tell me.

@sushantdhiman sushantdhiman merged commit f2b0aec into sequelize:master Jul 6, 2019
@sushantdhiman
Copy link
Contributor

🎉 This PR is included in version 5.9.4 🎉

The release is available on:

Your semantic-release bot 📦🚀

schmod pushed a commit to schmod/sequelize that referenced this pull request Jul 10, 2019
…ndle-deadlock

* 'master' of github.com:sequelize/sequelize: (22 commits)
  docs(migrations): use timestamps with seed (sequelize#11160)
  test: remove redundant test (sequelize#11156)
  fix(types): add literal to possible where options (sequelize#10990)
  fix(model): don't alter original scopes when combining them (sequelize#10722)
  fix(types): relax order typing (sequelize#10802)
  fix(types): add string to Includeable (sequelize#11003)
  docs(models-definition): correct spelling mistakes (sequelize#11147)
  fix(types): silent option for update (sequelize#11115)
  fix: update sequelize-pool (sequelize#11134)
  feat(hooks): beforeDisconnect / afterDisconnect (sequelize#11117)
  refactor: remove unused _templateSettings
  refactor(query-generation): remove lodash string templates (sequelize#11122)
  docs: improve datatype docs
  docs: explain defaults/where behavior for find/create (sequelize#11069)
  build: remove test*.js from .gitignore (sequelize#11108)
  docs(data-types): extending types
  fix(sequelize.close): update sequelize-pool (sequelize#11101)
  build: update dependencies (sequelize#11099)
  docs(migrations): foreign key example (sequelize#11097)
  fix(mariadb): properly escape json path key (sequelize#11089)
  ...
@sushantdhiman
Copy link
Contributor

🎉 This PR is included in version 7.0.0-next.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

defaultScope mutates after applying .scope(['defaultScope', 'otherScope'])
3 participants