Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gatsby): Materialization #16091

Merged
merged 50 commits into from Aug 30, 2019

Conversation

@freiksenet
Copy link
Contributor

commented Jul 25, 2019

Add a node backend independent materialization layer that triggers when nodes are searched on fields only available through resolvers.

In addition, this fixes various incompatibilities that loki had due to never being tested through materialization.

In addition, node tracking is now happening on node-model level too.

In addition, this adds extensions to decide which fields are sortable and searchable and which needs resolve, so we can materialize.

Can be tested via gatsby@materialization

@freiksenet freiksenet changed the title Materialization WIP: Materialization Jul 25, 2019
@freiksenet

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

Published as gatsby@2.14.0-rc.2 / gatsby@materialization

true,
true
)
if (!this._trackedRootNodes.has(node.id)) {

This comment has been minimized.

Copy link
@stefanprobst

stefanprobst Aug 27, 2019

Contributor

does this work with hot-reloading, i.e. when the node.id stays the same but some nested field changes and needs to be re-tracked?

This comment has been minimized.

Copy link
@freiksenet

freiksenet Aug 27, 2019

Author Contributor

Changing nodes resets node model.

@stefanprobst

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2019

getQueryFields, mergeObjects, merge, pathToObject, dropQueryOperators are now in node-model and can be removed from db/common/query i think

@freiksenet

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

Published as gatsby@2.14.0-rc.3 / gatsby@materialization

@freiksenet

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

getQueryFields, mergeObjects, merge, pathToObject, dropQueryOperators are now in node-model and can be removed from db/common/query i think

Cheers, fixed.

Copy link
Member

left a comment

I still have to go through packages/gatsby/src/schema/node-model.js but I already have some small nits. Nothing big, feel free to ignore

packages/gatsby/src/db/nodes.js Outdated Show resolved Hide resolved
packages/gatsby/src/db/sanitize-node.js Outdated Show resolved Hide resolved
packages/gatsby/src/db/sanitize-node.js Outdated Show resolved Hide resolved
packages/gatsby/src/db/sanitize-node.js Outdated Show resolved Hide resolved
packages/gatsby/src/schema/node-model.js Show resolved Hide resolved
packages/gatsby/src/schema/node-model.js Show resolved Hide resolved
packages/gatsby/src/schema/node-model.js Outdated Show resolved Hide resolved
packages/gatsby/src/schema/node-model.js Outdated Show resolved Hide resolved
packages/gatsby/src/schema/node-model.js Outdated Show resolved Hide resolved
freiksenet and others added 4 commits Aug 28, 2019
Co-Authored-By: Ward Peeters <ward@coding-tech.com>
@freiksenet

This comment has been minimized.

Copy link
Contributor Author

commented Aug 28, 2019

Published as gatsby@2.14.0-rc.4 / gatsby@materialization

Copy link
Member

left a comment

Small nit but I believe this is ready to go! <3

packages/gatsby/src/db/nodes.js Outdated Show resolved Hide resolved
packages/gatsby/src/db/nodes.js Outdated Show resolved Hide resolved
Co-Authored-By: Ward Peeters <ward@coding-tech.com>
Copy link
Member

left a comment

Awesome work @freiksenet! lets 🚢 it

@sidharthachatterjee sidharthachatterjee self-requested a review Aug 30, 2019
Copy link
Member

left a comment

Let’s ship this! 💃🏼

@freiksenet freiksenet merged commit ac10d3d into master Aug 30, 2019
20 checks passed
20 checks passed
Danger All good
Details
Peril All green. Good on 'ya.
Details
ci/circleci: bootstrap Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_development_runtime Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_gatsby-image Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_path-prefix Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_production_runtime Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_gatsby_pipeline Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_long_term_caching Your tests passed on CircleCI!
Details
ci/circleci: lint Your tests passed on CircleCI!
Details
ci/circleci: starters_validate Your tests passed on CircleCI!
Details
ci/circleci: themes_e2e_tests_development_runtime Your tests passed on CircleCI!
Details
ci/circleci: themes_e2e_tests_production_runtime Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_node10 Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_node12 Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_node8 Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_www Your tests passed on CircleCI!
Details
ci/circleci: windows_unit_tests Your tests passed on CircleCI!
Details
cypress: default-group 67 tests passed in 00:28
Details
unit_tests_windows Build #20190830.20 succeeded
Details
OSS Roadmap automation moved this from In progress to Done Aug 30, 2019
@delete-merged-branch delete-merged-branch bot deleted the materialization branch Aug 30, 2019
waltercruz added a commit to waltercruz/gatsby that referenced this pull request Sep 8, 2019
Add a node backend independent materialization layer that triggers when nodes are searched on fields only available through resolvers.

In addition, this fixes various incompatibilities that loki had due to never being tested through materialization.

In addition, node tracking is now happening on node-model level too.

In addition, this adds extensions to decide which fields are sortable and searchable and which needs resolve, so we can materialize.
@jonsherrard

This comment has been minimized.

Copy link

commented Sep 12, 2019

Hello all! I am way way out of my depth here - hot reloading is like complete magic to me, but I'm 50% sure this PR broke hot reloading in mdx-deck jxnblk/mdx-deck#474. I've explained a bit more in that issue... I'd like to help more but I think it'll take me literally weeks to understand what Materialization is 😅

If anyone can point me in any directions I'm happy to keep on with my knowledge-less brute-force debugging! 😄 @freiksenet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
OSS Roadmap
  
Done
8 participants
You can’t perform that action at this time.