Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add cache persistence and invalidation tests #22013

Merged
merged 4 commits into from Mar 23, 2020

Conversation

@pieh
Copy link
Contributor

pieh commented Mar 6, 2020

Description

This adds some NON-unit tests for cache persitence/invalidation behaviours.

Primarily used for checking state of nodes store. It's node backed agnostic, it only relies on getNodes function passed to all plugins, so it works both with redux and loki stores and with any other future stores that we come up with as long as getNodes is sync (if will turn async/deprecated it will be just matter of updating 2 lines).

Because we blow away entire cache on any change now, there is not much value in this (but it already did catch in loki persistance ( #22009) - so there's some value.

There are already existing assertions for selective cache invalidation scenarios - but they are unreachable in current master - they are hidden behind process.env.GATSBY_EXPERIMENTAL_SELECTIVE_CACHE_INVALIDATION check (env var name subject to change)

High level overview of how those tests work.

There is test site using multiple local plugins with some fixtures with programmatic way of adding/removing plugins from gatsby-config, as well as programmatic way of changing plugin files (which should trigger invalidation). Currently there are 2 configurations ("1" and "2" :) ), there are multiple things changing between configuration 1 and 2.

  • First pass: 2 builds with same configuration: gatsby clean -> gatsby build (with configuration 1) -> gatsby build (with configuration 1). In this case nothing of value should be invalidated and tests assert that.
  • Second pass: 2 builds with different configuration: gatsby clean -> gatsby build (with configuration 1)->gatsby build (with configuration 2). In this case there is multiple thing that should be invalidated:
    • persisted key-value store (cache)
    • nodes store (in master we invalidate all of it), but assertions here check for difference between end result from 1st run and 2nd run - that need to be the same regardless if blow away entire cache or we selectively invalidate cache (in particular this have lot of assertions hidden for now in preparation for selective cache invalidation feature)
    • query results using predefined fixtures (making sure final result is the what is expected - this is nice addition to unit tests, because this one doesn't rely on internal implementation (other than location of page-data files) and we use public gatsby APIs to trigger query running (by using page queries)
    • schema changes (for example - adding transformer to configuration should add type of transformer nodes to schema)

See https://github.com/gatsbyjs/gatsby/tree/cache-integraion-tests/integration-tests/cache-resilience/plugins for somewhat structure breakdown of what scenarios exist. Each scenario can have multiple local plugins that act as fixtures (currently some scenarios have 1 plugins and some have 2).

Related Issues

Split from #8379

[ch341]

Co-authored-by: Sidhartha Chatterjee me@sidharthachatterjee.com

Co-authored-by: Sidhartha Chatterjee <me@sidharthachatterjee.com>
@pieh pieh requested a review from gatsbyjs/core as a code owner Mar 6, 2020
Copy link
Member

sidharthachatterjee left a comment

This is an insane amount of work and thank you for finishing this! ❤️

@sidharthachatterjee sidharthachatterjee changed the title test: add cache persistance and invalidation tests test: add cache persistence and invalidation tests Mar 23, 2020
@sidharthachatterjee sidharthachatterjee merged commit 6d62a78 into master Mar 23, 2020
22 of 24 checks passed
22 of 24 checks passed
build-test Workflow: build-test
Details
ci/circleci: starters_validate Your tests failed on CircleCI
Details
TypoCheck Found a typo
Details
Cloud Tests
Details
Danger All good
Details
Gatsby Build Service Gatsby Build Service
Details
Peril All green. Good on 'ya.
Details
ci/circleci: bootstrap Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_development_runtime Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_gatsby-image Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_path-prefix Your tests passed on CircleCI!
Details
ci/circleci: e2e_tests_production_runtime Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_cache_resilience Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_gatsby_pipeline Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_long_term_caching Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_structured_logging Your tests passed on CircleCI!
Details
ci/circleci: lint Your tests passed on CircleCI!
Details
ci/circleci: themes_e2e_tests_development_runtime Your tests passed on CircleCI!
Details
ci/circleci: themes_e2e_tests_production_runtime Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_node10 Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_node12 Your tests passed on CircleCI!
Details
ci/circleci: unit_tests_www Your tests passed on CircleCI!
Details
ci/circleci: windows_unit_tests Your tests passed on CircleCI!
Details
cypress: default-group 139 tests passed in 02:59
Details
@sidharthachatterjee sidharthachatterjee deleted the cache-integraion-tests branch Mar 23, 2020
@sidharthachatterjee

This comment has been minimized.

Copy link
Member

sidharthachatterjee commented Mar 23, 2020

Some notes from our review

  • Transformer change test case does not handle a potential change in the source node as a result of a type name change in the child
  • Fields should be set to null if the added field key was removed or fields should be an empty object in general for consistency
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.