Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache SLAS callback using request processor #884

Merged
merged 9 commits into from
Jan 12, 2023
Merged

Conversation

kevinxh
Copy link
Collaborator

@kevinxh kevinxh commented Jan 6, 2023

Background

The retail react app uses SLAS for shopper authentication and the login flow has an endpoint /authorize returns 303 on successful authorization attempts and redirect user to a pre-registered redirect_uri that looks like https://mydomain/callback?usid=1&code=2. We do NOT need to follow the redirect because we have the information from the response headers.

However, for security reasons, browsers forcefully follow the redirect and there is no option to stop that. This PR attempts to aggressively cache the /callback endpoint on the CDN to make the network round trip faster.

Screenshot 2023-01-11 at 11 04 45 PM

Solution

CloudFront (MRT's CDN) caches pages based on the URL and headers, since every time /callback is hit with unique query parameters usid and code, by default the endpoint is not cached. We use the request processor (CloudFront's edge computing solution - Lambda@Edge) to remove the query parameters before the request hits the CDN. This will make the CDN cache the /callback endpoint as if there were no query parameters.

Types of Changes

Performance enhancement.

Changes

  • Update request processor to remove usid and code parameters on the /callback endpoint.

How to Test-Drive This PR

Testing method: query the /callback?usid=1&code=2 endpoint multiple times and vary the value of usid and code. Without the change, you will never get CDN cached response.
With the change, you always get cached responses.

Before

Environment: https://pwa-kit.mobify-storefront.com

This environment does NOT have the change from this PR.

# First try
curl -s -D - -o /dev/null https://pwa-kit.mobify-storefront.com/callback?usid=1&code=2

# Try again using different parameters
curl -s -D - -o /dev/null https://pwa-kit.mobify-storefront.com/callback?usid=3&code=4

Inspect the headers, you should always see x-cache: Miss from cloudfront.

HTTP/2 200
content-type: application/json
content-length: 0
server: CloudFront
date: Thu, 12 Jan 2023 06:46:47 GMT
referrer-policy: no-referrer
x-dns-prefetch-control: off
x-permitted-cross-domain-policies: none
x-xss-protection: 0
expect-ct: max-age=0
strict-transport-security: max-age=15552000; includeSubDomains
x-frame-options: SAMEORIGIN
x-amzn-remapped-content-length: 0
content-security-policy: img-src 'self' *.commercecloud.salesforce.com data:;script-src 'self' 'unsafe-eval' storage.googleapis.com;upgrade-insecure-requests;default-src 'self';base-uri 'self';block-all-mixed-content;font-src 'self' https: data:;frame-ancestors 'self';object-src 'none';script-src-attr 'none';style-src 'self' https: 'unsafe-inline'
x-amzn-remapped-connection: close
x-download-options: noopen
x-amz-apigw-id: enfxoHcRIAMF1Sw=
x-content-type-options: nosniff
x-amzn-trace-id: Root=1-63bfacd7-7ec58b2c5c5f2aed4399aef8;Sampled=0
x-amzn-remapped-date: Thu, 12 Jan 2023 06:46:47 GMT
x-amzn-requestid: fcd24de7-2657-4992-a6e7-7f4c452eda6f
x-cache: Miss from cloudfront            <================================= this means the endpoint is not cached
via: 1.1 9b34a4c8b07eed6c2ff20b5adbbfa7c4.cloudfront.net (CloudFront)
x-amz-cf-pop: YVR50-C1
x-amz-cf-id: FPJOo_3GFoG5nLDXbOhJMICos8cAonoKlGG9zctyo-4OjiqqfA8XYw==

After

Environment: https://scaffold-pwa-test-env.mobify-storefront.com

This environment does have the change from this PR.

# First try
curl -s -D - -o /dev/null https://scaffold-pwa-test-env.mobify-storefront.com/callback?usid=1&code=2

# Try again using different parameters (you might need to wait for a few seconds b/c there is a delay for CDN to populate cache)
curl -s -D - -o /dev/null https://scaffold-pwa-test-env.mobify-storefront.com/callback?usid=3&code=4

Inspect the headers, on the second try, you should see x-cache: Hit from cloudfront. This proves that the CDN caches the /callback regardless of the usid and code query parameters.

Checklists

  • Changes are covered by test cases
  • CHANGELOG.md updated with a short description of changes (not required for documentation updates)

@kevinxh kevinxh marked this pull request as ready for review January 12, 2023 06:57
@kevinxh kevinxh requested a review from a team as a code owner January 12, 2023 06:57
@@ -9,9 +9,6 @@
// 'request-processor.js' and update the processRequest function so that
// it processes requests in whatever way your project requires.

// Uncomment the following line for the example code to work.
// import {QueryParameters} from 'pwa-kit-react-sdk/utils/ssr-request-processing'
Copy link
Collaborator Author

@kevinxh kevinxh Jan 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This QueryParameters class is a custom version of the javascript native API URLSearchParams. I don't see a reason why we have to use QueryParameters?

The request processor should be super lean and fast since it is called on every request at the edge, we should not advocate for importing libraries from this file, especially when we have native alternatives.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the code in the SDK, the reason that we are using this custom QueryParameters class is because the order of queries is important (?a=1&b=2 might be a hit, but ?b=1&a=2 might not). The QueryParameters is here to maintain order when you are manipulating the search string.

Although not using that class doesn't effect this particular situation, the removal of the example boilerplate code might mean that partners that do decide to make changes to the request processor will do so incorrectly not knowing about this restriction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URLSearchParams also preserves the order; the key differences seem to be how null is handled and how spaces are encoded.

@kevinxh kevinxh changed the title Cache SLAS callback in request processor Cache SLAS callback using request processor Jan 12, 2023
@kevinxh kevinxh merged commit 3a2f185 into develop Jan 12, 2023
@kevinxh kevinxh deleted the feature/callback-rp branch January 12, 2023 20:06
alexvuong added a commit that referenced this pull request Feb 13, 2023
* New year, new look! (#876)

* New year, new look!

* Use JS to compute current year.

* Small bump to max deps allowed.

Needed to so that non-code changes will pass CI.
Hitting the limit can be addressed later.

* Update from 2.5.0 (#881)

* Starting release process for 2.5.0

* allow commerce react sdk to release

* bump version to 2.5.0 (#879)

* bump version to 2.5.0

* bump max packages

* Begin development on 2.6.0

* Set commerce-sdk-react package to private (#882)

* test-commerce: comment out not-implemented customer hooks (#877)

* Change padding (#899)

* [Snyk] Security upgrade eslint-import-resolver-webpack from 0.10.0 to 0.13.1 (#887)

* fix: packages/internal-lib-build/package.json & packages/internal-lib-build/package-lock.json to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-DEBUG-3227433

* upgrade eslint-webpack-plugin-import, regen lock file

Co-authored-by: Alex Vuong <alex.vuong@salesforce.com>
Co-authored-by: Will Harney <62956339+wjhsf@users.noreply.github.com>

* Cache SLAS callback using request processor (#884)

* cache callback in request processor

* fix import path

* cache callback for a year

* use native URLSearchParams

* revert use native URLSearchParams

* Rotate fingerprint ssh for deploy commerce sdk doc (#905)

* Feature/megamenu fixes (#875)

* [W-11996527] WIP - commit megamenu spike results / findings to feature branch

* resolve search error

* move default variables to constants file

* cleanup code

* add aria-live to menu categories

* lint

* implement aria live and busy

* fix failing tests

* update mock categories data, implement aria busy for listmenu

* update mock data and fix list-menu test

* lint

* resolve unmounted component error

* temporary testing server errors fix

* set spinner opacity to 0 and increase max package value

* lint

* remove mock data from app code, clean up

* allow padding of categories with loaded: true to bypass initial useEffect API call

* fix last failing test

* Update list-menu/index.jsx

Co-authored-by: Will Harney <62956339+wjhsf@users.noreply.github.com>

* fix aria-busy implementation and cleanup

* add more test cases - when there is no root categories

* remove mit license comment

* add depndency array to useeffect

* globally mock getcategory api call

* WIP

* fix tests WIP

* remove all instances of setupmockserver in test files

* globally mock istokenvalid AND many other cleanup items

* refactor tree walking code

* add locale checks and time to localstorage

* WIP removing tree walking logic

* remove tree walking code, update setroot and extract stale time

* add catch to promise all

* lint

* namespace constants, remove unnecessary try catch...

* final cleanup

* fix lint

Co-authored-by: Brian Feister <bfeister@salesforce.com>
Co-authored-by: Will Harney <62956339+wjhsf@users.noreply.github.com>
Co-authored-by: Kevin He <kevin.he@salesforce.com>

* Mega menu fixes (#910)

* Remove unnecessary catch clause.

* Update copyright.

* Don't return error when category is expected.

* Fix bug preventing cache invalidation.

* Set fetchTime when data is fetched, rather than every page load.

* Update comment.

* Convert promise chain to async/await for readability.

* Guard against missing category items.

* Move comment inside code block.

* Change useEffect back to promise chain, not async/await.

useEffect expects the callback to return nothing or a function;
returning a promise could break things.

* GitHub Actions (#854)

* Add test workflow

* Use actions/checkout v3

* Don't use CI image

* Add step install npm dependencies

* Add setup action

* Set DEVELOP and RELEASE env variables

* Add required shell property

* Update action.yml

* Update action.yml

* Adding test matrix

* cleanup

* Split setup windows and ubuntu

* Update test.yml

* Update test.yml

* Add Lighthouse and smoketest scripts

* Not using actions/cache for now

* Add generated matrix

* Add missing runs-on prop to generated matrix

* Add Setup Node step to generated matrix

* Add Setup machine to generated matrix

* Add cron schedule

* Add timeout-minutes 5

* Move timeout to generate project step

* Add Run tests on generated projects

* Use dynamic generated-project folder

* Run tests on test-project and retail-react-app-demo

* Add Push Bundle step

* Skip flaky test

* Disable fail-fast strategy

* Use env variables

* Re-arrange env

* Add step before push bundle

* cleanup

* cleanup

* Use temp test-env-3

* testing slack notifs

* testing

* add publish to npm step

* fix indent

* python-dev does not exist anymore

* use python2

* increase max packages

* test slack notifs

* add snyk cli and datadog step

* update mrt user credentials

* testing slack with pwa kit channel

* syntax

* fix conditionals

* test push bundle

* add push bundle step for generated

* syntax

* fix syntax error

* update slack payload

* run steps in container

* testing

* refactor

* syntax

* sudo container error

* testing

* update

* add pip

* use different docker

* no container

* container

* testing

* add user to container

* fix

* syntax add shell

* syntax errors

* remove container, use act

* syntax errors

* add snyk audit and other syntax stuff

* extract steps to own actions

* add inputs for actions

* add shell for steps in actions with run

* project cannot be generated in action file

* updated snyk token, uncommenting code

* Fix typo.

* Add missing appkey property.

* Use snake_case names for legibility.

* Restore missing clean check

* Fix skipped conversion to snake_case.

* Trim trailing whitespace.

* Extract conditionals to vars and clean up vars.

* Change env IS_TESTABLE_TEMPLATE to more clear IS_TEMPLATE_FROM_RETAIL_REACT_APP

* Fix YAML breaking conditional.

* Try explicitly checking value.

* Try explicitly checking true/false string values.

* Try string comparisons.

* Fix bad YAML.

* Replace " with '

* Get ready for the prime time!

* Fail fast!

* Update TODOs.

* Clean up npm version management.

* Add TODO to merge workflows.

* Update step names.

* End files with newline.

* Run on pull_request to support forked repos.

* Only run on push for develop/release.

We can assume all other branches will eventually have a PR.

* Only push to MRT when actually desired.

* Get that JavaScript nonsense outta here!

* Check DEVELOP in step conditional, not in action execution.

* Add some TODOs.

* Too many newlines!

Co-authored-by: yunakim714 <yunakim@salesforce.com>
Co-authored-by: yunakim714 <84923642+yunakim714@users.noreply.github.com>
Co-authored-by: Will Harney <wharney@salesforce.com>
Co-authored-by: Will Harney <62956339+wjhsf@users.noreply.github.com>

* GitHub Actions fixes (#915)

* Update is not fork check

* Add single quote to cron expression

* Add cron docs

* Remove Circle CI config, fix IS_NOT_FORK (#921)

* Fix develop branch name (#923)

* Fix wrong proptypes (#924)

* fix wrong proptypes

* Update packages/template-retail-react-app/app/components/recommended-products/index.jsx

Co-authored-by: Vincent Marta <vmarta@salesforce.com>

Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>
Co-authored-by: Vincent Marta <vmarta@salesforce.com>

* Update createOrder to send SLAS USID (#920)

* Update createOrder to send SLAS USID

* Modify some tests to include setting the header

Co-authored-by: echessman <37908171+echessman@users.noreply.github.com>

* Upgrade prettier to v2 (#926)

* Upgrade to prettier v2 for modern TypeScript support.

* Change trailing comma to prettier v1 default.

Minimizes changes during v2 upgrade.

* Apply prettier v2 formatting changes.

Yaaay touching all of the files!

* Set end of line to LF.

* Remove unnecessary map statement (#929)

* fix: packages/pwa-kit-runtime/package.json & packages/pwa-kit-runtime/package-lock.json to reduce vulnerabilities (#935)

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-UAPARSERJS-3244450

Co-authored-by: snyk-bot <snyk-bot@snyk.io>

* Update `develop` with v2.6.0 (#939)

* Starting release process for 2.6.0

* 🚢 Release v2.6.0 (#937)

* Update changelog files

* Set `commerce-sdk-react-preview` as public

* Version 2.6.0

* Begin development on 2.7.0

* Clean changelog files

* Allow support for multiple sites concurrently w/ `commerce-sdk-react` (#911)

* Namespace storage keys using the current siteId.

Co-authored-by: Will Harney <62956339+wjhsf@users.noreply.github.com>

* @W-11920099 Re-write 'npm push' in Typescript, warn if Node deprecated (#763)

* Rewrite ancient scripts in Typescript.
* Show warnings on push when using a deprecated Node version.
* Automatically select a `.mobify` credentials file based on `--cloud-origin`.

* Fix broken CI – confusing path to package.json in the 'dist' directory! (#946)

* Reliably look up project and pwa-kit-dev package.json in scripts

* fix: handle special characters in `boldString` util (#942)

+ Add a new util for escaping special regex characters
+ Apply new util to `boldString`
+ Add and update util tests

Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>

* Replace isomorphic jest mocks with msw handlers (#944)

* replace most manual mocks

* more replacements

* more replacements in addresses test

* lint

* resolve flaky cart test

* remove some jest mocks

* replace all isomorphic mocks

* cleanup

* fix auth modal mocks

* remove timers from auth hooks test, fix password test

* remove timer from create account test

* add timeout

* cleanup

---------

Co-authored-by: Brian Feister <bfeister@salesforce.com>

* Remove the PersistentCache functionality (#949)

* Initial pass at PersistentCache

* Drop test coverage

* Change test

* Update customer baskets cache when there is a basket mutation (#945)

* update customer baskets cache on basket mutations

* Fix layout shift for mega menu (#952)

* remove custom styling

* remove theme in theme file

* spinners in drawer menu should be visible

* Serialize category data once only (#953)

* serialize data only once

* remove console log

---------

Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>

* chore: update pwa-kit-dev eslint config (#950)

+ Bump `eslint-plugin-react` to latest version
+ Auto-detect React version

Co-authored-by: Adam Raya <adamraya@users.noreply.github.com>

* resolving testing/merge errors

* Add Shopper Experience hooks (#958)

* Initial commit

* Update license

* Update test project to use bjnl_dev

* Fix usePage hook and Refactor test page

* Add usePages hook

* Add usePages pdp and plp test cases

* Clean up

* Update Changelog & Restore `zzrf-001` config

---------

Co-authored-by: Ben Chypak <bchypak@salesforce.com>

* add changelog

* Apply changes from commerce react sdk in feature branch (#964)

* apply change from commerce react sdk in feature branch

* Remove wrong changlog

* keep the same as develop

* linting

* fix formatMessage

* bump sizes temporarily

* fix product list tests

* fix header tests

* fix add to cart modal

* skip tests and low test coverate temporarily

* linting

---------

Co-authored-by: Will Harney <62956339+wjhsf@users.noreply.github.com>
Co-authored-by: Alex Vuong <52219283+alexvuong@users.noreply.github.com>
Co-authored-by: vcua-mobify <47404250+vcua-mobify@users.noreply.github.com>
Co-authored-by: Vincent Marta <vmarta@salesforce.com>
Co-authored-by: Pavel <65617653+GoodNightBuddy@users.noreply.github.com>
Co-authored-by: Snyk bot <snyk-bot@snyk.io>
Co-authored-by: Alex Vuong <alex.vuong@salesforce.com>
Co-authored-by: yunakim714 <84923642+yunakim714@users.noreply.github.com>
Co-authored-by: Brian Feister <bfeister@salesforce.com>
Co-authored-by: Adam Raya <adamraya@users.noreply.github.com>
Co-authored-by: yunakim714 <yunakim@salesforce.com>
Co-authored-by: Will Harney <wharney@salesforce.com>
Co-authored-by: Brian Feister <47546998+bfeister@users.noreply.github.com>
Co-authored-by: echessman <37908171+echessman@users.noreply.github.com>
Co-authored-by: CC ProdSec <65211003+cc-prodsec@users.noreply.github.com>
Co-authored-by: Ben Chypak <bchypak@mobify.com>
Co-authored-by: Oliver Brook <o.brook@salesforce.com>
Co-authored-by: Brad Adams <hi@breadadams.com>
Co-authored-by: Kieran Haberstock <80915722+kieran-sf@users.noreply.github.com>
Co-authored-by: Ben Chypak <bchypak@salesforce.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants