Paginated sitemap.xml #1125
Closed
Paginated sitemap.xml #1125
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Minor codegen refactor * Add --codegen-unstable flag to build command * Maybe fix not found queries issue from monorepo * Changesets
* Add login and logout commands * Refactor graphql client and requests, add unit tests * Update shopify-config methods and tests * Update renderErrors to use cli Command * Rework env-pull to call login and remove silent param * Rework link command to call login and remove --shop flag * Update list command to call login and remove --shop flag * Update env-list command to call login and remove --shop flag * Oclif manifest * Cleanup * Rework log replacer * Mute cli-kit auth logs * Fix mock * Merge main branch * Fix tests * Minor changes * Update package-lock * Skip writing config when no directory is passed to login * Improve unit tests * Changesets
Use shop.primaryDomain to mark internal links --------- Co-authored-by: Bret Little <bret.little@shopify.com>
* Add login and logout commands * Refactor graphql client and requests, add unit tests * Update shopify-config methods and tests * Update renderErrors to use cli Command * Rework env-pull to call login and remove silent param * Rework link command to call login and remove --shop flag * Update list command to call login and remove --shop flag * Update env-list command to call login and remove --shop flag * Oclif manifest * Cleanup * Rework log replacer * Mute cli-kit auth logs * Fix mock * Update cli-kit version * Add GraphQL schema generator for local development * Use business platform login to render a list of shops * Merge main branch * Fix tests * Minor changes * Update package-lock * Apply suggestions from code review Co-authored-by: Gray Gilmore <graygilmore@gmail.com> * Skip writing config when no directory is passed to login * Improve unit tests * Changesets * Refactor to getUserAccount and add tests * Store shopName and email in local config * Show shopName and email after login * Force prompt during login when not passing --shop flag * Fix tests * Update cli-kit version and its deps * Cleanup --------- Co-authored-by: Gray Gilmore <graygilmore@gmail.com>
* Add login and logout commands * Refactor graphql client and requests, add unit tests * Update shopify-config methods and tests * Update renderErrors to use cli Command * Rework env-pull to call login and remove silent param * Rework link command to call login and remove --shop flag * Update list command to call login and remove --shop flag * Update env-list command to call login and remove --shop flag * Oclif manifest * Cleanup * Rework log replacer * Mute cli-kit auth logs * Fix mock * Update cli-kit version * Add GraphQL schema generator for local development * Use business platform login to render a list of shops * Improve login logs * Merge main branch * Fix tests * Minor changes * Update package-lock * Apply suggestions from code review Co-authored-by: Gray Gilmore <graygilmore@gmail.com> * Change link color * Skip writing config when no directory is passed to login * Improve unit tests * Changesets * Refactor to getUserAccount and add tests * Store shopName and email in local config * Show shopName and email after login * Force prompt during login when not passing --shop flag * Fix tests * Only show loader after pressing a key * Show short version of link in terminal * Update cli-kit version and its deps * Cleanup * Update package-lock * Remove duplicated code chunk --------- Co-authored-by: Gray Gilmore <graygilmore@gmail.com>
* Refactor: extract format-code utilities * Add setup-assets * Add file utils and tests * Setup Tailwind * Fix paths and add renderTask * Install Tailwind deps * Update package-lock * Fix asset replacement * Refactor: extract common build paths * Copy hello-world template to dist folder * Make hello-world the default starter * Fix timezone in CLI tests * Refactor: extract admin requests * Cleanup * Setup local starter * Refactor: split workflows * Prompt for package maanger when unknown * Change shop prompt * Group tasks * Remove .env variables when linking to shop * Make setupTailwind more modular * Add setup CSS step * Merge branch '2023-04' into fd-new-onboarding * Move CSS setup logic around and fix test * Ensure promises are chained * Prompt for creating h2 alias * Ensure promises are chained * Copilot comments * Refactor: extract ast replacers * Improve types and move files around * Support PostCSS setup * Prettify css strategy names * Fix type of flagsToCamelObject * Add install-deps flag to css-unstable * Make root links replacer more flexible * Add support for CSS modules setup * Prettify output * Extract css bundling injection * Add support for vanilla-extract setup * Update oclif manifest * Prompt for styling library when no arg is passed * Adjust prompts * Add prompts for i18n and routes * Add i18n setups * Export I18nLocale type * Fix TS version of domain and subdomains i18n setups * Add I18nLocale type to remix.env.d.ts * Adjust success message * Force choosing a CSS strategy * Adjust success message * Show helpful commands at the end * Adjust success message spacing * Fix AST grep for JS * Fix init test * Refactor generate route command * Hoist work to improve perf in route generation; add more tests * Improve output logs * Fix generate route * Allow passing locale prefix * Figure out locale prefix automatically * Make the locale variable optional in path * Scaffold all routes * Simplify prompts format * Add option to use mock.shop * Init git and create initial commit * Error handling * Show errors properly in at the end * Throw errors from replacers * Fix tests * Merge branch '2023-04' into fd-new-onboarding * Feedback update * Use AbortSignal in prompts * Minor adjustments * Align with design updates * Disable i18n and routes setup for now * Oclif manifest * Remove all variables from .env when linking the store * Update package-lock * Integrate init with login flow * Integrate create-storefront with new onboarding (#1058) * Integrate create storefront in init * Minor refactoring * Add tests for create-storefront * Infer path from storefront title * Defer create-storefront integration * Fix changesets * Make shopName and email required fields in local config * setUserAccount in init * New onboarding - extra setup (#1050) * Improve implementation of i18n templates * Enable extra setup in init * Add --styling flag * Add --mock-shop flag * Add --i18n flag * Add --routes flag * Allow negated flags * Update summary to latest design * Throw early when flags have invalid values * Rename pathname to subfolders * Show shortcut prompt earlier * Align with design updates * Add storefront name to package.json * Show storefront name in dev command * Update next steps * Oclif manifest * Skip some tests for now * Add HR as dev dependency for types on CI * Revert "Add HR as dev dependency for types on CI" This reverts commit 7ded27f. * Align with design updates * Fix race condition * Fix project overwrite * Fix detecting TS in i18n setup * Avoid importing types from HR directly * Show login banner * Refactor shortcut code files * Refactor generate route code files * Fix shortcut tests * Refactor: split init command in multiple files * Extra common prompts * Rename types * Refactor route generate input * Extract renderRoutePrompt * Disable dts in CLI to avoid Tsup bug copying d.ts files in templates * Make renderProjectReady more flexible * Extract flags * Refactor handleCliShortcut * Minor refactoring * Add 'setup' command * Ensure Remix versions are the same when copying assets * Fix name added to package.json * Changesets
* Do not overwrite version added by changeset * Remove create-hydrogen from version:post
* Improve support for Node 16 * Catch errors when adding virtual routes * Add gitignore to new projects
The SFAPI doesn't always return the same domain for primaryDomain and menu URLs. Check each menu URL if internal by seeing if it equals the primaryDomain or the PUBLIC_STORE_DOMAIN environment variable.
* Product form implementation Shopify/hydrogen-internal#23
#1040) * feat(hydrogen-react): include `applicable` in discountCode in defaultCartFragment The `useCart` hook from hydrogen-react uses the defaultCartFragment if not specified otherwise. Developers, when using discountCodes, will often want to know is the code is valid.
* add @total-typescript/ts-reset to skeleton and demo-store Co-authored-by: Helen Lin <helen.lin@shopify.com>
* Remove relative path indicators from placeholders * Add a skip option to CSS prompt
* Revert client-side render with proper cart fix
* Use skeleton template as starter instead of hello-world * Allow hello-world as a remote starter * Filter out non entry files * Refactor: turn generateRoute into generateProjectFile * Refactor: rename function * Fix reference to hello-world * Copy dependencies when generating routes * Support Remix v1 when generating routes with locales * Oclif manifest * Support multiline imports and exports * Support export statements * Support other extensions and index files * Refactor * Support .server.x patterns * Change skeleton template to v2 route convention * Generate routes from v2 route convention files * Update route summary * Generate root and entry files in setup command * Fix package.json name replacement * Add comments to route generation logic
* Rename i18n to markets * Highlight next step command * Change other commands to use 'markets' * Make commands stable and visible * Oclif manifest
* Reword prompts to use 'select' * Reword i18n prompt
* Load env vars in parallel to initial build; Do not throw on env var errors * Reload env variables when project is linked to storefront * Wrap all calls in try-catch * Resolve variables in parallel and avoid crashing on fetch fail * Add .env to Remix watcher in templates * Changesets
* Adjust CLI dependencies * Update Oclif manifest generation * Add a build check to ensure virtual-routes are created * Changesets
* Add h2 prefix to HR StorefrontClient errors and warnings * Replace errors and warnings with banners * Add log prefix to createStorefrontClient * Debounce logs * Add query and variables to GraphiQL * Use function name as the scope * Add prefix and graphql info to Storefront API errors * Show GraphiQL link on SFAPI errors * Add favicon to GraphiQL * Update to React 18 in GraphiQL * Update to GraphiQL v3 * Automatically prettify query and variables in GraphiQL * Sanitize error stack and improve log format * Add log-type to prefix pattern * Changesets * Minor refactor * Unit test for h2 patterns
* Support globs in route map * Group urls in summary * Fixes
* skeleton template 2.0 * fix eslint issues * Fix CLI tests * various fixes * Fix CLI * move cart query to server.ts * remove --codegen command from build * add codegen command * Fix CLI tests * fix dependencies --------- Co-authored-by: Fran Dios <fran.dios@shopify.com>
. cleanup . . .
juanpprieto
changed the title
|
We need to solve this the right way with SF API. On the roadmap. Closing for now. |
|
Re-opening this assuming we can pursue with offset-based pagination. Some quick thoughts: Leverage offset-based pagination to kick off multiple parallel subqueries At 500 sitemaps linked from the index, that gets us to 1.25M routes — which is I think OK for now / as a target. The current implementation, because of cursor-based pagination, means you're limited to ~12,500 products, right? (~50 pagination requests * 250). With offset-based, we could then in 1 query know how many sitemaps to link to from the sitemap index, and then pass the right We should consider simplifying the DX by packaging up some of this stuff Note that there's no rate limit difference between API tokens Update Direction has changed as we discovered technical constraints. Closing this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
/sitemap.xmlis now index/paginatedWith
250pageSizeWith mixed pageSizes (emulate multiple paginated sub sitemaps)
A paginated sitemap example