Skip to content
This repository has been archived by the owner on Jun 12, 2024. It is now read-only.

Create prepareSitemap Batfish helper to improve sitemap #427

Merged
merged 17 commits into from
Apr 14, 2021
Merged

Create prepareSitemap Batfish helper to improve sitemap #427

merged 17 commits into from
Apr 14, 2021

Conversation

katydecorah
Copy link
Contributor

@katydecorah katydecorah commented Apr 5, 2021 •
edited
Loading

This branch will merge into #431

This PR adds a prepareSitemap Batfish helper to improve sitemap output. The function will automatically remove pages with hideFromSearchEngines: true and splitPage: true from the sitemap. This will prevent URLs we don't want users to visit from appearing in the sitemap.

Fixes #415

How to test

There are tests in this PR that assert the output of this function and the catalog's site's sitemap.

QA checklist

Not applicable for this function.

Before merge

  • Add entry to CHANGELOG.md to describe changes.

@katydecorah katydecorah changed the base branch from main to 4.0.0 April 6, 2021 17:20
@katydecorah katydecorah marked this pull request as ready for review April 7, 2021 16:28
@katydecorah katydecorah requested a review from a team April 7, 2021 18:22
Katy DeCorah added 2 commits April 8, 2021 13:44
Merge branch '4.0.0' into prepare-sitemap
31db891 
* 4.0.0:
  Update dependencies (#433)
  Make code toggles in test cases interactive or label if they are not (#432)
Update package-lock.json
9d1f1e8
@danswick danswick requested review from danswick and removed request for a team April 14, 2021 02:00
danswick
danswick approved these changes Apr 14, 2021
View reviewed changes
Copy link
Contributor

@danswick danswick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great helper, @katydecorah. I just had a couple of questions about how implementation would work for sites that don't need to exclude any pages.

.travis.yml
@@ -6,3 +6,5 @@ notifications:
scripts:
- npm test
- npm run build-docs
# run sitemap tests after build to assert sitemap is correct
- jest src/helpers/batfish/__tests__/sitemap.test.js
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

docs/src/pages/guides/batfish-helpers.md
@@ -252,6 +253,69 @@ class PageShell extends React.Component {
{{JSON.stringify(splitPages, null,2)}}
```

## Prepare sitemap

`prepareSitemap` creates a data file that lists all files with `hideFromSearchEngines: true` or `splitPage: true` set in the page's frontMatter. Use this generated file with Batfish's [`sitemap.ignoreFile`](https://github.com/mapbox/batfish/blob/main/docs/configuration.md#sitemap) option to generate a sitemap that excludes any paths in the prepared file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you only need to worry about prepareSitemap if you're planning on hiding some files from search engines? Or will all dr-ui / Batfish sites need the sitemapIgnore: ... prepareSitemap() combo?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You only need to introduce this Batfish helper if you have pages that you want to exclude, otherwise the sitemap will build without intervention.

docs/src/pages/guides/batfish-helpers.md
return {
sitemap: {
// generated by the sitemapIgnore dataSelector
ignoreFile: '_site_tmp/data/sitemap-ignore.js'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar question here: you can use sitemap: true if you don't want to exclude some pages from the sitemap, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't want to intervene with the sitemap, you don't need to specific sitemap at all. For Batfish, the sitemap will automatically build if you define siteOrigin, ref: https://github.com/mapbox/batfish/blob/main/docs/configuration.md#sitemap

@katydecorah katydecorah merged commit de93e03 into 4.0.0 Apr 14, 2021
@katydecorah katydecorah deleted the prepare-sitemap branch April 14, 2021 13:34
katydecorah pushed a commit that referenced this pull request Apr 14, 2021
Merge branch '4.0.0' into jsx-no-bind
a28153d 
* 4.0.0:
  Create `prepareSitemap` Batfish helper to improve sitemap (#427)
  Add eslint-plugin-jsx-a11y (#435)
katydecorah pushed a commit that referenced this pull request Apr 14, 2021
Merge branch '4.0.0' into video-controls
ddf040e 
* 4.0.0:
  Create `prepareSitemap` Batfish helper to improve sitemap (#427)
  Add eslint-plugin-jsx-a11y (#435)
  🔧 Run prettier
  Improve color contrast on code and anchor elements (#434)
  Replace stateless components with pure components (#414)
  Update dependencies (#433)
  Make code toggles in test cases interactive or label if they are not (#432)
  Prepare 4.0.0 changelog
katydecorah pushed a commit that referenced this pull request Apr 16, 2021
Merge branch '4.0.0' into lh-ci
d98202b 
* 4.0.0:
  Make `filename` optional for `CodeSnippetTitle`
  Replace `id` with `aria-label` in VimeoPlayImage (#438)
  Create `prepareSitemap` Batfish helper to improve sitemap (#427)
  Add eslint-plugin-jsx-a11y (#435)
  🔧 Run prettier
  Improve color contrast on code and anchor elements (#434)
  Replace stateless components with pure components (#414)
  Update dependencies (#433)
  Make code toggles in test cases interactive or label if they are not (#432)
  Prepare 4.0.0 changelog
katydecorah pushed a commit that referenced this pull request Apr 22, 2021
Merge branch '4.0.0' into add-search-facade
687093a 
* 4.0.0:
  Use passive event listeners on `Search`, `NumberedCodeSnippet`, and `OnThisPage` to improve performance (#437)
  Make "With token" `DemoIframe` a no render test case (#440)
  Make `filename` optional for `CodeSnippetTitle`
  Replace `id` with `aria-label` in VimeoPlayImage (#438)
  Create `prepareSitemap` Batfish helper to improve sitemap (#427)
  Add eslint-plugin-jsx-a11y (#435)
katydecorah pushed a commit that referenced this pull request May 11, 2021
@danswick
⚡️ 4.0.0 (#431)
d35b4d6 
* Prepare 4.0.0 changelog

* Make code toggles in test cases interactive or label if they are not (#432)

* Update dependencies (#433)

* Update dependencies

* Update package-lock.json

* Update rehype-slug

* Fix links

* Update page-layout.md

* Update snapshots

* Replace stateless components with pure components (#414)

* Use PureComponent on stateless components

* Update package-lock.json

* 4.0.0-alpha.0

* Use PureComponents where components do not require rerenders

* 4.0.0-alpha.1

* prefer-const

* Update CHANGELOG.md

* Make more components pure

* 4.0.0-alpha.2

* 4.0.0-alpha.3

* Improve color contrast on code and anchor elements (#434)

* Expand a code color contrast improvement

* Increase color contrast on .color-gray code

* Increase color contrast for prism's .token.regex,.token.important

* Clean up test cases

* Update CHANGELOG.md

* Clean up test cases

* Clean up test cases

* Update docs/src/pages/guides/a11y.md

Co-authored-by: Colleen McGinnis <colleen.mcginnis@mapbox.com>

* Reduce changes

Co-authored-by: Colleen McGinnis <colleen.mcginnis@mapbox.com>

* 🔧 Run prettier

* Add eslint-plugin-jsx-a11y (#435)

* Add eslint-plugin-jsx-a11y

* Fix props

* Update CHANGELOG.md

* Update search.test.js.snap

* Update eslint

* Update eslint

* Create `prepareSitemap` Batfish helper to improve sitemap (#427)

* Create prepareSitemap Batfish helper

* Update CHANGELOG.md

* Clean up, add `docsPath`, and `outputDirectory`

* Exclude `splitPage` from sitemap

* Assert sitemap

* Extract urls

* Sort sitemap

* Remove snapshot

* Update sitemap.test.js

* Update batfish-helpers.md

* Update batfish-helpers.md

* Copyedit

* Update package-lock.json

* Replace `id` with `aria-label` in VimeoPlayImage (#438)

* Replace `id` with aria-label to prevent duplicated id

* Update CHANGELOG.md

* Make `filename` optional for `CodeSnippetTitle`

* Make "With token" `DemoIframe` a no render test case (#440)

* Make "With token" `DemoIframe` a no render test case

* Rename

* Use passive event listeners on `Search`, `NumberedCodeSnippet`, and `OnThisPage` to improve performance (#437)

* Use passive event listeners on `Search`, `NumberedCodeSnippet`, and `OnThisPage` to improve performance.

* Update CHANGELOG.md

* Reconfigure `Video` options to optimize for user's settings around autoplay (#276)

* show video controls on hover; show play button for browsers with disabled autoplay

* Merge branch 'main' into video-controls

* main:
  Add the catalog site  (#137)
  Delete dependabot.yml
  Only direct dependencies
  Create dependabot.yml (#296)
  Update Travis bad
  Update devDependencies (#294)

* Add example

* More comments

* Reduce changes

* eslint and fix button styles

* Clean up button

* Set playsInline to true if autoPlay is true; make `muted` a prop

* Update test cases description to indicate the autoplay state

* Update description

* Update snapshots

* Update CHANGELOG.md

* Create `playsInline` prop

* Append hash to src to enable preview on iOS

* Update video.js

* Adds optional `poster` prop

* Remove hack

* Add poster example

* Fix paths

* Update descriptions

* Apply suggestions from code review

* Load Search with a facade (#430)

* Load Search with a facade

* Add @loadable/babel-plugin

* Update CHANGELOG.md

* Fix logic

* Update search.test.js.snap

* Define development babel env to prevent `@loadable/babel-plugin` from loading on start (because it fails)

* Adjusting snapshots

* Revert "Adjusting snapshots"

This reverts commit d2e7fd9.

* Clean up props

* Comments

* Reduce changes

* Reduce changes

* Reduce changes

* Reduce changes

* Clean up props

* Use SearchInput in SearchBox

* Use SearchButton in SearchBox

* Reduce changes

* Add missing props

* Consolidate wrapper

* 4.0.0-alpha.4

* Try dynamic import

* Update package-lock.json

* 4.0.0-alpha.5

* Update package-lock.json

* Reduce changes

* Reorganize files

* 4.0.0-alpha.6

* Switch to noRenderCase

* Reduce changes

* Add passive event listener

* Allow autofocus

* Enable react/jsx-no-bind (#442)

* jsx-no-bind

* Update CHANGELOG.md

* Simplify, reduce changes

* The `NumberedCodeSnippet` prop `onCopy` is deprecated and will automatically send a track event to Segment

* Remove unused onCopy

* Simplify

* Add onCopy

* Create onCopy function

* Update src/components/code-snippet/on-copy.js

* Update dependencies

* Update caniuse

* Update devDependencies

* Update CHANGELOG.md

* Fix NavigationAccordion tests (#443)

* 4.0.0-alpha.7

* Fix prepareSitemap: append index.html to files ending in index.js (#444)

* 4.0.0-alpha.8

* Update batfish

* Update caniuse-lite

* Fix props on search toggle (#448)

* Fix props on search toggle

* 4.0.0-alpha.9

* For searches with no results, retry if Swiftype thinks there is a spelling error (#451)

* pass through swiftype API spelling option

* move prettier'd comment

* Update search-ui, downshift

* Update caniuse-lite

* Update @sentry/browser

* Send search results with no queries to Sentry (#452)

* Send search results with no queries to Sentry

* Update CHANGELOG.md

* Update prettier

* Fix Sentry test

* 4.0.0-alpha.10

* Switch Sentry environment back to `main`

* Fix Search/Sentry, only send when results change

* 4.0.0

* Clean up

Co-authored-by: Colleen McGinnis <colleen.mcginnis@mapbox.com>
Co-authored-by: Dan Swick <dan.swick@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@danswick danswick danswick approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Proposal: add sitemapIgnore Batfish helper
2 participants
@katydecorah @danswick