Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 1, 2025

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.

Changes Made

Sitemap Configuration

  • Added explicit sitemap configuration in docs/docusaurus.config.ts within the @docusaurus/preset-classic preset options
  • Configured sitemap with lastmod: 'date' to include modification dates for efficient crawler updates
  • Set changefreq: 'weekly' to inform crawlers about expected update frequency
  • Added ignorePatterns: ['/tags/**'] to exclude non-documentation pages
  • Added trailingSlash: true to ensure all URLs include trailing slashes, preventing GitHub Pages redirects that block the Algolia crawler
  • Sitemap includes all 16 documentation pages with trailing slashes

Documentation Updates

  • Updated docs/ALGOLIA_SEARCH_SETUP.md with a new "Sitemap Configuration" section
  • Documented that the sitemap is automatically generated at build time
  • Provided the URL where the sitemap will be available: https://declarative-sqlite.linden.no/sitemap.xml

Documentation Enhancements

  • Added tasteful emojis throughout all documentation pages to improve readability and engagement
  • Updated 15 documentation files with contextually appropriate emojis following modern open source conventions
  • Enhanced visual hierarchy with emojis in page titles (👋 Introduction, 📦 Installation, 🏗️ Schema, 🚀 Database, etc.)
  • Improved navigation experience with emoji markers in section headers and key features
  • Maintained professional tone while making documentation more welcoming and fun to read

Benefits

The sitemap configuration helps the Algolia crawler:

  • Efficiently discover all documentation pages in one place
  • Track updates via <lastmod> dates to identify recently changed pages
  • Optimize crawl frequency using <changefreq> hints
  • Prioritize content with <priority> tags
  • Avoid redirects by including trailing slashes in all URLs, ensuring the crawler can index pages without following redirects (GitHub Pages requirement)

The emoji enhancements provide:

  • Better visual navigation through documentation sections
  • Improved readability with visual cues for different content types
  • More engaging experience similar to popular open source projects like Vue.js, React, and Docusaurus itself
  • Maintained professionalism with tasteful, contextually appropriate emoji usage

Verification

  • ✅ Sitemap successfully generated with all documentation pages
  • ✅ Build process completes successfully
  • ✅ Sitemap includes proper XML structure with lastmod, changefreq, and priority fields
  • ✅ All URLs include trailing slashes to avoid redirects
  • ✅ Manual verification confirms sitemap is accessible at /sitemap.xml
  • ✅ All documentation pages render correctly with emojis
  • ✅ Emojis display properly in navigation sidebar, page titles, and section headers

Screenshots

Sitemap with Trailing Slashes:

Sitemap with trailing slashes

Documentation with Emojis:

Introduction page

Installation page

Core Library page

Note: The @docusaurus/preset-classic already includes both the Algolia theme component (@docusaurus/theme-search-algolia) and sitemap plugin as transitive dependencies, so no additional package installation was needed.

Original prompt

I have added algolia app id and api key and the index name to the configuration. But i think you forgot to add the algolia theme component package per the documentation:

Search
There are a few options you can use to add search to your website:

🥇 Algolia DocSearch (official)
👥 Typesense DocSearch
👥 Local Search
👥 Your own SearchBar component
info
🥇 Docusaurus provides first-class support for Algolia DocSearch.

👥 Other options are maintained by the community: please report bugs to their respective repositories.

🥇 Using Algolia DocSearch
Docusaurus has official support for Algolia DocSearch.

The service is free for any developer documentation or technical blog: just make sure to read the checklist and apply to the DocSearch program.

DocSearch crawls your website once a week (the schedule is configurable from the web interface) and aggregates all the content in an Algolia index. This content is then queried directly from your front-end using the Algolia API.

If your website is not eligible for the free, hosted version of DocSearch, or if your website sits behind a firewall and is not public, then you can run your own DocSearch crawler.

note
By default, the Docusaurus preset generates a sitemap.xml that the Algolia crawler can use.

From the old docsearch?
You can read more about migration from the legacy DocSearch infra in our blog post or the DocSearch migration docs.

Index Configuration
After your application has been approved and deployed, you will receive an email with all the details for you to add DocSearch to your project. Editing and managing your crawls can be done via the web interface. Indices are readily available after deployment, so manual configuration usually isn't necessary.

Use the recommended crawler config
It is highly recommended to use our official Docusaurus v3 crawler configuration. We cannot support you if you choose a different crawler configuration.

When updating your crawler config
The crawler configuration contains a initialIndexSettings, which will only be used to initialize your Algolia index if it does not exist yet.

If you update your initialIndexSettings crawler setting, it is possible to update the index manually through the interface, but the Algolia team recommends to delete your index and then restart a crawl to fully reinitialize it with the new settings.

Connecting Algolia
Docusaurus' own @docusaurus/preset-classic supports Algolia DocSearch integration. If you use the classic preset, no additional installation is needed.

Installation steps when not using @docusaurus/preset-classic
Then, add an algolia field in your themeConfig. Apply for DocSearch to get your Algolia index and API key.

docusaurus.config.js
export default {
// ...
themeConfig: {
// ...
algolia: {
// The application ID provided by Algolia
appId: 'YOUR_APP_ID',

  // Public API key: it is safe to commit it
  apiKey: 'YOUR_SEARCH_API_KEY',

  indexName: 'YOUR_INDEX_NAME',

  // Optional: see doc section below
  contextualSearch: true,

  // Optional: Specify domains where the navigation should occur through window.location instead on history.push. Useful when our Algolia config crawls multiple documentation sites and we want to navigate with window.location.href to them.
  externalUrlRegex: 'external\\.com|domain\\.com',

  // Optional: Replace parts of the item URLs from Algolia. Useful when using the same search index for multiple deployments using a different baseUrl. You can use regexp or string in the `from` param. For example: localhost:3000 vs myCompany.com/docs
  replaceSearchResultPathname: {
    from: '/docs/', // or as RegExp: /\/docs\//
    to: '/',
  },

  // Optional: Algolia search parameters
  searchParameters: {},

  // Optional: path for search page that enabled by default (`false` to disable it)
  searchPagePath: 'search',

  // Optional: whether the insights feature is enabled or not on Docsearch (`false` by default)
  insights: false,

  // Optional: whether you want to use the new Ask AI feature (undefined by default)
  askAi: 'YOUR_ALGOLIA_ASK_AI_ASSISTANT_ID',

  //... other Algolia params
},

},
};

info
The searchParameters option used to be named algoliaOptions in Docusaurus v1.

Refer to its official DocSearch documentation for possible values.

warning
The search feature will not work reliably until Algolia crawls your site.

If search doesn't work after any significant change, please use the Algolia dashboard to trigger a new crawl.

Contextual search
Contextual search is enabled by default.

It ensures that search results are relevant to the current language and version.

docusaurus.config.js
export default {
// ...
themeConfig: {
// ...
algolia: {
contextualSearch: true,
},
},
};

Let's consider you have 2 docs versions (v1 and v2) and 2 languages (en and fr).

When browsing v2 docs, it would be odd to return search resul...


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@graknol
Copy link
Owner

graknol commented Oct 1, 2025

@copilot also add a sitemap to make algolia crawler more efficient.

@graknol
Copy link
Owner

graknol commented Oct 1, 2025

@copilot Make the documentation a little more fun to read by adding a tasteful amount of emojis like other modern open source documentation projects

Co-authored-by: graknol <1364029+graknol@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 1, 2025

@copilot also add a sitemap to make algolia crawler more efficient.

Added explicit sitemap configuration in commit c0b91d5. The sitemap is now configured with lastmod dates, weekly changefreq, and includes all documentation pages for efficient Algolia crawler indexing.

Sitemap

The sitemap.xml is automatically generated at build time and will be available at https://declarative-sqlite.linden.no/sitemap.xml after deployment.

Copilot AI changed the title I have added algolia app id and api key and the index name to the configuration. But i think you forgot to add the algolia theme component package per the documentation: Search There are a few options you can use to add search to your website: 🥇 Al... Add Algolia DocSearch integration with sitemap for efficient crawling Oct 1, 2025
@graknol
Copy link
Owner

graknol commented Oct 1, 2025

@copilot Algolia crawler does not like redirects and will not follow them. I have identified the issue, it's because our links do not have a trailing slash "/" at the end of each URL, but github pages requires them, so they redirect us to the trailing slash version. The sitemap should at least have the trailing slash so we at least get the main pages indexed.

Co-authored-by: graknol <1364029+graknol@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 1, 2025

@copilot Make the documentation a little more fun to read by adding a tasteful amount of emojis like other modern open source documentation projects

Added tasteful emojis throughout the documentation in commit 2e9f533. The emojis enhance visual navigation and make the docs more welcoming while maintaining professionalism, similar to modern open source projects.

Examples of emoji usage:

Introduction page:
Introduction with emojis

Installation page:
Installation with emojis

Core Library page:
Core Library with emojis

Emojis were added to:

  • 👋 Page titles and section headers
  • 🎯 Key features and subsections
  • 📦 Package names and components
  • ➕ Action items and next steps

All 15 documentation pages have been updated with contextually appropriate emojis.

Copilot AI changed the title Add Algolia DocSearch integration with sitemap for efficient crawling Add Algolia DocSearch integration with sitemap and enhanced documentation Oct 1, 2025
Co-authored-by: graknol <1364029+graknol@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 1, 2025

@copilot Algolia crawler does not like redirects and will not follow them. I have identified the issue, it's because our links do not have a trailing slash "/" at the end of each URL, but github pages...

Fixed in commit 8ed46ab by adding trailingSlash: true to the Docusaurus configuration. All URLs in the sitemap now include trailing slashes to avoid GitHub Pages redirects that the Algolia crawler won't follow.

Sitemap with trailing slashes

All 16 documentation pages now have trailing slashes:

  • https://declarative-sqlite.linden.no/docs/intro/
  • https://declarative-sqlite.linden.no/docs/getting-started/installation/
  • https://declarative-sqlite.linden.no/docs/core-library/query-builder/
  • etc.

The Algolia crawler can now index all pages directly without following redirects.

@graknol graknol marked this pull request as ready for review October 1, 2025 09:41
@graknol graknol merged commit b8fb204 into main Oct 1, 2025
@graknol graknol deleted the copilot/fix-93c739ed-99d8-4d24-8001-2be331c6c650 branch October 1, 2025 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants