Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📥 Load TOC from toc entry #1188

Merged
merged 44 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
fe7148c
feat: load TOC
agoose77 May 1, 2024
807fa6e
chore: appease linter
agoose77 May 7, 2024
cd23aec
feat: enable glob
agoose77 May 8, 2024
f6b33f2
docs: drop JB toc
agoose77 May 8, 2024
510ea64
docs: wip
agoose77 May 8, 2024
951c484
feat: add --write-toc support
agoose77 May 9, 2024
053710b
docs: drop reference to _toc.yml`
agoose77 May 9, 2024
059d9d2
fix: improve warning for non-files
agoose77 May 9, 2024
2498904
test: fix test
agoose77 May 9, 2024
a1d88c1
chore: appease linter
agoose77 May 9, 2024
dce3dbf
chore: use main package-lock
agoose77 May 10, 2024
5ba8c4c
chore: update lockfile
agoose77 May 10, 2024
aee0012
fix: add missing file
agoose77 May 10, 2024
ff7dcfb
feat!: drop URL entry type
agoose77 May 14, 2024
bb2b584
test: add single end-to-end test
agoose77 May 14, 2024
f2ef55d
Revert "feat!: drop URL entry type"
agoose77 May 15, 2024
b68cffc
chore: add top-level attribute
agoose77 May 15, 2024
0303fbc
📦 Bump myst-toc version in monorepo packages
fwkoch May 22, 2024
31aa783
🧪 Update tests and types for new toc
fwkoch May 22, 2024
c8504b9
🥇 Prioritize config toc over jupyterbook toc and allow upgrade path f…
fwkoch May 22, 2024
4a9fd75
✨ Do not use project _toc.yml for article export if project.toc is av…
fwkoch May 23, 2024
c643735
🪵 Improve log messages when running myst init with/without --write-toc
fwkoch May 23, 2024
2382b59
🧪 Update tests to check toc vs _toc.yml priority for article export
fwkoch May 23, 2024
1b15d33
🚦 Do not hard fail on url in toc - only warn
fwkoch May 23, 2024
586cb70
🪵 Add debug log message when encountering nested _toc.yml file
fwkoch May 23, 2024
8eeb770
🔧 Restore support of export providing a toc file
fwkoch May 23, 2024
817ef38
🎚 Respect top_level during export
fwkoch May 23, 2024
53ff9e1
🧪 Update multi-page e2e tests
fwkoch May 23, 2024
023e473
🔧 Fix export top-level warning
fwkoch May 23, 2024
40486e9
🧪 Add e2e test for nested toc
fwkoch May 23, 2024
812d900
Clean up table of contents description and page metadata
choldgraf May 24, 2024
987a3f3
fix: restore old link
agoose77 May 24, 2024
e8a20c5
docs: move @choldgraf changes
agoose77 May 24, 2024
aa7aaa6
Update docs/table-of-contents.md
agoose77 May 26, 2024
55fe407
chore: improve error message
agoose77 May 26, 2024
7ed16d8
chore: drop vitest coverage
agoose77 May 26, 2024
e620d77
Update docs/table-of-contents.md
agoose77 May 26, 2024
921adfc
Merge remote-tracking branch 'refs/remotes/origin/agoose77/feat-load-…
agoose77 May 26, 2024
240417f
test: add tests
agoose77 May 28, 2024
9070a22
docs: restore legacy section
agoose77 May 28, 2024
87e54bd
Merge branch 'main' into agoose77/feat-load-toc-from-frontmatter
agoose77 May 28, 2024
5afe685
Relax some changes to introduce new toc more gently (#1242)
fwkoch May 28, 2024
3a6ace7
fix: comma
agoose77 May 28, 2024
fc5fc6b
test: fix test cases
agoose77 May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/sharp-dogs-pretend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'myst-toc': patch
---

Comment out unused myst toc properties
84 changes: 0 additions & 84 deletions docs/_toc.yml

This file was deleted.

85 changes: 85 additions & 0 deletions docs/myst.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ project:
TLA: Three Letter Acronym
OA: Open Access
CSV: comma-separated values
TOC: table of contents
plugins:
- directives.mjs
- unsplash.mjs
Expand All @@ -61,6 +62,90 @@ project:
- /jtex/create-a-latex-template
- /jtex/create-a-beamer-template
- /jtex/contribute-a-template
toc:
- file: index
- title: Quickstart Tutorials
children:
- file: quickstart.md
- file: quickstart-myst-websites.md
- file: quickstart-myst-documents.md
- file: quickstart-myst-markdown.md
- file: quickstart-jupyter-lab-myst.md
- title: Authoring
children:
- file: typography.md
- file: admonitions.md
- file: figures.md
- file: math.md
- file: tables.md
- file: code.md
- file: cross-references.md
- file: external-references.md
- file: embed.md
- file: citations.md
- file: proofs-and-theorems.md
- file: exercises.md
- file: blocks.md
- file: diagrams.md
- file: asides.md
- file: dropdowns-cards-and-tabs.md
- file: glossaries-and-terms.md
- file: writing-in-latex.md
- title: Executable Content
children:
- file: interactive-notebooks.ipynb
- file: notebooks-with-markdown.md
- file: reuse-jupyter-outputs.md
- file: integrating-jupyter.md
- file: execute-notebooks.md
- title: Websites
children:
- file: website-templates.md
- file: table-of-contents.md
# - file: themes-and-extensions.md
- file: seo-and-social.md
- file: accessibility-and-performance.md
# - file: search.md
- file: analytics.md
- file: website-downloads.md
- file: deployment.md
children:
- file: deployment-github-pages.md
- file: deployment-curvenote.md
- file: deployment-netlify.md
# - file: publishing.md
- title: Documents
children:
- file: documents-exports.md
- file: creating-pdf-documents.md
- file: creating-word-documents.md
- file: creating-jats-xml.md
- title: Extensions
children:
- file: plugins.md
- title: Reference
children:
- file: background.md
- file: guiding-principles.md
- file: installing.md
- file: installing-prerequisites.md
- file: commonmark.md
- file: syntax-overview.md
- file: directives.md
- file: roles.md
- file: frontmatter.md
- file: document-parts.md
- file: settings.md
- file: glossary.md
- file: xref.md
- title: Contribute
children:
- file: contributing.md
- file: contribute-templates.md
- file: contribute-docs.md
- file: contribute-build-locally.md
- file: contribute-add-feature.md

site:
title: MyST Markdown Guide
domains:
Expand Down
169 changes: 140 additions & 29 deletions docs/table-of-contents.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,135 @@
---
title: Table of Contents
description: The Table of Contents is the left-hand navigation for your site, it can be auto-generated or can be explicitly defined in a _toc.yml.
description: The Table of Contents is the left-hand navigation for your site, it can be auto-generated or can be explicitly defined in `myst.yml`.
thumbnail: thumbnails/table-of-contents.png
---

+++

The Table of Contents is the left-hand navigation for your site. It can either be auto-generated, following some simple heuristics described below, or can be explicitly defined in a `_toc.yml` using the `jb-book` format.
The Table of Contents defines the structure of your MyST project. For website exports, it defines the left-hand navigation for your site.
It is defined in the `toc` attribute of [the project frontmatter](frontmatter.md#in-a-myst-yml-file).

## Generating a Table of Contents

By default the table of contents is left implicit, and follows rules laid out in the next section. To make this table of contents _explicit_, you can call:
To automatically add a `toc` section to your `myst.yml` file using filenames to define ordering, use the following command:

```shell
myst init --write-toc
```

This will create a `_toc.yml` in the current directory, you can read more about the [table of contents format](#toc-format) below.

+++

## Auto-generating a Table of Contents
(toc-format)=

## Structure the Table of Contents

The MyST TOC comprises a simple tree structure, built from `file`s, `url`s, `pattern`s, and `children`. For example, a simple TOC consisting of files:

:::{code} yaml
:filename: myst.yml

version: 1
project:
toc:
- file: root.md
- file: first-child.md
- file: second-child.md
:::

### URL entries

URLs can be defined in the TOC. These URLs are links to external references within your table of contents. URLs are ignored in non-web exports.

:::{warning}
Currently these URLs are also ignored in MyST sites. Support will be added in a future release.
:::

:::{code} yaml
:filename: myst.yml

version: 1
project:
toc:
- file: root.md
- url: 'https://google.com'
:::

### Glob pattern matching

You can specify glob-like patterns in the TOC with the `pattern` key.
The files matched by `pattern` will be expanded as a series of `file` entries.

For example, with a folder with `root.md`, `first-child.md`, `second-child.md`, the following two `toc` entries are equivalent:

:::::{grid} 1 2 2 2
::::{card} Pattern-matching
:::{code} yaml
:filename: myst.yml

version: 1
project:
toc:
- file: root.md
- pattern: '*-child.md'
:::
::::
::::{card} No pattern-matching
:::{code} yaml
:filename: myst.yml
:linenos:
:emphasize-lines: 5,6

version: 1
project:
toc:
- file: root.md
- file: first-child.md
- file: second-child.md
:::
::::
:::::

### Nesting pages and dropdowns

For larger projects, you can group the content using the `children` key, which can be defined for both `url` and `file` entries:

:::{code} yaml
:filename: myst.yml

version: 1
project:
toc:
- file: root.md
- file: part-1.md
children:
- file: part-1-first-child.md
- file: part-1-second-child.md
- file: part-2.md
children:
- file: part-2-first-child.md
- file: part-2-second-child.md
:::

You can nest children under a `title` without specifying a parent `file`.
This will create a dropdown of pages in the Table of Contents.

:::{code} yaml
:filename: myst.yml

version: 1
project:
toc:
- file: root.md
- title: Part 1
children:
- file: part-1-first-child.md
- file: part-1-second-child.md
- title: Part 2
children:
- file: part-2-first-child.md
- file: part-2-second-child.md
:::

When there is no `_toc.yml` defined an implicit table of contents is defined by the file system structure. All markdown and notebook files will be found in the working directory and all sub-directories. Filenames are not treated as case sensitive, and files are listed before folders. All hidden directories are ignored (e.g. `.git`) and the `_build` directory is also ignored.

## Implicit Table of Contents from filenames

When there is no `toc` field defined in your root `myst.yml`, the TOC is defined by the file system structure. All markdown and notebook files will be found in the working directory and all sub-directories. Filenames are not treated as case sensitive, and files are listed before folders. All hidden directories are ignored (e.g. `.git`) and the `_build` directory is also ignored.

The ordering of the table of contents will sort alphabetically as well as order by number, ensuring that, for example, `chapter10` comes after `chapter9`.

Expand Down Expand Up @@ -53,9 +160,6 @@ The “root” of a site is the page displayed when someone browses to the index
7. `main.ipynb`
8. The first `.ipynb` file found alphabetically

```{note}
All of these can be over-ridden by choosing an explicit `_toc.yml`, when that is present it will be used.
```

### Excluding Files

Expand All @@ -78,11 +182,31 @@ project:

Note that when these files are excluded, they can still be specifically referenced by other files in your project (e.g. in an {myst:directive}`include directives <include>` or as a download), however, a change in those files will not trigger a build. An alternative in this case is to generate a table of contents (see [](./table-of-contents.md)). By default hidden folders (those starting with `.`, like `.git`), `_build` and `node_modules` are excluded.

(toc-format)=
## Nested files will have flattened URLs

If a file is nested under a folder within your MyST project, for web-based exports its URL will be flattened to have a "slug" that removes folder information. For example:

- `folder1/folder2/01_my_article.md` becomes `/my-article`

All internal links will automatically be updated, and there is a `file` property that is exported as metadata in your site.
See [](xref.md) for more details on how cross-references are stored.
agoose77 marked this conversation as resolved.
Show resolved Hide resolved

:::{note} URL Nesting
URL nesting that matches the folder structure is a requested feature that is being tracked in [#670](https://github.com/executablebooks/mystmd/issues/670).
:::

:::{note} Compatibility with JupyterBook
:class: dropdown

(toc-format-legacy)=
## Defining a `_toc.yml` using Jupyter Book’s format
agoose77 marked this conversation as resolved.
Show resolved Hide resolved

The `_toc.yml` can be defined for a site, and uses the format describe by Jupyter Book, the documentation for the format is fully described in [Jupyter Book](https://jupyterbook.org/en/stable/structure/toc.html). Briefly, it defines a `format` as `jb-book` and can list a number of `chapters` with files. The file paths are relative to your `_toc.yml` file and can optionally include the extension.
:::{warning}
Support for `_toc.yml` exists only for compatability reasons, and will be removed in future.
New users should use `myst.yml` instead.
:::

Site table of contents may be defined with a `_toc.yml` file, following the Jupyter Book format. The documentation for this format is fully described in [Jupyter Book](https://jupyterbook.org/en/stable/structure/toc.html). Briefly, it defines a `format` as `jb-book` and can list a number of `chapters` with files. The file paths are relative to your `_toc.yml` file and can optionally include the extension.

```yaml
format: jb-book
Expand All @@ -109,17 +233,4 @@ parts:
- file: path/to/part2/chapter1
- file: path/to/part2/chapter2
```

+++

## Nesting of Files in URLs

MyST can have any level of nesting in a file-system of your project, however, when it is displayed in the URL in `mystmd`, these nesting will be flattened to have a single "slug" that is contained in the project.

- `project/folder2/01_my_article.md` becomes `project/my-article`

All internal links will automatically be updated, and there is a `file` property that is exported as metadata. Add `.json` to the end of any url in your site to see the full data of the page.

:::{note} URL Nesting
URL nesting that matches the folder structure is a requested feature that is being tracked in [#670](https://github.com/executablebooks/mystmd/issues/670).
:::
Loading
Loading