Material 5.0.0 Status

[squidfunk]

TL;DR This issue is meant for updates and discussions on the progress of the rework of the theme. Please feel free to checkout and try the refactor/rxjs-typescript branch and comment on the current state of development.

Description

The last major releases of Material for MkDocs were mainly issued for compatibility reasons. While version 2 and 3 were related to backward incompatible changes of MkDocs (0.17.1 and 1.0 respectively), version 4 was released due to backward incompatible changes related to users with Chinese system languages (#911). The next iteration, version 5, is a substantial rewrite of the underlying JavaScript functionality to a TypeScript and RxJS-based architecture. This will yield the following major benefits:

1. All features that are provided by the theme (e.g. the sidebars, tabs, search, etc.) will be observable by 3rd party JavaScript. This allows for very easy extension and customization. This will address issues like Add 'selected' as toc item state #1102 where users want to extend the functionality of the theme without re-compiling or forking it. Also, disabling functionality should be easier.

2. Improving search performance. Search will be moved to a web worker and exhibit better caching and re-building behavior. This will mitigate problems were the UI freezes due to large search indexes which pre-building on the server-side only partly solves. Furthermore, search will be re-architected to fit into the plugin concept introduced by MkDocs 1.x.

3. Color customization will be re-implemented using CSS variables. Changing the color to your custom brand colors will be even simpler and the theme will become smaller (as the palette CSS file will not be necessary anymore). Support for CSS variables is pretty great by now, but there are some browsers that don't support it, respectively IE 11, Opera mini, QQ browser and Baidu browser. These browsers will receive a (probably neutral) fallback color. If you must support those browsers, you can still extend the theme and customize the build will custom CSS colors as already described in the customization guide.

There might also be some slight face lifting, but the main aspects of Material 5 is the introduction of a more modern architecture. The rewrite will also make the theme testable, so unit tests can and will be written to ensure functionality.

I will document my progress here and there will be a beta phase similar to before the release of 1.0.0 (#46). Some lists with things that need to be done will follow.

Evaluate

• Consider flexsearch as an alternative to lunr.js

TODOs

5.0

General

• Move everything except JavaScript compilation out of Webpack into Makefile
• Rewrite Webpack configuration in TypeScript
• Setup TypeScript project structure
• Implement experimental instant loading
• Add a plugin section to the docs
• Rewrite all components to new architecture
• Check browser support before release

Functionality

General

• Polyfill <details>/<summary> functionality
• Wrap all data tables for better overflow scrolling
• Automatically open <details> after child-anchor jump
• Automatically open <details> before printing
• Automatically close drawer on anchor jump
• General keyboard handlers
• Clipboard.js integration
• Cleanup code block CSS

Scroll-related

• Toggle header shadow on scroll
• Toggle header title swap on scroll
• Toggle hero visibility on scroll
• Toggle tabs visibility on scroll

Sidebars

• Position sidebar with navigation
• Position sidebar with table of contents
• Blur links in table of contents
• Apply scrollfix for active navigation for iOS
• Collapse/expand nested navigation

Search

• Move search into web worker (currently w/o fallback, do we need one?)
• Persist search index in local storage see Material 5.0.0 Beta 1 #1465 (comment)
• Compress search index see Material 5.0.0 Beta 1 #1465 (comment)
• Focus input after form reset
• Focus input after opening search
• Focus query if search is open and character is typed
• Add (multi-)language support
• Search keyboard handlers
• Fix margin / spacing of search bar if no source repository is given
• Lock body for active search on mobile

Accessibility

• Handle tabbing context for better accessibility
• Reset tabbing behavior

Repository

• Make repository icon configurable
• Retrieve facts for the given repository
• Render repository information

Issues

• Add 'selected' as toc item state (Add 'selected' as toc item state #1102)
• Update Font Awesome to LATEST (Update Font Awesome to 5.0 – breaking change #756)
• Fix rendering issues in table of contents (Table of Contents is blurry #1292)
• Fixed meta title and description only rendering first character bug (Fixed meta title and description only rendering first character #1324)
• Add announcement bar (Feature request: add an annoucement bar #1190)
• Update Material icons to LATEST (Upgrade Material Design icons to latest version #1174)
• Add missing background-color definition on root (If you set either foreground or background color by default, please also set the other #1418)
• Fix error on clipboard in combination with superfences (Adding superfences removes copy to clipboard #1440)
• Refactor color customization to use CSS variables (Introduce CSS variables to make color customization more flexible #1297)
• Fix navigation CTRL-click issues ([Bug] [UI] CTRL + Click of Navigation Group causes overlap #1177)
• Decide on main footer block (Add main footer block #1351)

5.x

• Add dark mode (Dark mode CSS file #1305)
• Add dark mode automatism via prefers-color-scheme (Color schemes: choose scheme based on user preference #1266)

[wilhelmer]

• Provide a link to donate to @squidfunk so he can order pizza and have more time for coding

[squidfunk]

@wilhelmer nice idea 😁 I actually already thought about it. However, listening to the latest React Podcast with Kitze I learned that having a Donate button will probably not help you fund the work of your project. People don’t tend to donate for something that they can already use for free which is why I decided not to include one.

[wilhelmer]

Well you‘d get at least 1 pizza, better than nothing 😄

[facelessuser]

1 🍕 > 0 🍕

Yup, checks out.

[squidfunk]

I started to document my progress regarding the rewrite of the current features in the original post.

Furthermore I think I finally got the grasp of RxJS and boy, is it beautiful. If somebody has some experience with RxJS, I would really appreciate some comments on the architecture I'm building on the refactor/rxjs-typescript branch. See this folder.

After migrating everything to RxJS, implementing instant loading (aka "turbolinks" which I prototyped three years (!) ago here) will be a breeze, as re-binding components will be super easy. This would also greatly improve search UX, as the page will more or less behave like an SPA, and thus the search index has to be built only once.

[squidfunk]

I took the time and implemented instant loading as can be seen in the latest version of refactor/rxjs-typescript. All internal links that are not anchors (i.e. #...) are intercepted and dispatched via XHR. Then, the new components are extracted from the parsed response, replaced in the DOM and all downstream observables/listeners are re-bound.

Works pretty well already and it's super fast! v5 will add instant loading as an experimental feature similar to tabs. Some open questions remain, e.g. what to do with Google Analytics, Disqus and custom JavaScript but it should all be solvable.

[wilhelmer]

I had to look up what instant loading is, but now that I know: Sounds great! 👏🏻

[squidfunk]

@wilhelmer don’t really know what to call it. There’s this JavaScript library called Turbolinks which does something similar but that does sound very Wordpressy.

[Stanzilla]

Do you want tickets for v5 bugs yet?

I only found two small problems so far:

• https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/base.html#L422 this will cause a console error, needs src="{{ 'assets/javascripts/app.js' | url }}"
• I think https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/assets/javascripts/index.ts#L92 should be as string[] and then https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/assets/javascripts/index.ts#L132 names as [] just started learning TS though.

[squidfunk]

@Stanzilla thanks for checking it out already. For now just post your findings here. If we hit beta / release candidate, we can open a separate issue and maybe project to keep track of things without flooding the issue tracker.

https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/base.html#L422 this will cause a console error, needs src="{{ 'assets/javascripts/app.js' | url }}"

What do you mean exactly? It's dependent on the app.js script, that's correct. Same behavior as before, except everything exported by the script is exported into the global variable app. I think of probably exporting all watch* functions into the global namespace but haven't decided yet.

I think https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/assets/javascripts/index.ts#L92 should be as string[] and then > https://github.com/squidfunk/mkdocs-material/blob/refactor/rxjs-typescript/src/assets/javascripts/index.ts#L132 names as [] just started learning TS though.

Thanks for catching them. Typings are ongoing :-) The component union type must be put somewhere else, will fix the typings then. Maybe don't focus on the code base (e.g. typing errors) right now but on the functionality because I sometimes commit stuff that is not quite ready yet / needs work, but the branch is WIP anyway.

How does the instant loading work for you? I think it works pretty great already, though a little jittery when skimming through the browser history. Still trying to map out how to smoothen things out.

[Stanzilla]

Alright! And I was getting some "app is not defined" errors before but I just checked again and they are gone, sorry!

Functionality is pretty nice atm, I haven't checked which of my workarounds from v4 I can drop, I made some to workaround the collapse issue for examples. Also I edited the partial so I can have external links open in a new window instead, that currently causes a few JS errors in the console. Is that something you would like to support natively in the future? Maybe as a parameter from mkdocs.yml?

[squidfunk]

Functionality is pretty nice atm, I haven't checked which of my workarounds from v4 I can drop, I made some to workaround the collapse issue for examples.

The collapse issue (will) be fixed completely as we'll now listen on the change event and not click like in v4.

Also I edited the partial so I can have external links open in a new window instead, that currently causes a few JS errors in the console.

Good to know, will look into it. CMD + Click should not be intercepted but open in a new tab as expected.

Is that something you would like to support natively in the future? Maybe as a parameter from mkdocs.yml?

Nope, we removed it for UX reasons. It's however super easy to add the behavior in v5 via custom JavaScript that runs automatically, probably something like:

merge(load$, switch$)
  .subscribe(document => {
    document.querySelectorAll("a").forEach(el => {
      el.setAttribute("target", "_blank")
    })
  })

... whereas load$ (initial render) and switch$ (subsequent render) are exposed by the application.

[Stanzilla]

That's slightly less ugly than my

{% if nav_item.title == "A" or nav_item.title == "B" %}
    <a
      href="{{ nav_item.url | url }}"
      title="{{ nav_item.title | striptags }}"
      class="md-nav__link" target="_blank"
    >
    {% else %}
    <a
      href="{{ nav_item.url | url }}"
      title="{{ nav_item.title | striptags }}"
      class="md-nav__link"
    >
    {% endif %}
      {{ nav_item.title }}
    </a>

:D

Oh and I had to add the webpack copy plugin back for a custom JS file.

[squidfunk]

I moved everything that is unnecessary out of Webpack into the Makefile for much faster subsequent builds, as the Makefile correctly tracks dependencies and does not recompile everything upon a single change. Also copying, removing and replacing stuff is now entirely handled by the good ol' friends of UNIX.

[Stanzilla]

Just thought about the external links thing again, would still love to see that come back as an option because I'd still have to flag certain links with manual edits in the template for your JS solution to work properly.

[squidfunk]

The only external links that are added directly through Material are those in the footer and the source link. The Navigation is entirely local, so it should open in the same window. Furthermore, opening something in a new window is bad UX, as it's not transparent to the user. For this reason I can definitely say it's not coming back, I'm sorry.

[Stanzilla]

Yeah I just want to open social links and the edit one in a new window

[markallasread]

* [ ]  Provide a link to donate to @squidfunk so he can order pizza and have more time for coding

Not-so-much that there'll be more time for coding, just that we can compensate you for your contributions. I don't expect more by donating, I already received everything I need! :)

[squidfunk]

@wilhelmer @markallasread I started an Amazon wish list – in case somebody wants to give something back. 😊

[outofphase]

@squidfunk the wish list link takes me to amazon.com...are you in the US?

[squidfunk]

@outofphase nope, Germany. You're correct. I'll fix it and migrate it to Amazon Germany.

[squidfunk]

Migrated the lis to my German account and fixed the link above, can you check again?

[outofphase]

Yes that's worked, although not surprisingly everything is now in German which made navigating through the Amazon maze a little tricky for me. I really hope that I haven't just started a 30 day free trial of Amazon Prime. With luck you should get the book by the XKCD guy on Weds.

I think that one of the Amazon prompts was saying "buying from somewhere else?" which I should have clicked, but that also was in German so I wasn't sure. You should probably give guidance about that for non-German speakers.

[squidfunk]

I started a migration guide for v5, documenting the changes that are necessary to the templates and partials if the user has overridden them in their build here. Those changes are crucial for the new (and more future-proof) application structure. I will also add another section on configuration changes and customization such as colors etc. In general the overall structure did not change much, especially custom CSS should continue to work in 99% of the cases.

I'm also hoping to finish the missing features within the next days, so we can put out a fully working RC. Sorry for the delay, quite hard to find some spare time at the moment.

[Stanzilla]

For me, the build process changes were/are the biggest hurdle, might be worth mentioning as well!

[squidfunk]

@Stanzilla What specifically would be worth mentioning? Would you mind writing down the hurdles you experienced?

[Stanzilla]

The move from webpack to the makefile, I have a few custom things in the webpack config and the new structure was a bit confusing to me, also probably less Windows friendly. I also sometimes have to npm clean before I can build because it does not see changes even though I did them

[squidfunk]

If you could isolate what files do not get rebuilt when changed we can fix the Makefile

[Stanzilla]

If you could isolate what files do not get rebuilt when changed we can fix the Makefile

I just remembered, webpack config changes. If I do build, change something webpack.config.ts and then build again, it does nothing.

[squidfunk]

@Stanzilla should be fixed with 9b9527f – but I'm pretty sure that Webpack config changes are also not picked up on master.

[squidfunk]

Small update: took the afternoon and finished the keyboard handlers (general and search-related) and copy-to-clipboard integration. Needs some cleanup (the main index.ts is a mess), but should be functionally working. For the clipboard message we'll switch to a snackbar/toast design, deprecating the message next to the button. Furthermore the clipboard buttons now have a proper outline on focus which is better for accessibility. I'll cleanup the main index.ts when the RC is out and all functionality has been implemented.

[squidfunk]

... and we're back to Webpack as of b486d0b. I ditched the whole Makefile-based approach. For a project the size of Material compatibility is an important issue and I ran into problems with asset versioning etc. where I would've had to build a lot of stuff from scratch. It just makes no sense. Sorry for any confusion this has caused with forks.

[Stanzilla]

Yay!

[squidfunk]

Small facelifting of Admonitions in a018ed0:

Before:

After:

[squidfunk]

Making progress, most stuff is functional. I'll try hard to push out an RC in the middle of next week! The small facelift will also include a new logo:

[facelessuser]

Looks nice!

[squidfunk]

Fixed a lot of stuff today including some long standing errors. @facelessuser you're the only one I know of who is running the dev build in production. Could I ask you to update to the latest version to see if something breaks?

Bugs fixed in refactoring branch:

• Fixed anchor jump from mobile search – a399540
• Fixed search body lock on mobile – a399540

Bugs fixed which are still in current master:

• Fixed header shadow disappearing on iOS Safari – a539eab, 694eaca
• Fixed lines wrapping in code blocks on iOS Safari and Safari – a6b22a4
• Fixed permalink characters (i.e. ¶) being part of textContent. This should hopefully solve the problem that some search engines index them and add them to the search summary – ca260e0

New features not in master:

• Added scroll snapping for navigation and toc – 67406d2
• Added scroll snapping for search – 9fe7c62
• Revamped code block line number styles – a6b22a4

[facelessuser]

Yeah, I'll try it out and get back to you later today or tomorrow.

[facelessuser]

@squidfunk, seems to be working for me. Nothing obvious seems broken.

[squidfunk]

Thanks for your support. I‘m not sure what to do now – if I have to fork the theme anyway for Chrome support, the localsearch plugin is pointless because I‘ll be better off integrating all necessary modifications in the fork. I need to think about that ...

@wilhelmer I made some steps towards a solution for making localsearch work with v5 in fdffefd. It's now possible to pass a Promise resolving with the contents of search_index.json which would allow for the technique we discussed before: inlining the search_index.json into the template via a meta tag, retrieving it from there and passing it explicitly.

What remains is making the Worker work locally. I tried pseudo-worker and it would work if it didn't make an XHR request to retrieve and eval the worker script. However, when I replace the Worker with PseudoWorker in a hosted non-local context, it works perfectly in the main thread.

Thus, inlining the search_index.json and mimicking the API of Worker are IMHO the only things necessary to make it work. We can probably tackle that after the final release of v5. To mimic the worker API, we would need to implement addEventListener, postMessage and importScripts. We could probably do this by wrapping the Worker constructor and implement the following logic:

1. Add a new script to the end of document.body which makes a copy of self (which equals window in the main thread), so we can restore it later. Furthermore, overwrite self with our custom implementations of the three functions mentioned above. Basically, we only need to hook up the function passed to addEventListener to postMessage and we should be good to go.

2. Now add another new script with the URL of the web worker. It will hook into our new self pseudo worker and setup everything. We could probably just use the pseudo-worker implementation. For all scripts passed to importScripts, add another synchronously script to the DOM to mimic the behavior.

3. Add another script which restores self to window. Then, after evaluating all scripts, it's probably safe to remove them all.

[wilhelmer]

Thanks for your effort! I created #1464 just while you typed this 😄

I hope we can continue the discussion there. I won't be able to do the coding myself, but my colleague will join in.

[squidfunk]

Just released Material 5.0.0b1. See #1465. Would be awesome to get some feedback, especially on the brand new search. Please report any problems, issues or incompatibilities in the linked issue.

[squidfunk]

Just released Material 5.0.0b2. See #1469. Again, would be great to get some feedback, and finally a new feature - instant loading!

[squidfunk]

Besides a brand new logo, v5 will include a new default (fav)icon to better reflect the core values of Material: Documentation for people

[mnogue]

Much better than the other one. I love it!

[squidfunk]

@mnogue it's only the default icon, though 😊Changeable with a single line of code. The official docs will however include the new logo I posted earlier.

[squidfunk]

🎉Beta 3 just landed in #1483 and since we're feature complete, I'm closing this issue. I would encourage everybody to test the latest beta, as it's the last one before the final RC. There's still some cleaning up to do, but there's not much to change after beta 3. The more people test it, the earlier we can roll out a release candidate and finally get v5 out of the door!