Japanese search is broken after v4.0.1

[celestialphineas]

Description

The Japanese/Chinese search is no longer working for mkdocs-material v4.0.2. But the behaviors are as expected for v4.0.1 and earlier versions.

I believe this unexpected behavior is related to the update of lunr-languages, or some DOM behavior changes of the search box. But I have not figured it out yet.

Expected behavior

Actual behavior

And strangely the hint is saying "Type to start searching", not "No matching document".

Steps to reproduce the bug

1. Update your mkdocs-material to v4.0.2
2. It starts to fail working as expected.

Package versions

• Python: 3.7.1
• MkDocs: 1.0.4
• Material: 4.0.2

Project configuration

theme:
  name: 'material'
  language: 'zh'
  palette:
    primary: 'red'
    accent:  'red'
  font:
    text: "Roboto"
extra:
  search:
    language: 'en, jp'

System information

• OS: Deepin 15.9.1 Desktop x64
• Browser: Chrome 70.0.3538.77

[squidfunk]

Thanks for reporting and narrowing it down! Will look into it ASAP!

[squidfunk]

I tried to reproduce the behavior you reported, but it works fine for me. I tested with mkdocs-material-4.0.2 installed via pip in a clean venv, this is the result:

Japanese (jp):

Chinese (zh):

EDIT: Are there errors shown in the browser's console?

[celestialphineas]

There are no errors, but it simply won't work for me to use v4.0.2

celestialphineas@celestialphineas:/media/celestialphineas/36303C04303BCA1F/development/ckcers-tutorial$ mkdocs serve
INFO    -  Building documentation... 
INFO    -  Cleaning site directory 
[I 190310 13:23:38 server:298] Serving on http://127.0.0.1:8000
[I 190310 13:23:38 handlers:59] Start watching changes
[I 190310 13:23:38 handlers:61] Start detecting changes
[I 190310 13:23:47 handlers:132] Browser Connected: http://localhost:8000/

And everything goes well for v4.0.1 and versions earlier.

If you don't mind, you can check out my repository below. Hopefully it will reproduce the problem.

celestialphineas/ckcers-tutorial

[celestialphineas]

celestialphineas@celestialphineas:/media/celestialphineas/36303C04303BCA1F/development/ckcers-tutorial$ pip uninstall mkdocs-material
Uninstalling mkdocs-material-4.0.2:
  Would remove:
    /home/celestialphineas/anaconda3/lib/python3.7/site-packages/material/*
    /home/celestialphineas/anaconda3/lib/python3.7/site-packages/mkdocs_material-4.0.2.dist-info/*
    /home/celestialphineas/anaconda3/lib/python3.7/site-packages/src/*
Proceed (y/n)? y
  Successfully uninstalled mkdocs-material-4.0.2
celestialphineas@celestialphineas:/media/celestialphineas/36303C04303BCA1F/development/ckcers-tutorial$ pip install mkdocs-material==4.0.1
Collecting mkdocs-material==4.0.1
  Using cached 

...Resolving requirements

/home/celestialphineas/anaconda3/lib/python3.7/site-packages (from Jinja2>=2.7.1->mkdocs>=1->mkdocs-material==4.0.1) (1.1.1)
Installing collected packages: mkdocs-material
Successfully installed mkdocs-material-4.0.1
celestialphineas@celestialphineas:/media/celestialphineas/36303C04303BCA1F/development/ckcers-tutorial$ mkdocs serve
INFO    -  Building documentation... 
INFO    -  Cleaning site directory 
[I 190310 13:28:33 server:298] Serving on http://127.0.0.1:8000
[I 190310 13:28:33 handlers:59] Start watching changes
[I 190310 13:28:33 handlers:61] Start detecting changes
[I 190310 13:28:43 handlers:132] Browser Connected: http://localhost:8000/

And the search turns out to work quite well after the reinstalling.

[squidfunk]

I’m sorry but I don’t have he time to debug individual repositories. My tests have shown that the package works exactly as intended, so the error is most likely within your setup. Could you try running it using the Docker container?

[celestialphineas]

Browser console errors

There appears to be errors in the browser console:

Uncaught ReferenceError: module is not defined
    at lunr.jp.js:1

Uncaught (in promise) TypeError: Cannot read property 'wordCharacters' of undefined
    at Function.o.multiLanguage (lunr.multi.js:1)
    at D.Builder.<anonymous> (application.d9aa80ab.js:1)
    at D (application.d9aa80ab.js:6)
    at l (application.d9aa80ab.js:1)

In Firefox 60.0.1 64-bit:

ReferenceError: module is not defined
lunr.jp.js:1:1
TypeError: o[e[s]] is undefined
lunr.multi.js:1:366
TypeError: o[e[s]] is undefined
lunr.multi.js:1:366
TypeError: o[e[s]] is undefined
lunr.multi.js:1:366
TypeError: o[e[s]] is undefined

File lunr.jp.js:

module.exports=require("./lunr.ja");

Not a configuration problem of python

I ran the whole project in a docker container and the problem occurs as well.

FROM python:3.7.1
WORKDIR /ckcers-tutorial
RUN git clone https://github.com/celestialphineas/ckcers-tutorial /ckcers-tutorial
RUN pip install mkdocs mkdocs-material python-markdown-math
CMD mkdocs serve

$ docker build -t jp-rep .
$ docker run -it --network host jp-rep

A few remarks on my repo

• It's basically a whole bunch of markdown documents and images, with no logics.
• The only configuration file on the master branch is mkdocs.yml.
• I use mdx_math to render equations.
• A few markdown files are ill-formed, but it is not bothering anything though.

Here is a list of extensions I am using:

extra_javascript: 
  - https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML
markdown_extensions:
  - admonition
  - codehilite
  - footnotes
  - meta
  - toc:
      permalink: true
  - pymdownx.arithmatex
  - pymdownx.betterem:
      smart_enable: all
  - pymdownx.caret
  - pymdownx.critic
  - pymdownx.emoji:
      emoji_generator: !!python/name:pymdownx.emoji.to_svg
  - pymdownx.inlinehilite
  - pymdownx.magiclink
  - pymdownx.mark
  - pymdownx.smartsymbols
  - pymdownx.superfences
  - pymdownx.tasklist:
      custom_checkbox: true
  - pymdownx.tilde
  - mdx_math

Additional info

• It does not help by changing jp to ja.
• Things go well with the English stemmer only, except that I cannot search for Chinese/Japanese keywords anymore.

[squidfunk]

Thanks for your research - the error in the console is a good indicator of what's going wrong. I will look into it.

[squidfunk]

Regarding Docker: I was asking whether you could try the official MkDocs Material Docker image, see https://hub.docker.com/r/squidfunk/mkdocs-material/

[celestialphineas]

I can still reproduce this problem in the official MkDocs Material Docker image.

Dockerfile

FROM squidfunk/mkdocs-material:latest
WORKDIR /ckcers-tutorial
RUN git clone https://github.com/celestialphineas/ckcers-tutorial /ckcers-tutorial
RUN pip install python-markdown-math

Command lines

$ docker run --rm -it --network host jp-rep serve
INFO    -  Building documentation... 
INFO    -  Cleaning site directory 
[I 190311 13:13:04 server:298] Serving on http://127.0.0.1:8000
[I 190311 13:13:04 handlers:59] Start watching changes
[I 190311 13:13:04 handlers:61] Start detecting changes
[I 190311 13:13:08 handlers:132] Browser Connected: http://localhost:8000/

In Chrome

Uncaught ReferenceError: module is not defined
    at lunr.jp.js:1
lunr.multi.js:1 Uncaught (in promise) TypeError: Cannot read property 'wordCharacters' of undefined
    at Function.o.multiLanguage (lunr.multi.js:1)
    at D.Builder.<anonymous> (application.d9aa80ab.js:1)
    at D (application.d9aa80ab.js:6)
    at l (application.d9aa80ab.js:1)
2lunr.multi.js:1 Uncaught (in promise) TypeError: Cannot read property 'wordCharacters' of undefined
    at Function.o.multiLanguage (lunr.multi.js:1)
    at D.Builder.<anonymous> (application.d9aa80ab.js:1)
    at D (application.d9aa80ab.js:6)
    at l (application.d9aa80ab.js:1)

[celestialphineas]

It is not related to python-markdown-math either. I ran the original official docker with no math configuration and it is still not working. So the problem might be with some other markdown extensions.

$ docker run --rm -it --network host -v /media/celestialphineas/36303C04303BCA1F/development/ckcers-tutorial:/docs squidfunk/mkdocs-material serve
INFO    -  Building documentation... 
INFO    -  Cleaning site directory 
[I 190311 13:23:25 server:298] Serving on http://127.0.0.1:8000
[I 190311 13:23:25 handlers:59] Start watching changes
[I 190311 13:23:25 handlers:61] Start detecting changes
[I 190311 13:23:30 handlers:132] Browser Connected: http://localhost:8000/

Diff of the extention configuration:

markdown_extensions:
  - admonition
  - codehilite
  - footnotes
  - meta
  - toc:
      permalink: true
  - pymdownx.arithmatex
  - pymdownx.betterem:
      smart_enable: all
  - pymdownx.caret
  - pymdownx.critic
  - pymdownx.emoji:
      emoji_generator: !!python/name:pymdownx.emoji.to_svg
  - pymdownx.inlinehilite
  - pymdownx.magiclink
  - pymdownx.mark
  - pymdownx.smartsymbols
  - pymdownx.superfences
  - pymdownx.tasklist:
      custom_checkbox: true
  - pymdownx.tilde
  # - mdx_math

[squidfunk]

I was able to finally reproduce the issue. A potential fix is included with commit 9a16b9c in master. Could you try the latest master and check whether the issue persists?

[celestialphineas]

I don't know if I am building the master branch correctly in a docker container. Is there an image to use? Here is my dockerfile:

FROM python:2.7
WORKDIR /tmp
RUN rm -rf /tmp/*
RUN git clone https://github.com/squidfunk/mkdocs-material /tmp
RUN pip install -r /tmp/requirements.txt
RUN pip install python-markdown-math
RUN python /tmp/setup.py install
RUN git clone https://github.com/celestialphineas/ckcers-tutorial /ckcers-tutorial
WORKDIR /ckcers-tutorial
RUN rm -rf /tmp
EXPOSE 8000
CMD mkdocs serve

The building process is smooth though.

Command line:

$ docker run --rm -it --network host mkdocs-material

And it does not build the site in the right way, with some .js scripts missing.

Browser console

2localhost/:887 GET http://localhost:8000/assets/javascripts/application.51c3f73e.js net::ERR_ABORTED 404 (Not Found)
lunr.stemmer.support.js:1 Uncaught TypeError: Cannot set property 'stemmerSupport' of undefined
    at lunr.stemmer.support.js:1
    at lunr.stemmer.support.js:1
    at lunr.stemmer.support.js:1
(anonymous) @ lunr.stemmer.support.js:1
(anonymous) @ lunr.stemmer.support.js:1
(anonymous) @ lunr.stemmer.support.js:1
lunr.multi.js:1 Uncaught TypeError: Cannot set property 'multiLanguage' of undefined
    at lunr.multi.js:1
    at lunr.multi.js:1
    at lunr.multi.js:1
(anonymous) @ lunr.multi.js:1
(anonymous) @ lunr.multi.js:1
(anonymous) @ lunr.multi.js:1
(index):906 Uncaught ReferenceError: app is not defined
    at (index):906

Terminal

INFO    -  Building documentation... 
INFO    -  Cleaning site directory 
[I 190313 04:03:37 server:298] Serving on http://127.0.0.1:8000
[I 190313 04:03:37 handlers:59] Start watching changes
[I 190313 04:03:37 handlers:61] Start detecting changes
[W 190313 04:03:40 web:2162] 404 GET /assets/javascripts/application.51c3f73e.js (127.0.0.1) 2.99ms
[W 190313 04:03:40 web:2162] 404 GET /assets/javascripts/application.51c3f73e.js (127.0.0.1) 2.95ms
[I 190313 04:03:40 handlers:132] Browser Connected: http://localhost:8000/

[squidfunk]

I probably forgot to mention that you have to change from en, jp to en, ja. ja is the official language code for Japanese, lunr-languages switched from jp to ja and provided an upgrade path still supporting jp as an alias, but it would be too costly to support both in Material. Therefore only ja is supported from now on. Does changing to ja fix your problem?

[celestialphineas]

I'm afraid I was building mkdocs-material in an improper way. Some JS scripts are missing when running mkdocs serve (see my comment above).

Thank you so much for the bug fix, Martin. If you find it is working alright, just draft new release and I'll check it out in the new release right away.