Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.1.0 problem with extending the lexers dictionary #6497

Closed
jmckenna opened this issue Jun 17, 2019 · 9 comments

Comments

@jmckenna
Copy link

commented Jun 17, 2019

We extend the lexers dictionary (this has worked since around 2008) such as:

  def setup(app):
      from sphinx.highlighting import lexers
      lexers['myalias'] = MyLexer()

which since the 2.1.0 release causes these warnings and doesn't trigger the highlighting:

  WARNING: Could not lex literal_block as "myalias".  Highlighting skipped.
@tk0miya

This comment has been minimized.

Copy link
Member

commented Jun 17, 2019

Thank you for reporting.
At 2.1.0, sphinx makes custom lexers sensitive to syntax errors. In detail, it automatically installs raiseonerror filter to the lexers. That is not intentional change. So I'll fix it in next release.

@jmckenna

This comment has been minimized.

Copy link
Author

commented Jun 17, 2019

Thank you, I will test the next release. We can close this issue for now.

tk0miya added a commit that referenced this issue Jun 18, 2019

Merge pull request #6502 from tk0miya/6497_custom_lexers_fails_highli…
…ghting

Fix #6497: custom lexers fails highlighting when syntax error
@tk0miya

This comment has been minimized.

Copy link
Member

commented Jun 18, 2019

Fixed by #6502.
I'll release 2.1.2 within a few days. Please wait until then.
Thanks,

@tk0miya tk0miya closed this Jun 18, 2019

@jmckenna

This comment has been minimized.

Copy link
Author

commented Jun 20, 2019

Verified the fix with version 2.1.2 Great, thanks!

@tk0miya

This comment has been minimized.

Copy link
Member

commented Jun 22, 2019

@jmckenna Good to know.

BTW, since v0.6, we've provided app.add_lexer() API for this purpose. To make extensions less coupled (not to touch module global variable directly), I recommend you to use it to install custom lexers.
Thanks,

@jmckenna

This comment has been minimized.

Copy link
Author

commented Jun 22, 2019

So would the syntax be something like the following:

app.add_lexer('mapfile', MapFileLexer())

where MapFileLexer() is the instance and 'mapfile' is the alias. Is this syntax correct for >2.1.0 ?

@tk0miya

This comment has been minimized.

Copy link
Member

commented Jun 22, 2019

Yes, it working now.
for Sphinx-2.1.0 or newer, it takes lexer class as 2nd argument.
https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx.application.Sphinx.add_lexer

So it would be nice if possible:

import sphinx

def setup(app):
    if sphinx.version_info >= (2, 1):
        app.add_lexer('mapfile', MapFileLexer)
    else:
        app.add_lexer('mapfile', MapFileLexer())

Since 2.1, Sphinx provides :force: option for some code directives (code-block, literalinclude and so on). The class style add_lexer() API allows you to use it on custom lexers.

@jmckenna

This comment has been minimized.

Copy link
Author

commented Jun 23, 2019

With 2.1.2, if I try:

app.add_lexer('mapfile', MapFileLexer)

I get the same error:

WARNING: Could not lex literal_block as "mapfile". Highlighting skipped.

But it works if I call it by instance instead:

app.add_lexer('mapfile', MapFileLexer())
@tk0miya

This comment has been minimized.

Copy link
Member

commented Jun 29, 2019

Oh, sorry. I did not describe the API in detail. A lexer registered as a class raises an error on syntax error. On the other hand, A lexer registered as an instance ignores syntax error. You can choose either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.