Whitespace in code receives underline for some themes, which can be confused with a _ character

[abravalheri]

Hello, let me get this opportunity to first thank the authors and maintainers for the amazing package.
I also would like to apologize in advance if I am submitting the issue to the wrong repository.

Recently on setuptools, we received a bug report regarding our documentation: pypa/setuptools#2999.

It seems that after the latest versions of pygments got released the default dark theme used in Sphinx for pygments started showing underlined whitespace, which is confusing for users (as reported in the linked issue):

After some quick investigation:

I think the idea here is that the colour of the whitespace (.w class, currently #666666) should be the same as the background (#202020), but it ended up being misconfigured - at least that is what happens in the light version of the theme (no idea why it is necessary to preserve the underline configuration instead of just removing it).

However I could not find out if this theme is something Sphinx itself is providing (in that case it would be better to open an issue there, right?), or if it comes directly from pygments.

My hypothesis is that this is a consequence of the whitespace improvements introduced in 2.11.0.

Do you guys have any advice/information about this?

[Anteru]

First: Thanks :) Happy to hear that you like Pygments!

I'd suspect it's the Sphinx theme unless they're using a built-in theme. @blu-base has gone through dozens (hundreds?) of lexers to correctly highlight whitespace, which was previously hit and miss, and I'm not surprised if some themes have incorrect styling for whitespace. As far as I can tell (on a quick glance) we don't use underline in our styles for whitespace, so I'd assume it's Sphinx, but it would be good to know what style they're actually using to confirm that.

[amitkummer]

Looks like setuptools uses the furo sphinx theme:

https://github.com/pypa/setuptools/blob/6092cdcbd659110e613e9841cdd5a0c43e61f33b/docs/conf.py#L110

Which in turn uses the tango style, which comes with pygments:

https://github.com/pradyunsg/furo/blob/e6809a2e80138c2a7f2b4f073bcec9c1d652e65a/src/furo/theme/furo/theme.conf#L4

And the tango style introduces underline for whitespace:

pygments/pygments/styles/tango.py

Line 58 in 4c21628

Whitespace: "underline #f8f8f8", # class: 'w'

I am pretty certain that the unwanted underline CSS rule in the example screenshot you provided, come from the above line (I successfully reproduced the underlines with the tango theme).

I am not sure why whitespace is underlined in the tango theme. I don't see any other style doing it, and the underline is not visible anyway because it has the same color as the background. I think the correct decision here is to remove the underline from the tango style.

[Anteru]

Agreed. Also, gone now. Thanks @amitkummer for the investigation and sorry I collided with your PR :(

[amitkummer]

Haha, no worries, it was just a couple of minutes work :)

[abravalheri]

Thank you very much! You guys are ninjas!

I had tried to look in furo to find out which theme it was using, but I did not manage to... So I was (wrongly) assuming it was something provided by default by Sphinx.

[Anteru]

... and I did search for underline in the source code and missed that one usage in Tango -.-

I'll prepare a 2.11.2 release tomorrow to resolve this, given I use the Furo style myself :).

[Anteru]

@abravalheri Pygments 2.11.2 with the fix included is available now.