Native syntax highlighting in dcc.Markdown

[wbrgss]

This PR removes dcc.SyntaxHighlighter in favor of Markdown-flavoured code syntax.

Example usage (EDIT: import syntax has changed; see below for discussion):

import dash_core_components as dcc

...

    dcc.Markdown(dedent(
    '''
    # Markdown title

    ## Markdown subtitle — highlighted syntax below

    ``` python
    # Describe the layout, or the UI, of the app
    app.layout = html.Div([
        html.H3('Hello World'),
        dcc.Dropdown(
            id='my-dropdown',
            options=[
                {'label': 'Coke', 'value': 'COKE'},
                {'label': 'Tesla', 'value': 'TSLA'},
                {'label': 'Apple', 'value': 'AAPL'}
            ],
            value='COKE'
        ),
        dcc.Graph(id='my-graph')
    ])
    ```
    '''
   # use the dark theme and add styles around the Markdown block
    ), highlight_config={'dark': True}, style={'border-left': '1px solid lightgray'}),

[wbrgss]

@alexcjohnson This isn't very robust, because it must be manually edited after a build. But I'm not sure how to change the auto-generated _imports_.py. I figured it might require a change to dash-generate-components. However, it may not be needed. The dependency, highlight.js, is much smaller than the now-unneeded react-syntax-highlighter.

[alexcjohnson]

Hmm yeah that's not great... also as it is, markdown will always be imported anyway so we haven't saved its bytes (aside from what we've already saved by swapping highlight.js in for react-syntax-highlighter). __init__.py isn't auto-generated, right? Seems like we can:

• leave _imports_.py and Markdown.py where they go automatically
• in __init__.py, filter out Markdown by changing the imports like:

import ._imports_ as _imports
__all__ = []
for _name in dir(_imports):
    if _name != 'Markdown' and not _name.startswith('_'):
        locals()[_name] = _imports[_name]
        __all__.append(_name)

• add a markdown.py that looks exactly like your markdown.__init__.py

Then, to use markdown.py and include its dist files, you have to do from dash_core_components.markdown import Markdown

One problem with this is if people make a mistake and import dash_core_components.Markdown it won't throw an error in python, it'll just fail to render. I suppose we could at least make the if (!window.hljs) block throw a clear error telling you how to import the component correctly? (same for react-markdown if we manage to pull that out too)

[chriddyp]

Similarly if people do dcc.markdown.Markdown('''...'''') after the app's start (e.g. inside a callback), then the dist won't be included right?

[alexcjohnson]

Right, that wouldn't work either. OK, so maybe it's premature to try and break this stuff out, too many ways to break it and I don't see a clean way around it. Let's file this under "should be solved by lazy loading" and put Markdown back on par with everything else - ie a regular top-level import whose js deps are always available.

[chriddyp]

👍

[wbrgss]

OK, I'll do this in a more conventional way and include react-markdown and highlight.js in the dcc _js_dist. If I do this, it will only add about 20kB. Before, with SyntaxHighlighter, it added 188kB from react-syntax-highlight, and 740kB (!) from highlight.js. Installing highlight.js via NPM is ~533kB 🙀.

The default build of highlight.js is 50kB with support for "24 commonly used languages". My custom build is only 12kB; IIRC I'm only including Python, CSS, Javascript and Markdown support. You can easily create a customized highlight.js here.

I'll suggest a highlight.js with the following languages supported (20kB minified):

• CSS
• HTTP
• JavaScript
• Python
• Bash
• JSON
• Markdown
• HTML, XML
• SQL (?)
• "Shell Session" (``` shell ... ```)

Obviously we can always add more later without introducing any breaking changes. Any languages I'm missing?

[chriddyp]

R

[alexcjohnson]

Oh that's awesome! I'd just add TypeScript, and yes let's keep SQL.

At some point we can make some utilities to pick from a few bundle options for both highlight.js and plotly.js - including perhaps an option to turn both of these off entirely and let you put your own in assets.

[alexcjohnson]

And in fact, if react-markdown is a single thing and isn't expected to be configurable, I'd leave it in the main dcc bundle, keep it off window

[wbrgss]

This could be more specific — so far it accepts just one parameter.

[alexcjohnson]

Good call to make this an object we can extend later, but yeah, let's make it exact and enumerate its contents. Also:

• rather than dark: false | true let's do theme: 'light' | 'dark' so we can add more themes later if we choose.
• give it a defaultProps entry of {}, that'll simplify the logic a bit. (You're probably aware of this, but don't put nested values like {theme: 'light'} in a default, that leads to unpleasant consequences when the user overrides part but not all of the prop)

[wbrgss]

Closes #559

[alexcjohnson]

Can we get react-markdown into a separate run-from-window file too?
Edit: no, leave it, see #562 (comment), didn't realize that one was so small!

[alexcjohnson]

ooh that's too greedy (and potentially slow) - can we restrict it to this component?

[wbrgss]

Yeah, agreed. querySelectorAll is really slow. I changed it to use a React ref callback combined with getElementsByTagName.

[alexcjohnson]

Not sure exactly what we used customStyle for previously, but this one will behave a bit differently as it's applied to the container <div>, not the <pre>. Anyway seems reasonable to include it, but let's restrict the docstring to describing what the prop does now, and put a description of its relation to SyntaxHighlighter in our migration guide in the docs.

[wbrgss]

let's restrict the docstring to describing what the prop does now

OK, done. One thing to keep in mind is that customStyle was one level deeper before; i.e. it was applied to the <Markdown> child of the parent <div>, not the parent <div> itself, as it is now. The thing is, Markdown isn't really a container, more of a mutator. I think there are docs examples with e.g. style={'border-left': 'grey'}.

[alexcjohnson]

Beauty 💃

[alexcjohnson]

not sure why Percy didn't get any snapshots but whatever... merged.

[Luvideria]

I was using Syntax Highlighter before, I had

dcc.SyntaxHighlighter(id='click-data1', language='C++',showLineNumbers='true',theme='dark'),

That I cannot really replicate anymore.
I now use

dcc.Markdown(id='click-data1',highlight_config={'showLineNumbers':'true','theme':'dark','customStyles':{'showLineNumbers':'true'}})

and I wrap my text between back quotes but it does not recognize c++, it works with python though.
Also the line numbers don't show up.

In short:
Should I open another issue?
How do I highlight C++ syntax?
How do I display line numbers?

[byronz]

How do I highlight C++ syntax?

if you scroll back enough, I think it's not in the initial 20-ish list @wbrgss customized, but we can add C++ syntax if it makes sense.

How do I display line numbers?

I think it's simply not supported by highlightjs https://highlightjs.readthedocs.io/en/latest/line-numbers.html

[alexcjohnson]

Thanks @Luvideria - I wasn't sure this got much use outside of our own docs :)

Sounds like we need 2 new issues in fact:

1. Additional languages: are there others we should add by default? With the caveat that we don't want to include all available languages, that would be too big. So we should also document the process to make & use your own hljs build (https://highlightjs.org/download/#cdns) - also requires that we implement something like Support replacing resources dash#655
2. Re-implement line numbers as an option.

Are there any other features SyntaxHighlighter had that we should work to reintroduce with Markdown?

[damienrj]

Is there a full example available? I am trying to figure out why

app.layout = html.Div([
    cyto.Cytoscape(
        id='graph',
        layout={'name': 'dagre','rankDir': 'LR'},
        elements=elements,
        stylesheet=stylesheet,
        style={'width': '100%', 'height': '450px'}
    ),
    dcc.Markdown(id='tapNodeData-json',highlight_config={'theme':'dark'})
])


@app.callback(Output('tapNodeData-json', 'children'),
              [Input('graph', 'tapNodeData')])
def displayTapNodeData(data):
    

    return """
#### Dash and Markdown
``` python
x = 1
if x == 1:
    # indented four spaces
    print("x is 1.")

isn't working. Everything looks great except no syntax highlighting on the code block, and the dark theme doesn't seem to be doing anything. Other options like adding a border box did work. This is getting rendered in a Juypter notebook which might be the issue.

[wbrgss]

@damienrj Your code appears to work for me. You can run a demo online here, which shows that the dark mode, syntax highlighting, and callback works. Are you closing the triple quotes (""") and backticks (```)? Indeed, Jupyter could be the problem as well; I haven't tested it.

Code for the above demo (click arrow to expand)

import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_cytoscape as cyto
from dash.dependencies import Input, Output

import json

app = dash.Dash(__name__)
server = app.server
app.config.suppress_callback_exceptions = True

app.layout = html.Div([
    dcc.Markdown(id='tapNodeData-json',highlight_config={'theme':'dark'}),
    cyto.Cytoscape(
        id='graph',
        elements=[
            {'data': {'id': 'one', 'label': 'Node 1'}, 'position': {'x': 50, 'y': 50}},
            {'data': {'id': 'two', 'label': 'Node 2'}, 'position': {'x': 200, 'y': 200}},
            {'data': {'source': 'one', 'target': 'two','label': 'Node 1 to 2'}}
        ],
    ),
])


@app.callback(Output('tapNodeData-json', 'children'),
              [Input('graph', 'tapNodeData')])
def displayTapNodeData(data):
    return ["""
#### Dash and Markdown
``` python
x = 1
if x == 1:
    # indented four spaces
    print("x is 1.")
` ` ` # spaces to escape backticks for GitHub
# Node Data:
## Click on a node to display the relevant metadata
""",
"""
``` python
{}
` ` `
""".format(json.dumps(data))]

if __name__ == '__main__':
    app.run_server(host='0.0.0.0', debug=True)

Just FYI, in the future, if you wish to get help with your syntax or usage of Dash, please open a new topic in the Community Forum. If you think you've discovered a bug, please open a new issue in this repo. These are more suitable places to discuss this kind of problem than a merged PR.

[damienrj]

Thanks, I will do that in the future. I mistakenly posted in here to see if there was more documentation tied to the PR. I will probably go ahead and make a PR to include an example to show the syntax highlighting for general use. Will work on chasing down the issue separately from dash now, thanks again!