Generate Dash components at buildtime rather than runtime (#150)

[rmarren1]

This roughly follows the suggestions in (#150) to create dash components at build-time via importing from an intermediary python class file rather than at creating them all at runtime.

Summary of changes:

In dash/development/base_component.py

• Added a function generate_class_string which generates the intermediary class string which was previously created in generate_class. This does not change generate_class as it just moves some work to a subroutine.
• Added a function generate_class_file which prepends the proper import statement (from dash.development.base_component import Component) to the class string from generate_class_string and writes to file.

In dash/development/component_loader.py

• Moved the metadata loading step to a subroutine _get_metadata
• Added a function generate_classes which consumes a metadata.json at a filepath and writes all the classes within to file in an output directory. The filepath and output directory are 'lib/metadata.json' and 'lib' respectively making the no-arguments call work in the components libraries.

In tests/development/metadata_test.py

• Added this new file to compare to in test cases.

In tests/development/test_base_component.py

• Added tests for the generate_class_string and generate_class_file functions.

In tests/development/test_component_loader.py

• Added tests for the new generate_classes function, specifically that the build-time generate classes produce the same objects as the run-time classes by comparing their reprs.

[chriddyp]

Amazing stuff @rmarren1 ! I'm on the road now, I'll take a deeper look at this on Friday

[chriddyp]

@rmarren1 - This looks great to me. Excellent tests and code organization.

The only change that I'd like to make is including a comment at the top of the auto-generated files that says:

# AUTO-GENERATED FILE - DO NOT EDIT

And now that we have this framework in place, we can start working on improving the autogenerated code. I think that this can happen in a separate PR. I'm thinking changes like:

• More semantic comment strings
• Templating all of the names in __init__.py instead of doing them at import. I think this will help out with IDE support. The codegen step will have to overwrite the __init__.py file, so anything that is developer-configurable (like the additional CSS and JS files) will probably need to be placed in a separate file and imported. That way, the developer can run codegen without over-writing any of their stuff.
• Switch from **kwargs in the component arguments to named arguments (something like figure=..., id=..., ..). This is a harder one. We can't do figure=None because then we can distinguish between when a user explicitly sets a property to None vs when a user just wants to omit the property. If our docstrings are good enough, then maybe that will suffice for tab-completion. If not, then maybe we introduce a new dash class that signifies an omitted property like Graph(id=Omit, figure=Omit).

With those fixes in place, then editors should stop complaining 😸

I'll create a new issue with these.

[rmarren1]

Added those comments here, and in all the dash-html-components and dash-core-components files.

[chriddyp]

Looks great @rmarren1 ! I'm 💃 to this. Anyone else from @plotly/dash have feedback?

[rmarren1]

I had to make some small changes to properly set the _namespace attribute of the python classes (e.g. rmarren1/dash-html-components@39629d8, I had been accidentally setting the name space to the name of the folder containing metadata.json).

I also now generate an _imports_.py file (e.g. https://github.com/rmarren1/dash-html-components/blob/c5069e8ee5145f543bffbd50d715a64dfe7f0490/dash_html_components/_imports_.py) in the generate_classes step, and replaced the convoluted directory listing method (https://github.com/rmarren1/dash-html-components/blob/f34703f61970a057d46c73f160d1fe06e090f439/dash_html_components/__init__.py#L20-L33) with a simple from ._imports_ import *. This cleans up the code, and gives a quick fix to a lot of the linter issues (specifically the pylint E1101 Module "..." has no "..." member, and pylint E1102 "..." is not callable).

edit: I just got PyCharm up, it also fixes the reference not found in __init__.py inspection error.

[ngnpope]

We can't do figure=None because then we can distinguish between when a user explicitly sets a property to None vs when a user just wants to omit the property.

Just use a sentinel object:

sentinel = object()

class Component(...):
    def __init__(self, figure=sentinel, id=sentinel, ...):
        ...

This avoids the need to introduce new API such as the suggested Omit allowing None to be used for that purpose.

[rmarren1]

It would be nice if we could set the default to the actual default value of the property though, since those values do show up in text editors (they are showing up in Atom) and it would quickly let users know the type of an argument.

We could do something like this:

def explicitize(func):
    def wrapper(**kwargs):
        return func({'_explicit_keys': list(kwargs.keys()), **kwargs})
    return wrapper

class Component(...):
    @explicitize
    def __init__(self, children=None, n_clicks=0, color='blue', _explicit_keys=[], **kwargs):
        kwargs = {k: v for k, v in kwargs.items()
                        if k in_explicit_keys or k.startswith('data-') or k.startswith('aria-')}

[chriddyp]

if we could set the default to the actual default value of the property though, since those values do show up in text editors (they are showing up in Atom) and it would quickly let users know the type of an argument.

And these default properties get set on the React-side in defaultProps, and should be accessible in the metadata.json file, e.g.
https://github.com/plotly/dash-core-components/blob/16515bb2279e11102f39f46781cde26b5de8c3f7/dash_core_components/metadata.json#L195-L198

One caveat is that adding in the default properties would be a backwards incompatible change, as the current behaviour is "if the value isn't specified, it's None", and many users will have their callback logic built around that, like:

app.layout = html.Button(id='button')

@app.callback(..., [Input('button', 'n_clicks')])
def run(n_clicks):
    if n_clicks is None:  # but the default property would be `0`
          raise dash.exceptions.PreventException()

That's not to say that we shouldn't do it, but it would require a major version upgrade and some careful announcements to make sure everyone is aware when they upgrade.