explicit Components classes to allow autocomplete

[sdementen]

metadata.zip
Currently, the Dash components classes are generated only at runtime. Therefore, it is not possible to get autocomplete feature or doc/help contextual hints in IDE (e.g. pycharm).
I would propose to explicit the classes as a real python module and so convert the metadata.json in a metadata.py that could be imported in a standard way.
It could also ease the debugging when adapting these component classes to implement the different TODOs in the code.

I have slightly adapted the component_loader.py as well as the __init__.py of the dash_core_components and dash_html_components to achieve this.
You can find in attachements the resulting metadata.py for both modules.

As this requires changes in different repos (dash but also the dash_*_components), I haven't yet done a PR on this.
If you find the approach valuable, I am ready to follow your guidelines on how to push the changes.

[sdementen]

I have pushed the changes in the branches classgen-#150 in https://github.com/sdementen/dash/tree/classgen-%23150
https://github.com/sdementen/dash-core-components/tree/classgen-%23150
https://github.com/sdementen/dash-html-components/tree/classgen-%23150

[sdementen]

btw, I am not yet happy with the ISUNDEFINED and MUSTBEDEFINED sentinel approach...

[chriddyp]

Thanks @sdementen ! Excited to check this out.

I'm having a hard time finding the commits in classgen-#150 - am I looking in the right place?

[sdementen]

sorry, it wasn't pushed correctly. You should be able to see them now.

on the approach, I wonder if the metadata.py file should be "compiled" during the development/release cycle (and so included in the source), or generated on the fly (with some caching mechanism that checks metadata.json is newer than metadata.py)

[chriddyp]

on the approach, I wonder if the metadata.py file should be "compiled" during the development/release cycle (and so included in the source)

Great idea, I think that's it exactly.

Here is a rough set of steps that would be required for creating these components at build-time vs run-time. cc @alysivji

1. Modify dash.development.base_component.generate_class and dash.development.component_loader.load_components

These functions are called at runtime by the component libraries, see e.g. https://github.com/plotly/dash-core-components/blob/c898d75a1a189943bcf92bc73b95bd08432ddd7d/dash_core_components/__init__.py#L8-L11

They consume a metadata.json file that is included in the package folder (e.g. https://github.com/plotly/dash-core-components/blob/c898d75a1a189943bcf92bc73b95bd08432ddd7d/dash_core_components/metadata.json#L1-L26)

Note that metadata.json is generated from comments in the component author's React.js code with react-docgen in the extract-metadata step in the package.json: https://github.com/plotly/dash-components-archetype/blob/67912c1a56a58fa68e1f00c90136af1d7ae444d9/package.json#L37

We'll want to explore modifying this step so that it generates python code. Note that this template for the component code is already written - we'll just need to convert it from an exec to a file write:

dash/dash/development/base_component.py

Lines 198 to 255 in 3cd6a78

	c = '''class {typename}(Component):
	"""{docstring}
	"""
	def __init__(self, {default_argtext}):
	self._prop_names = {list_of_valid_keys}
	self._type = '{typename}'
	self._namespace = '{namespace}'
	self.available_events = {events}
	self.available_properties = {list_of_valid_keys}
	for k in {required_args}:
	if k not in kwargs:
	raise Exception(
	'Required argument `' + k + '` was not specified.'
	)
	super({typename}, self).__init__({argtext})
	def __repr__(self):
	if(any(getattr(self, c, None) is not None for c in self._prop_names
	if c is not self._prop_names[0])):
	return (
	'{typename}(' +
	', '.join([c+'='+repr(getattr(self, c, None))
	for c in self._prop_names
	if getattr(self, c, None) is not None])+')')
	else:
	return (
	'{typename}(' +
	repr(getattr(self, self._prop_names[0], None)) + ')')
	'''
	filtered_props = reorder_props(filter_props(props))
	list_of_valid_keys = repr(list(filtered_props.keys()))
	docstring = create_docstring(
	typename,
	filtered_props,
	parse_events(props),
	description
	)
	events = '[' + ', '.join(parse_events(props)) + ']'
	if 'children' in props:
	default_argtext = 'children=None, **kwargs'
	argtext = 'children=children, **kwargs'
	else:
	default_argtext = '**kwargs'
	argtext = '**kwargs'
	required_args = required_props(props)
	d = c.format(**locals())
	scope = {'Component': Component}
	exec(d, scope)
	result = scope[typename]
	return result

We'll want to create a function like load_components that will generate the code for each component, from a metadata.json file, and write it to the appropriate folder. Calling the function for e.g. dash-core-components should result in a folder structure like:

- dash_core_components/
- dash_core_components/__init__.py
- dash_core_components/Checklist.py
- dash_core_components/DatePickerRange.py
- dash_core_components/DatePickerSingle.py
- dash_core_components/Dropdown.py
- dash_core_components/Graph.py
- dash_core_components/Input.py
- dash_core_components/Interval.py
- ...

2. Modify the dash-components-archetype build step

This is the starter package that is used to generate Dash packages from React.js code. Each component library is initialized from this repository and each component library uses the NPM scripts from this component library through a package called "builder".

We'll need to modify these package's scripts to generate the python code. The command for this script will live here: https://github.com/plotly/dash-components-archetype/blob/67912c1a56a58fa68e1f00c90136af1d7ae444d9/package.json#L29-L43 and we can call it something like "generate-python-components" and we'll need to stick it after each extract-metadata script.

For example, build-dist https://github.com/plotly/dash-components-archetype/blob/67912c1a56a58fa68e1f00c90136af1d7ae444d9/package.json#L33
will become

builder run clean-lib && 
builder run extract-metadata &&
builder run generate-python-components &&
webpack --config=node_modules/dash-components-archetype/config/webpack/webpack.config.dist.js"

"generate-python-components" could look something like:

python -c "import dash; dash.development.component_loader.generate_components()"

where the generate_components function might look for a local metadata.json file.

3. Modify the dash-components-archetype __init__.py

The standard component __init__.py calls the component loader: https://github.com/plotly/dash-core-components/blob/c898d75a1a189943bcf92bc73b95bd08432ddd7d/dash_core_components/__init__.py#L6-L11. We'll need to remove this and add whatever is necessary so that the standard imports work:

import dash_core_components as dcc
dcc.Graph(...)

4. Modify the dash-html-components and dash-core-components to use the patches in dash-components-archetype and dash

• Update dash-components-archetype in each package (e.g. https://github.com/plotly/dash-core-components/blob/c898d75a1a189943bcf92bc73b95bd08432ddd7d/package.json#L28)
• Manually update the __init__.py files in each package to modify the new __init__.py template (as modified in dash-components-archetype)
• Run the new build commands in each package to generate the components

4. Version management
It would be great to not break anyone's code. If a user upgrades dash but doesn't upgrade the new version of dash-core-components, their code shouldn't break.
So, we should keep the existing functions in dash.development.load_components and dash.development.generate_class and add all of this behaviour in new functions.

The new versions of the component libraries should also work with the new version of Dash but also the older versions of Dash. Since generating the components is done as a build-step instead of runtime, the components shouldn't even depend on the version of the dash.development code during runtime.

5. Testing

• There are some good tests in dash.development. We'll need to keep those so that we can maintain backwards compatibility: https://github.com/plotly/dash/blob/master/tests/development/test_component_loader.py#L103-L108
• We should add some new tests in dash/tests/development for the new behaviour.

[znd4]

Sorry, did you mean to close this issue when you closed #226 ?

[rmarren1]

This was added in #276

[michael72]

I don't think this issue was properly closed and it is totally annoying behavior - why would a callback for a button need to be called when the callback has not been clicked yet???

The reference in the push above does not affect in any way this issue.