Assets files & index customizations #286

T4rk1n · 2018-07-11T17:28:03Z

Assets includes & index customization

Solution for #265 Proposal for Offline CSS and JS and Customizable index.html

Assets include

Dash will now look for a folder named assets on the current work directory, if one is found, it will walk that directory and include js and css files on the index.

Related configs:

import dash

app = dash.Dash()

# default values
app.config.assets_folder = 'assets'     # The path to the assets folder.
app.config.include_asset_files = True   # Include the files in the asset folder
app.config.assets_external_path = ''    # The external prefix if serve_locally == False
app.config.assets_url_path = '/assets'  # the local url prefix ie `/assets/*.js`

Supported files:

.js, javascript files will be included after the components libs and before the dash-render.
.css, stylesheets will be included as a <link> in the head.
favicon.ico, include as a <link> in the head.

The files are included in alphabetic order.
The directories can be nested.

Advanced use:

To host your assets content externally:

set app.scripts.config.serve_locally = False
app.config.assets_external to your base host url, ie http://bucket.s3.amazonaws.com/
app.config.include_asset_files must still be set to True for the files to be indexed by dash.
Duplicate the file structure in your assets folder to your file hoster and the files will be loaded from there instead.

Index customization

Meta tags

It is now possible to add meta tags to the index of dash.

Example:

import dash

metas = [
    {'name': 'description', 'content': 'My description'}
]

app = dash.Dash(meta_tags=metas)

# alternatively
app.add_meta_tag({'http-equiv': 'X-UA-Compatible', 'content': 'IE=edge'})

Customizing the index

Add an index_string to change the default index dash use.

import dash

app = dash.Dash()
app.index_string = '''
<!DOCTYPE html>
<html>
    <head>
        {%metas%}
        <title>{%title%}</title>
        {%favicon%}
        {%css%}
    </head>
    <body>
        <div>My Custom header</div>
        {%app_entry%}
        <footer>
            {%config%}
            {%scripts%}
        </footer>
        <div>My Custom footer</div>
    </body>
</html>
'''

The {%key%}s will be formatted like in the default index.

Available keys:

{%metas%}        # optional - The registered meta tags.
{%favicon%}      # optional - A favicon link tag if found in `assets`.
{%css%}          # optional - link tags to css resources.
{%config%}       # required - Config generated by dash for the renderer.
{%app_entry%}    # required - The container where dash react components are rendered.
{%scripts%}      # required - Collected dependencies scripts tags.

Also added interpolate_index method on Dash, override it to get the context values of the index before rendering.

import dash

class CustomDash(dash.Dash):
    def interpolate_index(self,
                          metas, title, css, config,
                          scripts, app_entry, favicon):
        return '''
        <!DOCTYPE html>
        <html>
            <head>
                <title>My App</title>
            </head>
            <body>
                
                <div id="custom-header">My custom header</div>
                {}
                {}
                <div id="custom-footer">My custom footer</div>
            </body>
        </html>
        '''.format(app_entry, scripts)

I added tests for the meta keys and index customization. For the assets, I made a test in an external project to test if the files are included in that project, @plotly/dash I need help to integrate that test to run with the other tests.

nicolaskruchten · 2018-07-11T17:42:46Z

dash/dash.py

+                flask.url_for('assets.static', filename=self._favicon))
+        else:
+            favicon = ''
+        return self.interpolate_index(


nicolaskruchten · 2018-07-11T17:46:26Z

dash/_utils.py

+def interpolate_str(template, **data):
+    s = template
+    for k, v in data.items():
+        key = '{' + k + '}'


This is probably too brittle... in modern Javascript you could easily write something like setConfig({config}) and have someone paste that into index

Think doubling the brackets {{config}} would be enough or maybe add a character before %{config} ?

doubling is probably not enough... I would do something like {%dash_config%} or something personally just to make it super clear

nicolaskruchten · 2018-07-11T17:47:25Z

dash/dash.py

+    def interpolate_index(self,
+                          metas, title, css, config,
+                          scripts, app_entry, favicon):
+        return _interpolate(self.index_string,


There's probably not enough error-checking here... Shouldn't we raise a very helpful and clear message if they forget to add {scripts} or something?

Two options I thought of:

check in a property setter on the index_string.

fast only one check, but no checking on interpolate_index.

raise when set.

check after interpolate_index the string contains the ids of required elements.

check every time the index render.

raise only when browsing.

Which would be best ?

I would actually do both :) That way it fails fast for those using index but it still checks for those using interpolate_index ... better developer experience all around

chriddyp · 2018-07-11T18:34:01Z

dash/dash.py

    def run_server(self,
                   port=8050,
                   debug=False,
                   **flask_run_options):
+        bp = flask.Blueprint('assets', 'assets',
+                             static_folder=self.config.assets_folder,
+                             static_url_path=self.config.assets_url_path)


Do we need to ensure a minimum version of flask for this? i.e. >1.0?

It was add in 0.7, we have Flask>=0.12.

chriddyp · 2018-07-11T18:56:34Z

dash/dash.py

+
+        tags = []
+        if not has_charset:
+            tags.append('<meta charset="UTF-8"/>')


This is very thoughtful 👍

chriddyp · 2018-07-11T19:08:28Z

dash/dash.py

+                if len(splitted) > 1:
+                    base = '/'.join(slash_splitter.split(s))
+                else:
+                    base = splitted[0]


Could we use base = os.path.split(walk_dir)[0]?

NVM, I get it now, we have to replace \ with / for URL friendly paths.

chriddyp · 2018-07-11T19:16:43Z

dash/dash.py

+        {app_entry}
+        <footer>
+            {config}
+            {scripts}


Could we split out scripts into dash_renderer_scripts and dash_components_scripts? The main reason is that dash_renderer is going to become configurable in when we add in custom JS hooks. So, users will end up needing to do something like:

dash_renderer = DashRenderer({ request_hook: function(...) { ... } })

And so it'd be nice if they could replace this:

<!DOCTYPE html> <html> <head> {metas} <title>{title}</title> {favicon} {css} </head> <body> {app_entry} <footer> {config} {dash_renderer} {dash_component_scripts} </footer> </body> </html>

with something like this:

<!DOCTYPE html> <html> <head> {metas} <title>{title}</title> {favicon} {css} </head> <body> {app_entry} <footer> {config} dash_renderer = DashRenderer({ request_hook: function(...) { ... } }) {dash_component_scripts} </footer> </body> </html>

Now, I suppose the other way they could do this would be with the interpolated index function, where they would do something like:

class CustomDash(dash): def interpolated_index(metas, title, css, config, scripts, _app_entry, favicon): filtered_scripts = [ script for script in scripts if 'dash_renderer' not in script ] return ''' <!DOCTYPE html> <html> <head> {metas} <title>{title}</title> {favicon} {css} </head> <body> {app_entry} <footer> {config} dash_renderer = DashRenderer({ request_hook: function(...) { ... } }) {filtered_scripts} </footer> </body> </html> '''.format(metas, title, css, config, filtered_scripts, _app_entry, favicon)

Am I understanding that correctly?

I wasn't too sure about the js hooks so I left them out of this, right now the scripts are all bundled together in _generate_scripts_html that return a string with all the scripts tags, can't iterate over it but it could change.

I thought we could have a config to disable the includes of dash-renderer. Then the user can append a custom renderer to the scripts resources or include it a custom index.

If they're customizing the DashRenderer don't they need to have the tags to load the uncustomized one first?

chriddyp · 2018-07-11T19:21:06Z

Overall this is looking really good! A few topics of discussion below. Would love feedback @plotly/dash

import dash

metas = [
    {'name': 'description', 'content': 'My description'}
]

app = dash.Dash(meta_tags=metas)

I actually like this declarative syntax. This seems like a nice pattern that we could use elsewhere, like as a way to replace app.scripts.append_script:

app = dash.Dash(
     js_script_urls=['https://cdn.google.com/google-analytics.js'],
     css_stylesheet_urls=['https://cdn.bootstrap.com/bootstrap.css']
)

I never really liked the app.scripts.append_script syntax and I'd generally prefer for the dash.Dash() object to have as few methods and properties as possible. If everything could just be in the app.config and settable through the dash.Dash() constructor, I'd be very happy.

app.add_meta_tag({'http-equiv': 'X-UA-Compatible', 'content': 'IE=edge'})

I'm less fond of this syntax as I prefer if there would only be a single declarative way to do things. Now, I'm biased as I've been in the declarative-or-nothing plotly world for several years, so I'd be happy to hear other perspectives :). Anyway, here are my thoughts:

For documentation and answering questions in the community forum, it's easier to have a single way to explain how to do things. In particular, one scenario is when users aren't using the API correctly: if there is more than one way to do things, they'll try each way (incorrectly) and get frustrated that none of them work. They aren't sure if the methods are truly equivalent or if they need to use one method instead of the others.
I prefer to keep the number of methods on dash.Dash() small. That way, we only have to document the dash.Dash constructor and users can do everything they need by calling help(dash.Dash).
If we have add_meta_tag, then for consistency I'd expect there to be equivalent add_other_thing whenever we add other resources. For example, scripts, or css. Which we could do, but it's one more thing to keep track of.
Similarly, for some of the attributes, the add_* method doesn't make sense. For example, the title attribute: add_title doesn't really make sense because there can only be a single title. However, this becomes inconsistent with meta because meta can be set with two different ways (add_meta and the dash.Dash) but other properties in dash.Dash can only be set one way.
If they have app.add_meta_tag in a different file that maybe isn't getting imported correctly, then they might think that app.add_meta_tag isn't getting called. If we force users to use app.Dash(meta_tags=[...]), then it should always work and they don't need to worry about the location of their other function calls. This is sort of a general argument about why I prefer declarative configurations to imperative function calls (declarative attributes are order and placement independent)
Similarly, I think we encourage good code organization by forcing users to put the meta_tags inside the constructor, near where they are configuring everything else about their Dash app, rather than giving them control over where they want to place it.

Would love feedback on this @plotly/dash and community. Are these valid arguments or am I being too pedantic?

chriddyp · 2018-07-11T19:26:24Z

dash/dash.py

+            elif 'asset_path' in resource:
+                static_url = flask.url_for('assets.static',
+                                           filename=resource['asset_path'])
+                srcs.append(static_url)


I wonder if we should wire in cache-busting URLs while we're here. It's a really common issue in the community forum (e.g. https://community.plot.ly/t/reloading-css-automatically/11065).

I originally thought that we could just do this with a query string with the last modified timestamp but it seems like that's not recommended (https://css-tricks.com/strategies-for-cache-busting-css/#article-header-id-2). Instead, it seems like we'd need to somehow encode it into the resource name. Do you have a sense of how hard that might be?

Although I'm being pretty hypocritical here, looks like I did cache busting with the component libraries with query strings:

dash/dash/dash.py

Lines 194 to 199 in 3dfa941

return '{}_dash-component-suites/{}/{}?v={}'.format(

self.config['routes_pathname_prefix'],

namespace,

relative_package_path,

importlib.import_module(namespace).__version__

)

🙄

I did cache busting before with webpack and an extension. Was basically a template render that replaced the hash part of a filename with a new one.

In dash, I think it would be kinda hard to do that, but we could have a watcher on the asset folder, copy those assets with a filename including the hash to a temp static folder, keep the hash in a dict with the path as key and serve the file with the hash formatted in. When a file change, copy it to the temp folder with a new hash and put the new hash as the value for the path. Could also tell the browser to reload while we're at it.

Yeah that's a good idea. I'm a little worried about introducing a temp folder, I feel like users won't know why it's there or if they should commit it, etc.

Instead of having a watcher, could we just call this function on every page load (while in dev mode) and get the latest timestamp of the file? And then if we're not in dev mode, we could store the timestamps on the first page load?

The temp folder would be created with the tempfile module and located in the user temp directory (ie %appdata%/local/temp).

But yea, just checking the timestamps of the files before index and appending that in a query string would be a quick fix.

Cool, sounds like there are a couple of options. So, let's create a new GitHub issue about this and tackle it in a subsequent PR

chriddyp · 2018-07-11T19:28:05Z

For the assets, I made a test in an external project to test if the files are included in that project, @plotly/dash I need help to integrate that test to run with the other tests.

Could you include an assets folder inside the tests folder?

chriddyp · 2018-07-24T23:27:22Z

Let's give members from @plotly/dash another 12 hours to give a second review. If no one else steps in, let's go ahead and merge and release it :)

rmarren1

This looks great overall. One subtle point I think was missed:

rmarren1 · 2018-07-25T00:44:41Z

dash/dash.py

+                else:
+                    base = splitted[0]
+
+            for f in files:


os.walk calls os.listdir internally and the os.listdir docs state "The [returned] list is in arbitrary order" since the user's OS ultimately decides what ordering this returns. Definitely not desirable as apps will behave differently on different machines.

I had a hunch the test case was just getting lucky so I added more files and got this ordering: ['load_first', 'load_after6', 'load_after', 'load_after1', 'load_after2', 'load_after5', 'load_after11', 'load_after4', 'load_after3', 'load_after10', 'load_ after7']

This can be fixed by just adding

for current, _, files in os.walk(walk_dir): if current == walk_dir: base = '' else: s = current.replace(walk_dir, '').lstrip('\\').lstrip('/') splitted = slash_splitter.split(s) if len(splitted) > 1: base = '/'.join(slash_splitter.split(s)) else: base = splitted[0] files.sort() # ADDED LINE: Sort the files! for f in files: if base: path = '/'.join([base, f]) else: path = f

This sorts the files as: ['load_first', 'load_after', 'load_after1', 'load_after10', 'load_after11', 'load_after2', 'load_after3', 'load_after4', 'load_after5', 'load_after6', 'load_ after7']

I personally would prefer if 'load_after10' and 'load_after11' came last, which can be done by changing
files.sort() -> files.sort(key=lambda f: int('0' + ''.join(filter(str.isdigit, f))))

The former is what is expected, but the latter is what a programmer usually wants when sorting files. Either way I like the way everything else looks and am 💃 once the sorting works.

ned2 · 2018-07-25T11:42:08Z

I can take a look at this shortly, if you hold off merging for a couple of hours

…

On Wed, Jul 25, 2018, 10:47 Ryan Marren ***@***.***> wrote: ***@***.**** commented on this pull request. This looks great overall. One subtle point I think was missed: ------------------------------ In dash/dash.py <#286 (comment)>: > + res['external_url'] = '{}{}'.format( + self.config.assets_external_path, path) + return res + + for current, _, files in os.walk(walk_dir): + if current == walk_dir: + base = '' + else: + s = current.replace(walk_dir, '').lstrip('\\').lstrip('/') + splitted = slash_splitter.split(s) + if len(splitted) > 1: + base = '/'.join(slash_splitter.split(s)) + else: + base = splitted[0] + + for f in files: os.walk calls os.listdir internally and the os.listdir docs state <https://docs.python.org/3.6/library/os.html#os.listdir> "The [returned] list is in arbitrary order" since the user's OS ultimately decides what ordering this returns. Definitely not desirable as apps will behave differently on different machines. I had a hunch the test case was just getting lucky so I added more files and got this ordering: ['load_first', 'load_after6', 'load_after', 'load_after1', 'load_after2', 'load_after5', 'load_after11', 'load_after4', 'load_after3', 'load_after10', 'load_ after7'] This can be fixed by just adding for current, _, files in os.walk(walk_dir): if current == walk_dir: base = '' else: s = current.replace(walk_dir, '').lstrip('\\').lstrip('/') splitted = slash_splitter.split(s) if len(splitted) > 1: base = '/'.join(slash_splitter.split(s)) else: base = splitted[0] files.sort() # ADDED LINE: Sort the files! for f in files: if base: path = '/'.join([base, f]) else: path = f This sorts the files as: ['load_first', 'load_after', 'load_after1', 'load_after10', 'load_after11', 'load_after2', 'load_after3', 'load_after4', 'load_after5', 'load_after6', 'load_ after7'] I personally would prefer if 'load_after10' and 'load_after11' came last, which can be done by changing files.sort() -> files.sort(key=lambda f: int('0' + ''.join(filter(str.isdigit, f)))) The former is what is expected, but the latter is what a programmer usually wants when sorting files. Either way I like the way everything else looks and am 💃 once the sorting works. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#286 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACs1GFhwKbKGM8dgElualPuEPeSv3zB8ks5uJ8A4gaJpZM4VLi5g> .

ned2

Spotted a couple places for improvements but they're not blockers, so it's a 💃 from me also

(Edit: oops, does the explicit Approve do anything of significance?)

ned2 · 2018-07-25T13:27:27Z

dash/dash.py

+                        kwargs.get('app_entry'),
+                        kwargs.get('config'),
+                        kwargs.get('scripts'))
+


I'd have a preference for using named interpolations here, as I think this is good practice for such a long string being interpolated. eg '{scripts}' ... .format(scripts=kwargs['scripts']) But it's not that much of an issue.

ned2 · 2018-07-25T13:32:40Z

dash/dash.py

        )
+        missing = [missing for check, missing in checks if not check]
+        if missing:
+            raise Exception(


This should be a Dash-specific exception from the dash.exceptions module, but there's already a couple of other plain Exceptions in dash.py, so it could just be fixed in another PR.

bhuffman-usgs · 2018-07-25T22:46:54Z

Chris, is this now functioning? I've made an attempt to replicate your "Simple" and "Override Index" example however, the .css formatting isn't being applied.

chriddyp · 2018-07-25T23:08:44Z

Chris, is this now functioning? I've made an attempt to replicate your "Simple" and "Override Index" example however, the .css formatting isn't being applied.

Yeah this is. See https://community.plot.ly/t/dash-version-0-22-0-released/12098 for some official notes. I'm working on some official docs in https://github.com/plotly/dash-docs/pull/126

jonboone1 · 2022-05-05T23:43:00Z

I've been reading through documentation and am having a really hard time understanding how to accomplish what I want. I am trying to include additional tags in the header of the page, not just meta tags. I have both link and script tags I want to include, which doesn't seem to work with the meta_tags functionality.

Any thoughts on how to accomplish this? I've also tried to set the index_string but received the following error:
dash.exceptions.InvalidIndexException: Missing item new DashRenderer in index.

alexcjohnson · 2022-05-09T22:26:58Z

@jonboone1 by link and script tags do you mean you want to include your own .css and .js files on the page? Those are both included in the page automatically if you put them in the assets directory, no need to explicitly create the tags - see https://dash.plotly.com/external-resources

If the resources you want to add are elsewhere on the internet, you can use external_stylesheets and external_scripts - that's farther down on the same page https://dash.plotly.com/external-resources#adding-external-css/javascript

joshsmith2 · 2023-10-11T10:54:50Z

@jonboone1 - recognise this is now 18 months old - but if anyone else stumbles on this with requirements for custom <head> - in my case, the need to add custom attributes to the <script> tags I was embedding, I found that a missing {%renderer%} tag in the body was causing that InvalidIndexException.

The docs for this are under 'Option 1 - Index string' here.

T4rk1n added 16 commits July 10, 2018 11:51

Add support for static_path resources.

53ec2ef

Add static directory walk for files to include.

a14709a

pylint fixes.

69b8adc

Add support for meta tags.

7cafb47

Add support favicon located in static dir.

7ebab39

Fix static walking nested directories.

fc26544

Changed the meta tags dict to a list, added meta_tags to dash.__init__.

d2cce95

Add test for meta tags.

1212ee5

Fix bad line that was included in rebase.

846c8fc

Add support for static external resources.

4777731

Rename static to assets for user static file includes.

23195f0

Add _generate_meta_html to build html meta tags.

0c96dc7

Add index customization by string interpolations.

f943471

Re-add support for favicon in interpolate_index.

cd0c15c

Change add_meta_tag args to a dict to support every meta attributes.

0d9151c

Add test for index customization.

ff1cff3

T4rk1n changed the title ~~Assets index customizations~~ Assets files & index customizations Jul 11, 2018

nicolaskruchten reviewed Jul 11, 2018

View reviewed changes

chriddyp added the dash-meta-sponsored Work items whose development has been sponsored by a commercial partner https://plot.ly/dash/pricing label Jul 11, 2018

chriddyp reviewed Jul 11, 2018

View reviewed changes

dash/dash.py

tags = []

if not has_charset:

tags.append('<meta charset="UTF-8"/>')

Copy link

Member

chriddyp Jul 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very thoughtful 👍

chriddyp reviewed Jul 11, 2018

View reviewed changes

T4rk1n force-pushed the assets-index-customizations branch from 0ba032b to 84acbab Compare July 11, 2018 20:51

Add test for assets.

eaf91a1

Add docstring to interpolate_index.

6af0381

rmarren1 reviewed Jul 25, 2018

View reviewed changes

ned2 approved these changes Jul 25, 2018

View reviewed changes

ruairioconn mentioned this pull request Jul 25, 2018

Create CSS stylesheet ruairioconn/LRA-Test-GUI#6

Open

2 tasks

T4rk1n and others added 4 commits July 25, 2018 11:02

Ensure assets files are ordered, add more test files.

2c104a5

Update changelog and version.

1171f0a

Merge branch 'master' into assets-index-customizations

cd4cfa9

pylint fixes.

495762e

T4rk1n merged commit 54ca4a3 into master Jul 25, 2018

cpsievert mentioned this pull request Jul 25, 2018

More flexible index page templating plotly/dashR#42

Closed

chriddyp mentioned this pull request Jul 25, 2018

Cache-busting URLs for CSS included in the assets folder #303

Closed

chriddyp mentioned this pull request Jul 27, 2018

Proposal for Offline CSS and JS and Customizable index.html #265

Closed

T4rk1n deleted the assets-index-customizations branch July 31, 2018 20:17

This was referenced Aug 1, 2018

Improves cross-browser compatibility #275

Closed

Add external_js/css_urls to dash init. #305

Merged

xhluca mentioned this pull request Aug 17, 2018

Assets not included when added when the app is running #341

Closed

This was referenced Nov 4, 2019

Dash for R does not currently support adding meta tags plotly/dashR#141

Closed

Support for meta tags in Dash for R plotly/dashR#142

Merged

rpkyle mentioned this pull request Jan 14, 2020

Support for index page templating plotly/dashR#168

Merged

CaseGuide mentioned this pull request Jan 21, 2020

[Feature Request] make DocumentTitle changes adjustable #856

Closed

nicolaskruchten mentioned this pull request Mar 18, 2020

allow optional header and footer #171

Closed

HammadTheOne pushed a commit to HammadTheOne/dash that referenced this pull request May 28, 2021

Update all dependencies (plotly#286)

b5994e6

HammadTheOne pushed a commit that referenced this pull request Jul 23, 2021

Update all dependencies (#286)

929c097

	return '{}_dash-component-suites/{}/{}?v={}'.format(
	self.config['routes_pathname_prefix'],
	namespace,
	relative_package_path,
	importlib.import_module(namespace).__version__
	)

Assets files & index customizations #286

Assets files & index customizations #286

Conversation

T4rk1n commented Jul 11, 2018 • edited Loading

Assets includes & index customization

Assets include

Index customization

Meta tags

Customizing the index

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chriddyp commented Jul 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chriddyp commented Jul 11, 2018

chriddyp commented Jul 24, 2018

rmarren1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ned2 commented Jul 25, 2018 via email

ned2 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bhuffman-usgs commented Jul 25, 2018

chriddyp commented Jul 25, 2018

jonboone1 commented May 5, 2022

alexcjohnson commented May 9, 2022

joshsmith2 commented Oct 11, 2023

T4rk1n commented Jul 11, 2018 •

edited

Loading

ned2 left a comment •

edited

Loading