Multiple entrypoints on the same page causes chunks to be sourced multiple times

[EthanZeigler]

Description
When using multiple bundles on the same page, shared dependencies are included more than once. For example, when using a "global" bundle and a specific app on the same page, shared chunks between them are sourced multiple times. This is especially problematic when global libraries like bootstrap are in those shared chunks.

Expected
When rendering a template or set of templates (extension), shared chunks are sourced only once

Actual
Shared chunks are included on the rendered page more than once, breaking libraries that rely on only having one instance.

Basic example
Given 2 bundles global and pageapp and shared chunk chunk...

{% load render_bundle from webpack_loader %}
{% render_bundle 'global' 'js' %}
{% render_bundle 'pageapp' 'js' %}

output currently is

<script type="text/javascript" src="http://localhost:8001/chunk.js"></script>
<script type="text/javascript" src="http://localhost:8001/global.js"></script>
<script type="text/javascript" src="http://localhost:8001/chunk.js"></script>
<script type="text/javascript" src="http://localhost:8001/pageapp.js"></script>

[karolyi]

I have the same problem, but it's independent of this module.

The output of webpack-bundle-tracker (the provided json file) contains the shared modules for the modules you use, and thus they will appear in the generated template, unfortunately. I'm trying to find a way for webpack to implement some kind of a singleton pattern to have the modules run only once even if they are included multiple times.

[fjsj]

webpack-bundle-tracker is a library from django-webpack-loader team as well, so we could fix it if necessary.
However, I think the problem here is your use case. You have multiple entrypoints per HTML page. AFAIK, this is not recommended by webpack as explained here: django-webpack/webpack-bundle-tracker#63 (comment)

After PR #232, this module tries to focus on including entrypoints (with all the chunks) in Django templates. To handle how chunks are divided between entrypoints, try a proper webpack config.

Have you tried the dependOn option + optimization.runtimeChunk: 'single' from Webpack 5? I think it enables you to separate your "global" entrypoint: https://webpack.js.org/guides/code-splitting/#prevent-duplication

[karolyi]

Hey,

thanks for your elaborate response.

Yes I use runtimeChunk: 'single' already. My case is, I have a common.js module I include on every page (from the extended base template) , that has some initializations on the menu structure and some more smaller stuff. This common.js module also has a couple exports (utilities) that the entry points sometimes use. So the issue is, the common.js gets included many times, and in reality, as mentioned in the related issue, I only need a singleton-like pattern. At least, if it's not avoidable having it included many times, which by the looks of the generated JSON bundle report, seems to be the case.

As for using the dependOn option, I would avoid using it, since webpack should be able to do automatic tracing of what modules are used for every entry point by analyzing the provided code. I didn't really get why it was implemented in the first place, but that might be just me.

[EthanZeigler]

dependsOn

I'm currently using the chunks keyword, not depends on, however it still seems wrong to source a shared chunk more than once such as bootstrap.

singleton

I looked into this when I first identified the issue. Because of the minimal featureset of the Django template language, this is very difficult to do in a way that supports concurrent requests.

not recommended by webpack

I understand that, however Vue is intended to do things like this as part of its design philosophy. My use case involves a global, legacy code base using bootstrap. Individual pages are being replaced with vue in an effort to get away from jquery.

[fjsj]

Could you please check for a similar behavior in https://github.com/jantimon/html-webpack-plugin ?

Does it automatically deduplicates common chunks when rendering the script tags inside a single html with multiple entrypoints? It seems to do so (see code below), but could you please test it?

https://github.com/jantimon/html-webpack-plugin/blob/2f5de7ab9e8bca60e9e200f2e4b4cfab90db28d4/index.js#L696-L700

If that happens with html-webpack-plugin, I guess we should implement something similar. If not, personally I don't think this is something django-webpack-loader should do.

[TeamTeaTime]

@fjsj I think you're correct about the behavior of html-webpack-plugin.

I'll post this here in case it is helpful to someone. I just threw this together this morning, so I really wouldn't use this. I did not test that this works with keyword arguments or do any testing at all beyond the simple case. I wrote an additional template tag render_bundle_once that checks and sets a context variable that just stores a list of the script tags that have already been loaded on the page. Subsequent calls to render_bundle_once will omit script tags that were already written.

class RenderBundleOnceNode(template.Node):
    def __init__(self, bundle_name, extension=None, config='DEFAULT', suffix='', attrs='', is_preload=False):
        self.bundle_name = bundle_name
        self.extension = extension
        self.config = config
        self.suffix = suffix
        self.attrs = attrs
        self.is_preload = is_preload

    def render(self, context):
        tags = utils.get_as_tags(
            self.bundle_name, extension=self.extension, config=self.config,
            suffix=self.suffix, attrs=self.attrs, is_preload=self.is_preload
        )

        used_tags = context.get("webpack_loader_used_tags", [])
        if not used_tags:
            context["webpack_loader_used_tags"] = []
        new_tags = [mark_safe(tag) for tag in tags if tag not in used_tags]
        context["webpack_loader_used_tags"] = context["webpack_loader_used_tags"] + new_tags
        
        return mark_safe('\n'.join(new_tags))

@register.tag
def render_bundle_once(parser, token):
    try:
        tag_name, bundle_name, *options = token.split_contents()
    except ValueError:
        raise template.TemplateSyntaxError(
            "%r tag requires a bundle name" % token.contents.split()[0]
        )
    if not (bundle_name[0] == bundle_name[-1] and bundle_name[0] in ('"', "'")):
        raise template.TemplateSyntaxError(
            "%r tag's argument should be in quotes" % tag_name
        )
    option_names = ["extension", "config", "suffix", "attrs", "is_preload"]
    # TODO fix type of is_preload using template variable
    # TODO test that the kwargs work at all?
    kwargs = dict(zip(option_names, options))
    return RenderBundleOnceNode(bundle_name[1:-1], **kwargs)

[karolyi]

This is awesome, a much better solution than me pestering the devs to implement the stuff in webpack!

I'm gonna give this a try and probably come up with improvements.

[fjsj]

Thanks @TeamTeaTime
If I can read the code correctly, one can use render_bundle_once multiple times, but there will be no repetition of <script> tags for common chunks, is that right? You're using context, which seems the right way to do it (check https://stackoverflow.com/a/33564990/145349).

What @karolyi is asking for requires this context-global behavior of removing duplicates the already included script tags (by previous calls of render_bundle). I don't love this behavior, but since base templates are very common in Django, we may need it.

I was thinking instead on a render_bundle_once that accepts multiple entrypoints as argument (like a comma separated list, or even multiple first args). But this would prevent the use of render_bundle 'global' in the base template, since it would force the user to group the call of all entrypoints in a single render_bundle call, like render_bundle 'global' 'pageapp' 'js'.

So I think my suggestion above won't solve most use cases. If you really need this global behavior across multiple render_bundle calls, we can add a keyword argument to render_bundle to prevent duplication. Like:

{% load render_bundle from webpack_loader %}
{% render_bundle 'global' 'js' %}
{% render_bundle 'pageapp' 'js' skip_common_chunks=True %}

Please feel free to open a PR with this. In a future major version, we can make this default True.

[TeamTeaTime]

I created a PR. Let me know what you think. I'm with you and I'm not sure it's a great idea to steer people toward including multiple bundles, but there are certainly cases where it could be easier than the alternative.

[EthanZeigler]

I'm not sure it's a great idea to steer people toward including multiple bundles

A good middle ground might be emitting a warning when the server is in develop mode.

[karolyi]

this is now solved in #297 that was merged a couple days ago in 1.4.0.