Skip to content

Add a setting to configure skip common chunks behaviour globally #318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 20, 2022
Merged

Add a setting to configure skip common chunks behaviour globally #318

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4aa0ee3
Add a setting to determine skip common chunks behaviour
hyshka May 26, 2022
7498eb6
Add a setting to configure skip common chunks behaviour globally
hyshka May 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ The `STATS_FILE` parameter represents the output file produced by `webpack-bundl

- `LOADER_CLASS` is the fully qualified name of a python class as a string that holds the custom webpack loader. This is where behavior can be customized as to how the stats file is loaded. Examples include loading the stats file from a database, cache, external url, etc. For convenience, `webpack_loader.loader.WebpackLoader` can be extended. The `load_assets` method is likely where custom behavior will be added. This should return the stats file as an object.

- `SKIP_COMMON_CHUNKS` is a flag which prevents already generated chunks from being included again in the same page. This should only happen if you use more than one entrypoint per Django template (multiple `render_bundle` calls). By enabling this, you can get the same default behavior of the [HtmlWebpackPlugin](https://webpack.js.org/plugins/html-webpack-plugin/). The same caveats apply as when using `skip_common_chunks` on `render_bundle`, see that section below for more details.

Here's a simple example of loading from an external url:

```py
Expand Down Expand Up @@ -239,7 +241,7 @@ The public path is based on `webpack.config.js` [output.publicPath](https://webp
Please note that this approach will use the original asset file, and not a post-processed one from the Webpack pipeline, in case that file had gone through such flow (i.e.: You've imported an image on the React side and used it there, the file used within the React components will probably have a hash string on its name, etc. This processed file will be different than the one you'll grab with `webpack_static`).

### Use `skip_common_chunks` on `render_bundle`
You can use the parameter `skip_common_chunks=True` to specify that you don't want an already generated chunk be included again in the same page. This should only happen if you use more than one entrypoint per Django template (multiple `render_bundle` calls). By using `skip_common_chunks=True`, you can get the same default behavior of the [HtmlWebpackPlugin](https://webpack.js.org/plugins/html-webpack-plugin/).
You can use the parameter `skip_common_chunks=True` or `skip_common_chunks=False` to override the global `SKIP_COMMON_CHUNKS` setting for a specific bundle.

In order for this option to work, `django-webpack-loader` requires the `request` object to be in the context, to be able to keep track of the generated chunks.

Expand Down
15 changes: 15 additions & 0 deletions tests/app/templates/home-duplicated-forced.jinja
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Example</title>
{{ render_bundle('app1', 'js') }}
{{ render_bundle('app2', 'js', skip_common_chunks=False) }}
</head>

<body>
<div id="react-app"></div>
{{ render_bundle('app1', 'js', skip_common_chunks=False) }}
{{ render_bundle('app2', 'js', skip_common_chunks=False) }}
</body>
</html>
194 changes: 175 additions & 19 deletions tests/app/tests/test_webpack.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -527,15 +527,13 @@ def test_emits_warning_on_no_request_in_jinja2engine(self):
_warn_mock.assert_not_called()
_warn_mock.reset_mock()

def test_skip_common_chunks_djangoengine(self):
"""Test case for deduplication of modules with the django engine."""
self.compile_bundles('webpack.config.skipCommon.js')
def _assert_common_chunks_duplicated_djangoengine(self, template):
"""
Verify that any common chunks between two bundles are duplicated in
the HTML markup.

django_engine = engines['django']
dups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" %}')) # type: Template
:param template: A Django template instance
"""
request = self.factory.get(path='/')
asset_vendor = (
'<script src="/static/django_webpack_loader_bundles/vendors.js" >'
Expand All @@ -546,7 +544,7 @@ def test_skip_common_chunks_djangoengine(self):
asset_app2 = (
'<script src="/static/django_webpack_loader_bundles/app2.js" >'
'</script>')
rendered_template = dups_template.render(
rendered_template = template.render(
context=None, request=request)
used_tags = getattr(request, '_webpack_loader_used_tags', None)

Expand All @@ -556,13 +554,24 @@ def test_skip_common_chunks_djangoengine(self):
self.assertEqual(rendered_template.count(asset_app2), 1)
self.assertEqual(rendered_template.count(asset_vendor), 2)

nodups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" skip_common_chunks=True %}')
) # type: Template
def _assert_common_chunks_not_duplicated_djangoengine(self, template):
"""
Verify that any common chunks between two bundles are not duplicated in
the HTML markup.

:param template: A Django template instance
"""
request = self.factory.get(path='/')
rendered_template = nodups_template.render(
asset_vendor = (
'<script src="/static/django_webpack_loader_bundles/vendors.js" >'
'</script>')
asset_app1 = (
'<script src="/static/django_webpack_loader_bundles/app1.js" >'
'</script>')
asset_app2 = (
'<script src="/static/django_webpack_loader_bundles/app2.js" >'
'</script>')
rendered_template = template.render(
context=None, request=request)
used_tags = getattr(request, '_webpack_loader_used_tags', None)

Expand All @@ -572,11 +581,56 @@ def test_skip_common_chunks_djangoengine(self):
self.assertEqual(rendered_template.count(asset_app2), 1)
self.assertEqual(rendered_template.count(asset_vendor), 1)

def test_skip_common_chunks_jinja2engine(self):
"""Test case for deduplication of modules with the Jinja2 engine."""
self.compile_bundles('webpack.config.skipCommon.js')
def _assert_common_chunks_duplicated_jinja2engine(self, view):
"""
Verify that any common chunks between two bundles are duplicated in
the HTML markup.

view = TemplateView.as_view(template_name='home-deduplicated.jinja')
:param view: A Django TemplateView instance
"""
settings = {
'TEMPLATES': [
{
'BACKEND': 'django_jinja.backend.Jinja2',
'APP_DIRS': True,
'OPTIONS': {
'match_extension': '.jinja',
'extensions': DEFAULT_EXTENSIONS + [_OUR_EXTENSION],
}
},
]
}
asset_vendor = (
'<script src="/static/django_webpack_loader_bundles/vendors.js" >'
'</script>')
asset_app1 = (
'<script src="/static/django_webpack_loader_bundles/app1.js" >'
'</script>')
asset_app2 = (
'<script src="/static/django_webpack_loader_bundles/app2.js" >'
'</script>')

with self.settings(**settings):
request = self.factory.get('/')
result = view(request) # type: TemplateResponse
content = result.rendered_content
self.assertIn(asset_vendor, content)
self.assertIn(asset_app1, content)
self.assertIn(asset_app2, content)
self.assertEqual(content.count(asset_vendor), 4)
self.assertEqual(content.count(asset_app1), 2)
self.assertEqual(content.count(asset_app2), 2)
used_tags = getattr(request, '_webpack_loader_used_tags', None)
self.assertIsNotNone(used_tags, msg=(
'_webpack_loader_used_tags should be a property of request!'))

def _assert_common_chunks_not_duplicated_jinja2engine(self, view):
"""
Verify that any common chunks between two bundles are not duplicated in
the HTML markup.

:param view: A Django TemplateView instance
"""
settings = {
'TEMPLATES': [
{
Expand Down Expand Up @@ -612,3 +666,105 @@ def test_skip_common_chunks_jinja2engine(self):
used_tags = getattr(request, '_webpack_loader_used_tags', None)
self.assertIsNotNone(used_tags, msg=(
'_webpack_loader_used_tags should be a property of request!'))

def test_skip_common_chunks_templatetag_djangoengine(self):
"""Test case for deduplication of modules with the django engine via the render_bundle template tag."""
self.compile_bundles('webpack.config.skipCommon.js')

django_engine = engines['django']
dups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" %}')) # type: Template
self._assert_common_chunks_duplicated_djangoengine(dups_template)

nodups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" skip_common_chunks=True %}')
) # type: Template
self._assert_common_chunks_not_duplicated_djangoengine(nodups_template)


def test_skip_common_chunks_templatetag_jinja2engine(self):
"""Test case for deduplication of modules with the Jinja2 engine via the render_bundle template tag."""
self.compile_bundles('webpack.config.skipCommon.js')

view = TemplateView.as_view(template_name='home-deduplicated.jinja')
self._assert_common_chunks_not_duplicated_jinja2engine(view)

def test_skip_common_chunks_setting_djangoengine(self):
"""The global setting should default to False and deduplicate chunks without changing the render_bundle template tag."""
self.compile_bundles('webpack.config.skipCommon.js')

django_engine = engines['django']
dups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" %}')) # type: Template
self._assert_common_chunks_duplicated_djangoengine(dups_template)

loader = get_loader(DEFAULT_CONFIG)
with patch.dict(loader.config, {"SKIP_COMMON_CHUNKS": True}):
self._assert_common_chunks_not_duplicated_djangoengine(dups_template)

def test_skip_common_chunks_setting_jinja2engine(self):
"""The global setting should default to False and deduplicate chunks without changing the render_bundle template tag."""
self.compile_bundles('webpack.config.skipCommon.js')

view = TemplateView.as_view(template_name='home-duplicated.jinja')
self._assert_common_chunks_duplicated_jinja2engine(view)

loader = get_loader(DEFAULT_CONFIG)
with patch.dict(loader.config, {"SKIP_COMMON_CHUNKS": True}):
self._assert_common_chunks_not_duplicated_jinja2engine(view)

def test_skip_common_chunks_setting_can_be_overridden_djangoengine(self):
"""The skip common chunks template tag parameters should take precedent over the global setting."""
self.compile_bundles('webpack.config.skipCommon.js')

django_engine = engines['django']
nodups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" skip_common_chunks=True %}')
) # type: Template
self._assert_common_chunks_not_duplicated_djangoengine(nodups_template)

loader = get_loader(DEFAULT_CONFIG)
with patch.dict(loader.config, {"SKIP_COMMON_CHUNKS": True}):
dups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" skip_common_chunks=False %}')) # type: Template
self._assert_common_chunks_duplicated_djangoengine(dups_template)

def test_skip_common_chunks_setting_can_be_overridden_jinja2engine(self):
"""The skip common chunks template tag parameters should take precedent over the global setting."""
self.compile_bundles('webpack.config.skipCommon.js')

view = TemplateView.as_view(template_name='home-deduplicated.jinja')
self._assert_common_chunks_not_duplicated_jinja2engine(view)

loader = get_loader(DEFAULT_CONFIG)
with patch.dict(loader.config, {"SKIP_COMMON_CHUNKS": True}):
view = TemplateView.as_view(template_name='home-duplicated-forced.jinja')
self._assert_common_chunks_duplicated_jinja2engine(view)

def test_skip_common_chunks_missing_config(self):
"""If the setting is not present we should default to allowing common chunks."""
self.compile_bundles('webpack.config.skipCommon.js')

loader = get_loader(DEFAULT_CONFIG)
# remove SKIP_COMMON_CHUNKS from config completely to test backward compatibility
skip_common_chunks = loader.config.pop('SKIP_COMMON_CHUNKS')

django_engine = engines['django']
dups_template = django_engine.from_string(template_code=(
r'{% load render_bundle from webpack_loader %}'
r'{% render_bundle "app1" %}'
r'{% render_bundle "app2" %}')) # type: Template
self._assert_common_chunks_duplicated_djangoengine(dups_template)

# return removed key
loader.config['SKIP_COMMON_CHUNKS'] = skip_common_chunks
3 changes: 3 additions & 0 deletions webpack_loader/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@
'IGNORE': [r'.+\.hot-update.js', r'.+\.map'],
'LOADER_CLASS': 'webpack_loader.loader.WebpackLoader',
'INTEGRITY': False,
# Whenever the global setting for SKIP_COMMON_CHUNKS is changed, please
# update the fallback value in get_skip_common_chunks (utils.py).
'SKIP_COMMON_CHUNKS': False,
}
}

Expand Down
4 changes: 3 additions & 1 deletion webpack_loader/templatetags/webpack_loader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@
@register.simple_tag(takes_context=True)
def render_bundle(
context, bundle_name, extension=None, config='DEFAULT', suffix='',
attrs='', is_preload=False, skip_common_chunks=False):
attrs='', is_preload=False, skip_common_chunks=None):
if skip_common_chunks is None:
skip_common_chunks = utils.get_skip_common_chunks(config)
tags = utils.get_as_tags(
bundle_name, extension=extension, config=config, suffix=suffix,
attrs=attrs, is_preload=is_preload)
Expand Down
8 changes: 8 additions & 0 deletions webpack_loader/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ def get_loader(config_name):
return _loaders[config_name]


def get_skip_common_chunks(config_name):
loader = get_loader(config_name)
# The global default is currently False, whenever that is changed, change
# this fallback value as well which is present to provide backwards
# compatibility.
return loader.config.get('SKIP_COMMON_CHUNKS', False)


def _filter_by_extension(bundle, extension):
'''Return only files with the given extension'''
for chunk in bundle:
Expand Down