Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error with django-compressor: ValueError: dictionary update sequence element #0 has length 5; 2 is required #414

Closed
JuroOravec opened this issue Mar 29, 2024 · 4 comments · Fixed by #415
Closed

Error with django-compressor: ValueError: dictionary update sequence element #0 has length 5; 2 is required #414

JuroOravec opened this issue Mar 29, 2024 · 4 comments · Fixed by #415

Comments

@JuroOravec
Copy link
Collaborator

JuroOravec commented Mar 29, 2024 •
edited
Loading

Setup

settings.py

INSTALLED_APPS = [
	# ...
    'api',
    'compressor',
]

# Compressor settings
COMPRESS_ROOT = STATIC_BASE_DIR / 'static'
COMPRESS_ENABLED = True
STATICFILES_FINDERS = ('compressor.finders.CompressorFinder',"django.contrib.staticfiles.finders.FileSystemFinder","django.contrib.staticfiles.finders.AppDirectoriesFinder",)

template.html

{% load compress %}
{# other... #}

<!DOCTYPE html>
<html lang="en">
  <head>
    {% compress css %}
      {% component_css_dependencies %}
    {% endcompress %} 
  </head>
  <body>
    ....
  </body>
</html>

Full trace

Traceback (most recent call last):
  File "/Users/presenter/repos/chk/appimage/baseapp/home/helpers/testutil_view.py", line 90, in test_view_cases
    self.do_test_view(case)
  File "/Users/presenter/repos/chk/appimage/baseapp/home/helpers/testutil_view.py", line 99, in do_test_view
    response = self.call_endpoint_for_view_case(case)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/appimage/baseapp/home/helpers/testutil_view.py", line 74, in call_endpoint_for_view_case
    response = self.do_get(endpoint, case)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/appimage/baseapp/home/helpers/testutil_view.py", line 68, in do_get
    response = case.client.get(url)
               ^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/client.py", line 927, in get
    response = super().get(path, data=data, secure=secure, headers=headers, **extra)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/client.py", line 457, in get
    return self.generic(
           ^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/client.py", line 609, in generic
    return self.request(**r)
           ^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/client.py", line 891, in request
    self.check_exception(response)
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/client.py", line 738, in check_exception
    raise exc_value
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/ninja/operation.py", line 400, in _sync_view
    return operation.run(request, *a, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/ninja/operation.py", line 114, in run
    return self.api.on_exception(request, e)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/ninja/main.py", line 490, in on_exception
    return handler(request, exc)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/ninja/errors.py", line 108, in _default_exception
    raise exc  # let django deal with it
    ^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/ninja/operation.py", line 107, in run
    result = self.view_func(request, **values)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
    return view_func(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/appimage/baseapp/home/views/project_outcome.py", line 35, in project_outcome_create_view
    return render_component(
           ^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/appimage/baseapp/theme/helpers/render.py", line 168, in render_component
    content = render_component_to_string(component, *(args or []), **(kwargs or {}))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/appimage/baseapp/theme/helpers/render.py", line 142, in render_component_to_string
    return mark_safe(component_inst.render(norm_context))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/component.py", line 177, in render
    return template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/utils.py", line 112, in instrumented_test_render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/templatetags/component_tags.py", line 402, in render
    rendered_component = component.render(context)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/component.py", line 177, in render
    return template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/utils.py", line 112, in instrumented_test_render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/templatetags/component_tags.py", line 402, in render
    rendered_component = component.render(context)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/component.py", line 177, in render
    return template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/utils.py", line 112, in instrumented_test_render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/templatetags/component_tags.py", line 402, in render
    rendered_component = component.render(context)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/component.py", line 177, in render
    return template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/test/utils.py", line 112, in instrumented_test_render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/templatetags/compress.py", line 160, in render
    return self.render_compressed(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/templatetags/compress.py", line 131, in render_compressed
    rendered_output = compressor.output(mode, forced=forced, basename=file_basename)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/js.py", line 49, in output
    ret.append(subnode.output(*args, **kwargs))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/js.py", line 51, in output
    return super().output(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/base.py", line 374, in output
    return self.handle_output(mode, filtered_output, forced, basename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/base.py", line 382, in handle_output
    return output_func(mode, content, forced, basename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/base.py", line 395, in output_file
    return self.render_output(mode, {"url": url})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/compressor/base.py", line 426, in render_output
    final_context = self.context.flatten()
                    ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/presenter/repos/chk/.venv/lib/python3.11/site-packages/django_components/component.py", line 172, in flatten
    flat.update(d)
ValueError: dictionary update sequence element #0 has length 5; 2 is required

What I've learnt so far:

  1. It seems like this issue is between django-compressor and django-components media dependencies. Because when I remove either of the two, the issue resolves:

    a) remove compress block:

    {% load compress %}
{# other... #}

<!DOCTYPE html>
<html lang="en">
  <head>
      {% component_css_dependencies %}
  </head>
  <body>
    ....
  </body>
</html>

    b) remove component_css_dependencies block:

    {% load compress %}
{# other... #}

<!DOCTYPE html>
<html lang="en">
  <head>
    {% compress css %}
    {% endcompress %} 
  </head>
  <body>
    ....
  </body>
</html>

  2. The issue relates to the Django's Context class. The actual error occurs in Django, when it calls Context.flatten. My understanding is that one of the dicts in context is actually a nested Context instance, and Django cannot work with that, it expects plain dicts.

  3. Workaround - I wasn't able to find out what's the root cause, but I was able to find 2 ways to resolve this issue:

    a. Resolve nested Contexts on django's side in django/template/context.py

    (Note: my issue is in django v4.2, but in v5.0, the flaten function is the same)

    class BaseContext:
	...
    def flatten(self):
        """
        Return self.dicts as one dictionary.
        """
        flat = {}
        for d in self.dicts:
            if isinstance(d, self.__class__):
                d = d.flatten()
            flat.update(d)
        return flat

    b. Resolve nested Contexts on django-component's side in django-components/component.py

        with context.update({FILLED_SLOTS_CONTENT_CONTEXT_KEY: updated_filled_slots_context}):
        def flatten(self):
            flat = {}
            for d in self.dicts:
                if isinstance(d, self.__class__):
                    d = d.flatten()
                flat.update(d)
            return flat
        
		# Modify the `Context.flatten` method for this Context instance
        import types
        context.flatten = types.MethodType(flatten, context)

        return template.render(context)

Conclusion:

I'd rather fix the underlying cause, if someone knows what it might be, please help. But if we won't be able to get to the root of it, we can deploy a quick workaround on django-components's side.
@JuroOravec JuroOravec changed the title Compress Error with django-compressor: ValueError: dictionary update sequence element #0 has length 5; 2 is required Mar 29, 2024
@EmilStenstrom
Copy link
Owner

Related discussion with the same error: #411

@EmilStenstrom
Copy link
Owner

EmilStenstrom commented Mar 29, 2024 •
edited
Loading

In component.Component.render we wrap context_data in Context(). Could that be the root cause? It was added in this commit 91b4acc#diff-569e56c0c056d2ede4e534ad3cae3717ff8e61ba7fdd6963a23df1adc8fd86d1R150 when we added support for rendering component as views.

@JuroOravec
Copy link
Collaborator Author

I was thinking of that too (not wrapping the context_data in Context), but didn't explore it further. I'll give it a try tomorrow

@JuroOravec JuroOravec mentioned this issue Mar 30, 2024
Merged
@JuroOravec
Copy link
Collaborator Author

Ok, so the issue was with Context, turns out that the context_data in component.Component.render may already be an instance of Context, so a simple check for that fixed it for me. 🎉

See #415

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: avoid nested django contexts when rendering JuroOravec/django-components
2 participants
@EmilStenstrom @JuroOravec