Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Component autodiscovery #13

Open
mixxorz opened this issue Jul 11, 2022 · 4 comments
Open

Component autodiscovery #13

mixxorz opened this issue Jul 11, 2022 · 4 comments

Comments

@mixxorz
Copy link
Owner

mixxorz commented Jul 11, 2022

Maintaining a yaml file of components is a little annoying. There should be a built-in way to auto-discover components.

I have a few ideas but if you have suggestions, please share. 🙂

@xshapira
Copy link

I'm not an expert in Python or Django, but I wanted to try to contribute. I most likely made various mistakes, yet you can get the idea of what I attempted to achieve.

#slippers.py

import yaml

def autodiscover_components():
    html_templates = get_template(".html")

    for tag_name, template_path in html_templates:

        register.tag(f"{tag_name}", create_component_tag(template_path))

        register.tag(f"#{tag_name}", create_component_tag(template_path))

    template = select_template(["components.yaml", "components.yml"])
    dictionary = {"components": {tag_name: f"'{template_path}/{tag_name}.html'"}}
    with open(template, "w") as yaml_file:
        yaml.dump(dictionary, stream=yaml_file, default_flow_style=False)
#apps.py

import asyncio
from pathlib import Path, PosixPath

from django.apps import AppConfig
from django.core.checks import Warning, register
from django.template.exceptions import TemplateDoesNotExist
from django.template.loader import select_template
from django.utils.autoreload import autoreload_started, file_changed

import yaml

from slippers.templatetags.slippers import autodiscover_components, register_components


def get_components_yaml():
    return select_template(["components.yaml", "components.yml"])


async def autodiscover():
    """Auto-discover components and add to components.yaml"""
    try:
        await asyncio.wait(autodiscover_components())
    except TemplateDoesNotExist:
        pass


async def register_tags():
    """Register tags from components.yaml"""
    try:
        template = get_components_yaml()
        components = yaml.safe_load(template.template.source)
        register_components(components.get("components", {}))
    except TemplateDoesNotExist:
        pass


def watch(sender, **kwargs):
    """Watch when component.yaml changes"""
    try:
        template = get_components_yaml()
        sender.extra_files.add(Path(template.origin.name))
    except TemplateDoesNotExist:
        pass


def changed(sender, file_path: PosixPath, **kwargs):
    """Refresh tag registry when component.yaml changes"""
    if file_path.name == "components.yaml":
        print("components.yaml changed. Updating component tags...")
        register_tags()


def checks(app_configs, **kwargs):
    """Warn if unable to find components.yaml"""
    try:
        get_components_yaml()
    except TemplateDoesNotExist:
        return [
            Warning(
                "Slippers was unable to find a components.yaml file.",
                hint="Make sure it's in a root template directory.",
                id="slippers.E001",
            )
        ]
    return []


class SlippersConfig(AppConfig):
    name = "slippers"

    def ready(self):
        loop = asyncio.get_event_loop()
        loop.run_until_complete(autodiscover())
        loop.close()

        register(checks)

        autoreload_started.connect(watch)
        file_changed.connect(changed)

When I'm running python3 runtests.py I receive the following output:

Creating test database for alias 'default'...
System check identified no issues (0 silenced).
...EEEEEEE........
======================================================================
ERROR: test_kwargs_with_filters (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: '#card'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 75, in test_kwargs_with_filters
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: '#card'. Did you forget to register or load this tag?

======================================================================
ERROR: test_pass_boolean_flags (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: '#button'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 107, in test_pass_boolean_flags
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: '#button'. Did you forget to register or load this tag?

======================================================================
ERROR: test_render_as_variable (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: 'avatar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 96, in test_render_as_variable
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: 'avatar'. Did you forget to register or load this tag?

======================================================================
ERROR: test_render_block_component (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: '#button'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 26, in test_render_block_component
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: '#button'. Did you forget to register or load this tag?

======================================================================
ERROR: test_render_inline_component (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: 'avatar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 15, in test_render_inline_component
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: 'avatar'. Did you forget to register or load this tag?

======================================================================
ERROR: test_render_nested (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: '#card'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 57, in test_render_nested
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: '#card'. Did you forget to register or load this tag?

======================================================================
ERROR: test_render_without_children (tests.test_templatetags.ComponentTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 470, in parse
    compile_func = self.tags[command]
KeyError: 'icon_button'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/maxshapira/Development/public/slippers/tests/test_templatetags.py", line 39, in test_render_without_children
    self.assertHTMLEqual(expected, Template(template).render(Context()))
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 155, in __init__
    self.nodelist = self.compile_nodelist()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 193, in compile_nodelist
    return parser.parse()
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 472, in parse
    self.invalid_block_tag(token, command, parse_until)
  File "/Users/maxshapira/Library/Caches/pypoetry/virtualenvs/slippers-0WtscrBH-py3.10/lib/python3.10/site-packages/django/template/base.py", line 531, in invalid_block_tag
    raise self.error(
django.template.exceptions.TemplateSyntaxError: Invalid block tag on line 2: 'icon_button'. Did you forget to register or load this tag?

----------------------------------------------------------------------
Ran 18 tests in 0.024s

FAILED (errors=7)
Destroying test database for alias 'default'...

This is because the components.yaml is empty. It was done deliberately to check if components.yaml is going to update automatically.

@jnns
Copy link
Contributor

jnns commented Sep 27, 2022

In line with how Django discovers templates by default in <app>/templates/ it would be nice if components could be put in a dedicated folder that slippers would look them up in if they are used as template tags. For example, <app>/templates/components/. The name of the folder for auto-discovery should be configurable via SETTINGS, of course.

@spookylukey
Copy link

spookylukey commented Jun 5, 2023

Here is the code we are using to register components automatically from any directory called components within template directories, with names based on filename e.g. components/foo.html becomes a foo component . This may not be generally applicable, so maybe it needs to be opt in, but it would be nice if this could be a simple setting to enable.

from pathlib import Path

from django.conf import settings
from slippers.templatetags.slippers import register_components

SLIPPERS_SUBDIR = "components"


def register():
    """
    Register discovered slippers components.
    """
    from django.template import engines

    slippers_dirs = []
    for backend in engines.all():
        for loader in backend.engine.template_loaders:
            if not hasattr(loader, "get_dirs"):
                continue
            for templates_dir in loader.get_dirs():
                templates_path = Path(templates_dir)
                slippers_dir = templates_path / SLIPPERS_SUBDIR
                if slippers_dir.exists():
                    register_components(
                        {
                            template.stem: str(template.relative_to(templates_path))
                            for template in slippers_dir.glob("*.html")
                        }
                    )
                slippers_dirs.append(slippers_dir)

    if settings.DEBUG:
        # To support autoreload for `manage.py runserver`, also add a watch so that
        # we re-run this code if new slippers templates are added

        from django.dispatch import receiver
        from django.utils.autoreload import autoreload_started, file_changed

        @receiver(autoreload_started, dispatch_uid="watch_slippers_dirs")
        def watch_slippers_dirs(sender, **kwargs):
            for path in slippers_dirs:
                sender.watch_dir(path, "*.html")

        @receiver(file_changed, dispatch_uid="slippers_template_changed")
        def template_changed(sender, file_path, **kwargs):
            path = Path(file_path)
            if path.exists() and path.is_dir():
                # This happens when new html files are created, re-run registration
                register()

We then call this register() function from within an AppConfig.ready method.

In addition you need an empty components.yaml:

components: {}

@jnns
Copy link
Contributor

jnns commented Jun 6, 2023

This is excellent! I like that component names are derived from the template filenames.

Are there use-cases in which one would want to use a both components.yaml and autodiscovery? If not, the presence of a setting (e.g. SLIPPERS_AUTODISCOVERY_DIR = "components") could switch between the existing behaviour (requiring components.yaml) and auto-discovery mode.

Maybe we should emit warnings when auto-discovery is activated and

  • there are subdirectories within the components folders
    (just as a hint that files in subdirs won't be loaded as components),
  • a filename appears more than once as a component due to multiple apps using components
    (e.g. app_1/components/button.html and app_2/components/button.html).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants