Consolidate frontend #9915

balloob · 2017-10-17T07:17:00Z

Description:

This moves all frontend related static assets to the polymer repo. Building scripts will also be moved to the repo. We will no longer contain a version of the frontend ready to go when doing a git checkout. Frontend will still be included in the PyPi package.

This PR is the first step. Probably some more cleanup PRs will follow after this.

Polymer PR: home-assistant/frontend#468

Breaking change:

Developers only: frontend has been refactored. The method register_panel has been turned into a coroutine function called async_register_panel. The parameter url_path has been renamed to frontend_url_path. For frontend, development, you no longer pass development: 1 to the http component but instead configure the frontend component to be in development mode by pointing it at a local checkout of the Polymer repo:

frontend:
  development_repo: ../home-assistant-polymer

andrey-git · 2017-10-17T18:10:41Z

Will pip3 install -e . used in script/setup still work?

I'm thinking of a developer who just wants to develop backend:
script/setup will make hass point to the git-checked-out-version, but that version now doesn't have FE files.

balloob · 2017-10-18T06:29:49Z

No. One will no longer be able to work on backend stuff without compiling the frontend once.

To make it easier to compile I'll include a Dockerfile. In a future PR we could add a script that takes the latest release from PyPi and extracts the frontend.

balloob · 2017-10-18T06:30:47Z

We're not going to merge these PRs before 0.56 release.

andrey-git · 2017-10-18T07:25:23Z

Will one be able to work on backend stuff if they don't have frontend: in their config?

I thought the intent of this PR was to decouple FE from the BE?

balloob · 2017-10-18T20:24:51Z

One can always work on any backend stuff without loading the frontend. You will just need to depend on tests instead of playing with it in the UI

emlove · 2017-10-18T21:45:08Z

Brainstorming idea:

What if we published the prebuilt polymer files on PyPI as an independent package? It could then be registered in frontend component using the existing REQUIREMENTS system. If the frontend component isn't configured with development: 1 it uses the resources from the PyPI package.

It's obviously messy that we'd have a PyPI package that only contains the polymer code, but we already include the frontend files in our current package. This would just be further separation of the two repos.

rytilahti · 2017-10-18T22:12:13Z

Tbh, I would actually prefer to have some way to get the system running without first fighting quite a bit to get the front-end stuff to compile & working. However, it feels wrong to ship some auxiliary binary package to pypi. Therefore I would encourage that we need find a simple way to get UI bits compiled in a simple manner (wrt. pip install x) instead of going for dependency hacks on some pypi snapshot from the frontend.

In my opinion providing an option to use docker for this is not enough. Just my 0.02e, I applaud the idea of this PR completely so do not take this as a disagreement :-)

emlove · 2017-10-19T02:39:53Z

I'm mainly concerned with raising the barrier to entry for contribution, and personally would prefer some channel to ship the precompiled frontend to developers. If we ballpark by issue/PR count, over 95% of our contributions are to the backend. If people aren't actively developing the frontend, I don't see why we need everyone to rebuild it from source themselves. Especially since we are attempting to isolate the frontend from the backend, it makes sense that we'd move toward shipping it as it's own product.

Docker certainly will make building the frontend easier, but it's a big additional hurdle to people who are often starting their first foray into open source, and are not familiar with all of the tooling. I just want to make sure we carefully consider the ramifications of the final solution on casual developers.

balloob · 2017-10-19T03:52:49Z

I like the idea of publishing it as a pypi package a lot!

balloob · 2017-10-19T06:34:48Z

I've added initial work to convert it to a Python package: home-assistant/frontend@ef7911c

houndci-bot · 2017-10-22T07:44:07Z

homeassistant/components/frontend/__init__.py

+        router.add_route(
+            'get', '/{}'.format(self.frontend_url_path), index_view.get)
+        router.add_route(
+            'get', '/{}/{{extra:.+}}'.format(self.frontend_url_path), index_view.get)


line too long (85 > 79 characters)

houndci-bot · 2017-10-22T07:44:07Z

homeassistant/components/frontend/__init__.py

-        if url not in _REGISTERED_COMPONENTS:
-            hass.http.register_static_path(url, path)
-            _REGISTERED_COMPONENTS.add(url)
+    @callback


too many blank lines (2)

balloob · 2017-10-22T07:48:59Z

@armills last commit has changed it to importing the frontend from a PyPi package. It will not use the PyPi package when you pass in a path to the polymer repo:

frontend:
  development_repo: ../home-assistant-polymer

Will still need to fix the tests.

andrey-git · 2017-10-17T17:49:12Z

homeassistant/components/frontend/__init__.py

+                                   os.path.join(FINAL_PATH, "robots.txt"))
+    hass.http.register_static_path("/static", FINAL_PATH)
+    if hass.http.development:
+        hass.http.register_static_path("/home-assistant-polymer", POLYMER_PATH)


You are now registering both dev & prod paths. Is this on purpose?

andrey-git · 2017-10-17T17:52:17Z

script/bootstrap_frontend

@@ -11,13 +11,6 @@ echo "Bootstrapping frontend..."
 git submodule update --init
 cd homeassistant/components/frontend/www_static/home-assistant-polymer


This is now homeassistant/components/frontend/home-assistant-polymer right?

I thought we're getting rid of the submodule completely, so devs will have this checked out at an arbitrary location they configure. I think we'd want to just remove script/bootstrap_frontend, and the docs will explain how to point to the other repo for development.

andrey-git · 2017-10-17T17:53:17Z

script/release

-python3 setup.py sdist bdist_wheel upload
+script/build_frontend
+
+# python3 setup.py sdist bdist_wheel upload


Should it be uncommented?

andrey-git · 2017-10-17T18:02:30Z

homeassistant/components/frontend/__init__.py

@@ -294,26 +300,6 @@ def reload_themes(_):
                           descriptions[SERVICE_RELOAD_THEMES])


-class BootstrapView(HomeAssistantView):


Why did you remove it?

andrey-git · 2017-10-22T08:00:34Z

homeassistant/components/frontend/__init__.py

 DATA_THEMES = 'frontend_themes'
 DATA_DEFAULT_THEME = 'frontend_default_theme'
 DEFAULT_THEME = 'default'

 PRIMARY_COLOR = 'primary-color'

 # To keep track we don't register a component twice (gives a warning)
-_REGISTERED_COMPONENTS = set()
+# _REGISTERED_COMPONENTS = set()


This, and the comment above is no longer needed, right?

andrey-git · 2017-10-22T08:08:18Z

Looks great!

Now, if I make a FE change and I would like to compile it and check that compiled code works - how do I do that?

balloob · 2017-10-22T08:15:09Z

Good question 🤔

So one way I have been doing it now is to run python3 setup.py sdist in polymer dir to generate a package. Then with HA virtualenv active, pip3 install ../home-assistant-polymer/dist/home-assistant-frontend-20171021.2.tar.gz --upgrade. Then start HASS with hass --skip-pip

I do realize that that's suboptimal. Guess we need another config option to load the compiled version from the repo. Shouldn't be too hard but for the sake of getting this monster merged, can we do that in another PR?

andrey-git · 2017-10-22T09:28:19Z

Yeah this shouldn't block merging this PR

Maybe put a sticky message (not only regarding compiling FE, but about the whole new dev process) in #dev_frontend and #dev chat until this is fixed and new documentation is pushed to the site.

emlove · 2017-10-22T16:33:27Z

Also, brace yourselves for recommits of the polymer submodule in every PR.

houndci-bot · 2017-10-22T18:36:52Z

tests/components/test_panel_iframe.py

            'url_path': 'router'
        }

-        assert self.hass.data[frontend.DATA_PANELS].get('weather') == {
+        assert self.hass.data[frontend.DATA_PANELS].get('weather').as_dict() == {


line too long (81 > 79 characters)

houndci-bot · 2017-10-22T18:36:52Z

tests/components/test_panel_iframe.py

@@ -53,20 +53,20 @@ def test_correct_config(self):
                },
            })

-        assert self.hass.data[frontend.DATA_PANELS].get('router') == {
+        assert self.hass.data[frontend.DATA_PANELS].get('router').as_dict() == {


line too long (80 > 79 characters)

houndci-bot · 2017-10-22T18:36:52Z

tests/components/test_panel_custom.py

 from homeassistant import setup
 from homeassistant.components import panel_custom

-from tests.common import get_test_home_assistant
+from tests.common import get_test_home_assistant, mock_coro, mock_component


'tests.common.get_test_home_assistant' imported but unused

houndci-bot · 2017-10-22T18:36:52Z

homeassistant/components/frontend/__init__.py

+            hass.data[DATA_PANELS].pop(self.frontend_url_path)
+
+
+        self.webcomponent_url = \


too many blank lines (2)

houndci-bot · 2017-10-22T18:36:52Z

tests/components/test_panel_iframe.py

            'url_path': 'router'
        }

-        assert self.hass.data[frontend.DATA_PANELS].get('weather') == {
+        assert self.hass.data[frontend.DATA_PANELS].get('weather').as_dict() == {


line too long (81 > 79 characters)

houndci-bot · 2017-10-22T18:36:52Z

tests/components/test_panel_iframe.py

@@ -53,20 +53,20 @@ def test_correct_config(self):
                },
            })

-        assert self.hass.data[frontend.DATA_PANELS].get('router') == {
+        assert self.hass.data[frontend.DATA_PANELS].get('router').as_dict() == {


line too long (80 > 79 characters)

houndci-bot · 2017-10-22T18:36:52Z

tests/components/test_panel_custom.py

 from homeassistant import setup
 from homeassistant.components import panel_custom

-from tests.common import get_test_home_assistant
+from tests.common import get_test_home_assistant, mock_coro, mock_component


'tests.common.get_test_home_assistant' imported but unused

houndci-bot · 2017-10-22T18:36:52Z

homeassistant/components/frontend/__init__.py

+            hass.data[DATA_PANELS].pop(self.frontend_url_path)
+
+
+        self.webcomponent_url = \


too many blank lines (2)

balloob · 2017-10-22T18:38:31Z

Fixed the tests, this is ready for review.

emlove · 2017-10-22T19:28:27Z

homeassistant/components/frontend/__init__.py

-        register_built_in_panel(hass, panel)
+    # Finalize registration of panels that registered before frontend was setup
+    # This includes the built-in panels from line above.
+    yield from asyncio.wait([finalize_panel(panel) for panel


We should update hass.data[DATA_FINALIZE_PANEL] before this in case any panels register while this is waiting.

emlove · 2017-10-22T19:37:59Z

script/bootstrap_frontend

@@ -11,13 +11,6 @@ echo "Bootstrapping frontend..."
 git submodule update --init
 cd homeassistant/components/frontend/www_static/home-assistant-polymer


I thought we're getting rid of the submodule completely, so devs will have this checked out at an arbitrary location they configure. I think we'd want to just remove script/bootstrap_frontend, and the docs will explain how to point to the other repo for development.

balloob · 2017-10-24T14:28:00Z

@armills @andrey-git shall we merge this?

emlove

Looks great! This is a huge step towards decoupling the frontend!

andrey-git · 2017-10-25T16:17:18Z

http:
  development: 1

is still used for cache control.
Expected?

andrey-git · 2017-10-25T17:22:25Z

When I want to run with FE dev mode I set development_repo in configuration.yaml pointing to home-assistant-polymer checked out from git.
I also run script/bootstrap there. Do I have to run script/build_frontend too?

It seems so, since mdi.html and serviceworker are not in the repo - they are only built by build_frontend.

balloob · 2017-10-25T18:35:43Z

Yes, build the frontend once. Removing Dev option from http component is on todo

homeassistant added the cla-signed label Oct 17, 2017

balloob mentioned this pull request Oct 17, 2017

Consolidate frontend home-assistant/frontend#468

Merged

fabaff added the Hacktoberfest label Oct 17, 2017

balloob force-pushed the consolidate-frontend branch from b631591 to bd171c1 Compare October 21, 2017 22:10

homeassistant added the cla-signed label Oct 21, 2017

balloob requested review from andrey-git and a team as code owners October 22, 2017 07:43

homeassistant added the cla-signed label Oct 22, 2017

houndci-bot reviewed Oct 22, 2017

View reviewed changes

balloob added the breaking-change label Oct 22, 2017

homeassistant added the cla-signed label Oct 22, 2017

balloob added 8 commits October 22, 2017 01:01

Consolidate frontend

6f764c0

Remove home-assistant-polymer submodule

97ed3f3

Convert to using a pypi package for frontend

e2b29c5

fix release script

3449799

Lint

cffa57f

Remove unused file

1da4630

Remove frontend related scripts

2333dc4

Move hass_frontend to frontend REQUIREMENTS

36f694e

balloob force-pushed the consolidate-frontend branch from 96368fb to 36f694e Compare October 22, 2017 08:01

homeassistant added the cla-signed label Oct 22, 2017

andrey-git reviewed Oct 22, 2017

View reviewed changes

Fix tests

041e05c

houndci-bot reviewed Oct 22, 2017

View reviewed changes

lint

6784b5e

emlove reviewed Oct 22, 2017

View reviewed changes

balloob added 3 commits October 22, 2017 23:08

Address comments

a3e98ea

Lint + fix tests in py34

418dd33

Fix py34 tests again

5f52951

emlove approved these changes Oct 24, 2017

View reviewed changes

fix typo

be44f2d

homeassistant added the cla-signed label Oct 24, 2017

andrey-git approved these changes Oct 24, 2017

View reviewed changes

balloob merged commit 2bdad53 into dev Oct 25, 2017

balloob deleted the consolidate-frontend branch October 25, 2017 02:36

balloob mentioned this pull request Nov 2, 2017

0.57 #10293

Merged

choongjoey mentioned this pull request Nov 6, 2017

/api/bootstrap not available #10404

Closed

home-assistant locked and limited conversation to collaborators Mar 2, 2018

		@@ -11,13 +11,6 @@ echo "Bootstrapping frontend..."
		git submodule update --init
		cd homeassistant/components/frontend/www_static/home-assistant-polymer

		@@ -294,26 +300,6 @@ def reload_themes(_):
		descriptions[SERVICE_RELOAD_THEMES])


		class BootstrapView(HomeAssistantView):

		hass.data[DATA_PANELS].pop(self.frontend_url_path)


		self.webcomponent_url = \

Navigation Menu

Consolidate frontend #9915

Consolidate frontend #9915

Conversation

balloob commented Oct 17, 2017 • edited

Description:

andrey-git commented Oct 17, 2017

balloob commented Oct 18, 2017

balloob commented Oct 18, 2017

andrey-git commented Oct 18, 2017

balloob commented Oct 18, 2017

emlove commented Oct 18, 2017

rytilahti commented Oct 18, 2017

emlove commented Oct 19, 2017

balloob commented Oct 19, 2017

balloob commented Oct 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

balloob commented Oct 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrey-git commented Oct 22, 2017

balloob commented Oct 22, 2017

andrey-git commented Oct 22, 2017

emlove commented Oct 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

balloob commented Oct 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

balloob commented Oct 24, 2017

emlove left a comment

Choose a reason for hiding this comment

andrey-git commented Oct 25, 2017

andrey-git commented Oct 25, 2017

balloob commented Oct 25, 2017

balloob commented Oct 17, 2017 •

edited