Centralized docker file for all use cases #200

GeoDerp · 2024-02-17T12:26:59Z

One centralized Docker file for Add-on, Standalone & Add-on Testing
This uses a build argument build_version with values: addon, addon-pip, addon-git, addon-local standalone to determine what EMHASS mode you wish to build.
This hopefully helps simplifying testing by having a singular base environment for all docker work.
(Take advantage that requirements.txt exists and are different per repo)
(Still testing).

codecov · 2024-02-17T12:28:49Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.63%. Comparing base (c9e4a20) to head (cb8fb74).
Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #200   +/-   ##
=======================================
  Coverage   87.63%   87.63%           
=======================================
  Files           6        6           
  Lines        1707     1707           
=======================================
  Hits         1496     1496           
  Misses        211      211

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

GeoDerp · 2024-02-17T12:41:26Z

@davidusb-geek , I have tested this as mutch as I could tonight. Published on both repos just in case you want to try it out on the weekend. Just note, its not fully tested. (struggling to test it inside of HA itself at the moment)
As Add-on is using pip and the webserver.py has been changed it's difficult to know if the updated of EMHASS version will work on EMHASS-ADD-ON.

davidusb-geek · 2024-02-17T21:19:49Z

Hi @GeoDerp, I really like this idea.
This seems to nicely improve all this testing.
However before I merge I think that we need to solve the automated docker image deployment using github workflow here: https://github.com/davidusb-geek/emhass/blob/master/.github/workflows/publish_docker.yaml
I think that it will fail as it is now because we need to pass that new --build-arg flag. The problem is that I'm not sure how to pass that flag on the workflow. Do you have any idea?

davidusb-geek · 2024-02-17T21:20:43Z

Please update the branch. After merging #196 there is a conflict on the erased Add-onEmulateDocker file

davidusb-geek · 2024-02-17T21:32:11Z

I think that the build step should be:

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          build-args: build_version=standalone
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

But not completely sure...

GeoDerp · 2024-02-17T22:10:15Z

Oh sure I didn't even think about GitHub Actions sorry. I'll wake up a little bit and help out 👍

GeoDerp · 2024-02-17T23:51:45Z

build-args: build_version=standalone

I think you are right with this. Will have to find out.

GeoDerp · 2024-02-18T01:39:40Z

Just updating the docs now with the updated examples. This will replace the wiki development page.

davidusb-geek · 2024-02-18T10:36:36Z

Ok I tested with docker standalone mode and there is this error:

2024-02-18 10:35:00,575 - web_server - INFO - Status: Optimal
2024-02-18 10:35:00,576 - web_server - INFO - Total value of the Cost function = -6.74
2024-02-18 10:35:00,957 - web_server - INFO -  >> Sending rendered template table data
2024-02-18 10:35:00,957 - web_server - ERROR - Exception on /template/table-template [GET]
Traceback (most recent call last):
  File "<frozen zipimport>", line 213, in get_data
KeyError: 'emhass/templates/template.html'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/jinja2/loaders.py", line 357, in get_source
    source = self._loader.get_data(p)  # type: ignore
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen zipimport>", line 215, in get_data
OSError: [Errno 0] : 'emhass/templates/template.html'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/src/emhass/web_server.py", line 83, in template_action
    template = env.get_template('template.html')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1010, in get_template
    return self._load_template(name, globals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 969, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/loaders.py", line 125, in load
    source, filename, uptodate = self.get_source(environment, name)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/loaders.py", line 359, in get_source
    raise TemplateNotFound(template) from e
jinja2.exceptions.TemplateNotFound: template.html

davidusb-geek · 2024-02-18T10:37:27Z

This is the result on the webui:

GeoDerp · 2024-02-18T11:13:39Z

Ok I tested with docker standalone mode and there is this error:

2024-02-18 10:35:00,575 - web_server - INFO - Status: Optimal
2024-02-18 10:35:00,576 - web_server - INFO - Total value of the Cost function = -6.74
2024-02-18 10:35:00,957 - web_server - INFO -  >> Sending rendered template table data
2024-02-18 10:35:00,957 - web_server - ERROR - Exception on /template/table-template [GET]
Traceback (most recent call last):
  File "<frozen zipimport>", line 213, in get_data
KeyError: 'emhass/templates/template.html'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/jinja2/loaders.py", line 357, in get_source
    source = self._loader.get_data(p)  # type: ignore
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen zipimport>", line 215, in get_data
OSError: [Errno 0] : 'emhass/templates/template.html'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/src/emhass/web_server.py", line 83, in template_action
    template = env.get_template('template.html')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1010, in get_template
    return self._load_template(name, globals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 969, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/loaders.py", line 125, in load
    source, filename, uptodate = self.get_source(environment, name)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/loaders.py", line 359, in get_source
    raise TemplateNotFound(template) from e
jinja2.exceptions.TemplateNotFound: template.html

Sorry @davidusb-geek, only just saw your reply. Ill have a look now

GeoDerp · 2024-02-18T11:14:40Z

Looks like a problem with the new template.html

davidusb-geek · 2024-02-18T11:19:13Z

Yes maybe just need to copy that html fine when building the docker image. Sorry this is related to the PR #196 that was merged..

davidusb-geek · 2024-02-18T11:20:04Z

The optimization actually ran ok.

GeoDerp · 2024-02-18T11:25:34Z

The optimization actually ran ok.

The weird thing is that it works for me.
But my version has issues finding style.css sometimes

GeoDerp · 2024-02-18T11:26:57Z

Ill have a look at manually adding the files in and see if its happy then.

GeoDerp · 2024-02-18T11:58:41Z

Sorry @davidusb-geek I'm stumped. Hit a tired wall. Just committed the in progress documents for the Docker changes. Will have a look at the Docs and that bug in the morning, More then welcome to try and work it out if you like (and edit the docs). Sorry I couldn't get it working by this weekend.

There is defiantly something weird going on with the /src/emhass/static files, I'm sure its something simple, I just cant work it out. (being in this tired state).

bash shell in the Docker container:

GeoDerp · 2024-02-18T12:08:34Z

I'm curious how you go if you try the docker in addon-local mode? If you still get template issue or style issues or neither.
I'm going to check if its a Networking interface blocking thing, or a copy from build thing tomorrow.

davidusb-geek · 2024-02-18T21:09:31Z

Hi. Ok I tested PR #196 merge and everything works fine.
The previous error was effectively just a missing line in the Dockerfile

COPY src/emhass/templates/template.html /app/src/emhass/templates/template.html

I completely tested using the Docker standalone mode and tested the responsiveness on the wbeui, evertything is ok.
I just tested the add-on build using the Add-onEmulateDocker file.
So I increased the core emhass code to v0.7.8.

This current PR #200 introduces a lot of changes on the Dockerfile and also the behavior of the add-on.
For now I don't fully understand how we will be able to integrate averything in one same Dockerfile (add-on included) and the erasing of the run and finish files in the add-on repo.
For the add-on (default mode) is not there a missing RUN command?
I will let you fully test this and when its finished we will merge.

GeoDerp · 2024-02-19T00:45:39Z

Hi. Ok I tested PR #196 merge and everything works fine.
The previous error was effectively just a missing line in the Dockerfile
COPY src/emhass/templates/template.html /app/src/emhass/templates/template.html
I completely tested using the Docker standalone mode and tested the responsiveness on the wbeui, evertything is ok.
I just tested the add-on build using the Add-onEmulateDocker file.
So I increased the core emhass code to v0.7.8.

This current PR #200 introduces a lot of changes on the Dockerfile and also the behavior of the add-on.
For now I don't fully understand how we will be able to integrate averything in one same Dockerfile (add-on included) and the erasing of the run and finish files in the add-on repo.
For the add-on (default mode) is not there a missing RUN command?
I will let you fully test this and when its finished we will merge.

I will refine the documents and fix the bugs and try and do a better job of explaining how the stages work. (And how emhass-add-on will operate)

GeoDerp · 2024-02-19T00:51:19Z

However for a quick reply, Entry point on the add-on default should be the add-ons run.

The run script would have been called by the hidden entry point on the old docker, we are just removing the need to call the run script.

GeoDerp · 2024-02-19T03:40:53Z

sorry @davidusb-geek, there was defiantly some confusion with talking about the old and new versions of the Dockerfile yesterday.
The current dockerfile in PR 200 didnt have a template issue as its passing the files in via directories. (which was why i was confused yesterday).
There is still an issue with the style sheet and images however that I would like to work out.

GeoDerp · 2024-02-19T10:36:37Z

@davidusb-geek , okay i'm relativity happy with the results. Everything seems to work besides pip and git modes that require the git pull of this PR to work with the path changes.

I have changed the path to /app by default as /data was being messed up with the mounts HA does for options.json on emhass-add-on.

Let me know if you need any information on how they the PR works. Feel free to read the develop.md on emhass, and the test.md on emhass-add-on for some info on how to use the changes,

davidusb-geek · 2024-02-20T15:06:13Z

@davidusb-geek , okay i'm relativity happy with the results. Everything seems to work besides pip and git modes that require the git pull of this PR to work with the path changes.

I have changed the path to /app by default as /data was being messed up with the mounts HA does for options.json on emhass-add-on.

Let me know if you need any information on how they the PR works. Feel free to read the develop.md on emhass, and the test.md on emhass-add-on for some info on how to use the changes,

Ok, will merge and test as soon as possible

GeoDerp · 2024-02-20T15:26:34Z

No stress at all 😁. Let me know if you need anything from me about this PR. Like if you want to discuss the comments on emhass-add-on pr.

davidusb-geek · 2024-02-25T18:56:56Z

Testing this right now. Could it be possible to update the wiki contributing part to explain the new testing methods? The different build args options..

davidusb-geek · 2024-02-25T21:07:08Z

Ok nevermind, I just saw that great documentation that you made explaining everything in detail here: docs/develop.md
Many thanks for that, greatly appreciated.

Init centralized docker

f56e444

GeoDerp mentioned this pull request Feb 17, 2024

Centralized docker file for all use cases davidusb-geek/emhass-add-on#77

Merged

GeoDerp added 2 commits February 18, 2024 10:14

Merge remote-tracking branch 'david/master' into centralized-Docker

d44a215

publish docker, modified workspace for new Docker build args

4cda2d0

webserver, try HA token env if url matches

9b3953e

Docker, added documentation

617c0ed

GeoDerp added 2 commits February 19, 2024 03:59

Docker fixed style files not copying

47c8327

Merge branch 'master' into centralized-Docker

0f78b24

GeoDerp added 3 commits February 19, 2024 04:58

spell-check

9ca6261

Docker and webserver, revered to app dir default

87de9c5

Docker tweaked file and docs

b5ab7a7

modified: docs/develop.md

a8bfb30

GeoDerp added 9 commits February 21, 2024 22:06

tweak develop.md

5f04d6c

develop.md, tweak

8be4649

Dockerfile add default to standalone mode

2b76a43

develop, tweak

bb15fba

Dockerfile, fix typo

1759b0b

Docker/Develop, add git option

877d77b

Slight typo fixes

eee05c6

Develop, proof read

0bc6f03

Docker, Develop, backport support

cb8fb74

davidusb-geek approved these changes Feb 25, 2024

View reviewed changes

davidusb-geek merged commit cabb1ee into davidusb-geek:master Feb 25, 2024
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Centralized docker file for all use cases #200

Centralized docker file for all use cases #200

GeoDerp commented Feb 17, 2024 •

edited

codecov bot commented Feb 17, 2024 •

edited

GeoDerp commented Feb 17, 2024 •

edited

davidusb-geek commented Feb 17, 2024

davidusb-geek commented Feb 17, 2024

davidusb-geek commented Feb 17, 2024

GeoDerp commented Feb 17, 2024

GeoDerp commented Feb 17, 2024

GeoDerp commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024 •

edited

GeoDerp commented Feb 18, 2024

GeoDerp commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

GeoDerp commented Feb 18, 2024 •

edited

GeoDerp commented Feb 18, 2024

GeoDerp commented Feb 18, 2024 •

edited

GeoDerp commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

GeoDerp commented Feb 19, 2024

GeoDerp commented Feb 19, 2024

GeoDerp commented Feb 19, 2024

GeoDerp commented Feb 19, 2024 •

edited

davidusb-geek commented Feb 20, 2024

GeoDerp commented Feb 20, 2024

davidusb-geek commented Feb 25, 2024

davidusb-geek commented Feb 25, 2024 •

edited

Centralized docker file for all use cases #200

Centralized docker file for all use cases #200

Conversation

GeoDerp commented Feb 17, 2024 • edited

codecov bot commented Feb 17, 2024 • edited

Codecov Report

GeoDerp commented Feb 17, 2024 • edited

davidusb-geek commented Feb 17, 2024

davidusb-geek commented Feb 17, 2024

davidusb-geek commented Feb 17, 2024

GeoDerp commented Feb 17, 2024

GeoDerp commented Feb 17, 2024

GeoDerp commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024 • edited

GeoDerp commented Feb 18, 2024

GeoDerp commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

GeoDerp commented Feb 18, 2024 • edited

GeoDerp commented Feb 18, 2024

GeoDerp commented Feb 18, 2024 • edited

GeoDerp commented Feb 18, 2024

davidusb-geek commented Feb 18, 2024

GeoDerp commented Feb 19, 2024

GeoDerp commented Feb 19, 2024

GeoDerp commented Feb 19, 2024

GeoDerp commented Feb 19, 2024 • edited

davidusb-geek commented Feb 20, 2024

GeoDerp commented Feb 20, 2024

davidusb-geek commented Feb 25, 2024

davidusb-geek commented Feb 25, 2024 • edited

GeoDerp commented Feb 17, 2024 •

edited

codecov bot commented Feb 17, 2024 •

edited

GeoDerp commented Feb 17, 2024 •

edited

davidusb-geek commented Feb 18, 2024 •

edited

GeoDerp commented Feb 18, 2024 •

edited

GeoDerp commented Feb 18, 2024 •

edited

GeoDerp commented Feb 19, 2024 •

edited

davidusb-geek commented Feb 25, 2024 •

edited