Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add runtime online image support #4710

Open
wants to merge 14 commits into
base: dev
Choose a base branch
from

Conversation

guillempages
Copy link
Contributor

@guillempages guillempages commented Apr 19, 2023

What does this implement/fix?

This PR implements the possibility of downloading PNG images at runtime and showing them in a compatible display.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Other

Related issue or feature (if applicable): fixes esphome/feature-requests#1463

Pull request in esphome-docs with documentation (if applicable): esphome/esphome-docs#2076

Test Environment

  • ESP32
  • ESP32 IDF
  • ESP8266
  • RP2040

Example entry for config.yaml:

# Example config.yaml
esphome:
  # ...
  on_boot:
    then:
      - wait_until: api.connected
      - component.update: example_image

online_image:
  - url: "https://www.example.org/image.png"
    id: example_image
    type: RGB565
    on_download_finished:
      - component.update: my_display
    on_error:
      - logger.log: "Could not download the image"

display:
  id: my_display
  # ...
  lambda: |-
    it.image(0, 0, id(example_image));

Checklist:

  • The code change is tested and works locally.
  • Tests have been added to verify that the new code works (under tests/ folder).

If user exposed functionality or configuration variables are added/changed:

@probot-esphome
Copy link

Hey there @guillempages,
Thanks for submitting this pull request! Can you add yourself as a codeowner for this integration? This way we can notify you if a bug report for this integration is reported.
In __init__.py of the integration, please add:

CODEOWNERS = ["@guillempages"]

And run script/build_codeowners.py

(message by NeedsCodeownersLabel)

@osos
Copy link

osos commented Apr 20, 2023

If I understand correct, the image is only downloaded when updated?

I have a dynamic image, which I which to download and display every minute or less. I suppose I now need to update then display ?

Feature suggestion: Event on update/download finished, e.g. for a call to display the image once downloaded.

@guillempages
Copy link
Contributor Author

guillempages commented Apr 20, 2023

@osos yes, the image needs to be updated and then the display as well.
I also had thought about this callback, but then thought it's easier/shorter to just do

 - component.update: my_image
 - component.update: my_display

than

  - component.update: my_image
# ...
on_image_downloaded:
  - component.update: my_display

Of course, I forgot about the "update_interval" case :-( That would be a "valid" use case for the on_downloaded callback. Neverhteless, I'd rather focus on getting it working good first (there are still sometimes issues when decoding that I haven't been able to track down / fix). There is already a workaround for your use case: Instead of using update_interval, you could define an

interval:
  - interval: 1min
    then:
    - component.update: my_image
    - component.update: my_display

and would achieve the same.

@peralta
Copy link

peralta commented Apr 27, 2023

I am unable to get esphome to build using the components in this PR (I was against PR #3255). This is the relevant config part:

external_components:
  - source: github://pr#4710
    components: [ online_image, image, display ]
online_image:
  - url: "https://www.example.org/image.png"
    id: example_image

Which fails with

Failed config

online_image: [source ll-test.yaml:70]
  
  Component not found: online_image.
  - url: https://www.example.org/image.png
    id: example_image

I have tried different things to get it to compile, but in the end I have realised that just changing the PR number to 3255 works and 4710 does not. Any idea of what might the problem be?

@peralta
Copy link

peralta commented Apr 28, 2023

Found the issue: missing dependencies and esphome silently eating the exception. Missing dependencies in the official docker container are reportlab and svglib.

@guillempages
Copy link
Contributor Author

guillempages commented May 11, 2024

@DominoLeChat Sorry, I have no idea what could possibly be wrong with that combination :-( I don't have a waveshare that I could try on.
Do you have the logs from before the crash? So that at least I could see where in the image component it is crashing? (if it is crashing there)

One thing that comes to mind, if I am reading your configuration right, your screen is a 3-color (white, black, red) one, right? Do you know how the pixels are being drawn to that screen? Or which format do images use to draw? It might be a mismatch between what the custom component driver expects as a color format, and what the online_image supports.

You might want to try (just to see if this helps) other pixel formats (type: TRANSPARENT_BINARY type: RGB565 ... on the online_image).

@ianByrne
Copy link

Hello - I am running into the following error when trying to install.

external_components:
  - source: github://pr#4710
    components: [ "online_image" ]
In file included from src/esphome/components/online_image/image_decoder.cpp:4:
src/esphome/components/online_image/online_image.h:79:3: error: 'ESPHOME_ALWAYS_INLINE' does not name a type; did you mean 'ALWAYS_INLINE'?
   ESPHOME_ALWAYS_INLINE bool auto_resize_() const { return fixed_width_ == 0 || fixed_height_ == 0; }
   ^~~~~~~~~~~~~~~~~~~~~
   ALWAYS_INLINE
*** [.pioenvs/bedroom-lilygo-display/src/esphome/components/online_image/image_decoder.cpp.o] Error 1
In file included from src/esphome/components/online_image/online_image.cpp:3:
src/esphome/components/online_image/online_image.h:79:3: error: 'ESPHOME_ALWAYS_INLINE' does not name a type; did you mean 'ALWAYS_INLINE'?
   ESPHOME_ALWAYS_INLINE bool auto_resize_() const { return fixed_width_ == 0 || fixed_height_ == 0; }
   ^~~~~~~~~~~~~~~~~~~~~
   ALWAYS_INLINE
src/esphome/components/online_image/online_image.cpp: In member function 'bool esphome::online_image::OnlineImage::resize_(int, int)':
src/esphome/components/online_image/online_image.cpp:143:7: error: 'auto_resize_' was not declared in this scope
   if (auto_resize_()) {
       ^~~~~~~~~~~~
src/esphome/components/online_image/online_image.cpp:143:7: note: suggested alternative: 'resize_'
   if (auto_resize_()) {
       ^~~~~~~~~~~~
       resize_
*** [.pioenvs/bedroom-lilygo-display/src/esphome/components/online_image/online_image.cpp.o] Error 1

@guillempages
Copy link
Contributor Author

@ianByrne sorry about that error. I did a fix for the beta, and somehow forgot, that the release is not yet out 🫤.

When the next release is released (should be on Wednesday if I am not mistaken) the issue will be automatically fixed.

Meanwhile I'll try to do some hack tomorrow, to see if I can make it work again on the previous versions as well.

@DohmenICT
Copy link

DohmenICT commented May 30, 2024

Hi!

This still doesn't work for me although my esphome is on 2024.5.4
This is my code:

external_components:
  - source: github://guillempages/esphome@online_image_buffer
    components: [online_image]

online_image:
  - url: "https://office.******.nl/EnergyPrices.png"
    id: energypricesimage
    on_download_finished:
      - component.update: epaperdisplay

and these are the errors I get when compiling:

INFO ESPHome 2024.5.4
INFO Reading configuration /config/esphome/epaper1.yaml...
INFO Generating C++ source...
INFO Compiling app...
Processing epaper1 (board: esp01_1m; framework: arduino; platform: platformio/espressif8266@4.2.1)
--------------------------------------------------------------------------------
HARDWARE: ESP8266 80MHz, 80KB RAM, 1MB Flash
Dependency Graph
|-- ESPAsyncTCP-esphome @ 2.0.0
|-- ESPAsyncWebServer-esphome @ 3.2.2
|-- DNSServer @ 1.1.1
|-- ESP8266WiFi @ 1.0
|-- ESP8266mDNS @ 1.2
|-- SPI @ 1.0
|-- ESP8266HTTPClient @ 1.2
|-- pngle @ 1.0.2
Compiling .pioenvs/epaper1/src/esphome/components/online_image/image_decoder.cpp.o
Compiling .pioenvs/epaper1/src/esphome/components/online_image/online_image.cpp.o
Compiling .pioenvs/epaper1/src/esphome/components/online_image/png_image.cpp.o
Compiling .pioenvs/epaper1/src/esphome/components/spi/spi_arduino.cpp.o
src/esphome/components/online_image/png_image.cpp: In member function 'virtual int esphome::online_image::PngDecoder::decode(uint8_t*, size_t)':
src/esphome/components/online_image/png_image.cpp:54:5: error: 'ESP_LOGD' was not declared in this scope
   54 |     ESP_LOGD(TAG, "Waiting for data");
      |     ^~~~~~~~
src/esphome/components/online_image/png_image.cpp:59:5: error: 'ESP_LOGE' was not declared in this scope; did you mean 'USE_LOGGER'?
   59 |     ESP_LOGE(TAG, "Error decoding image: %s", pngle_error(pngle));
      |     ^~~~~~~~
      |     USE_LOGGER
Compiling .pioenvs/epaper1/src/esphome/components/spi/spi_esp_idf.cpp.o
src/esphome/components/online_image/image_decoder.cpp: In member function 'uint8_t* esphome::online_image::DownloadBuffer::data(size_t)':
src/esphome/components/online_image/image_decoder.cpp:29:5: error: 'ESP_LOGE' was not declared in this scope; did you mean 'USE_LOGGER'?
   29 |     ESP_LOGE(TAG, "Tried to access beyond download buffer bounds!!!");
      |     ^~~~~~~~
      |     USE_LOGGER
*** [.pioenvs/epaper1/src/esphome/components/online_image/png_image.cpp.o] Error 1
*** [.pioenvs/epaper1/src/esphome/components/online_image/image_decoder.cpp.o] Error 1
src/esphome/components/online_image/online_image.cpp: In member function 'virtual void esphome::online_image::OnlineImage::update()':
src/esphome/components/online_image/online_image.cpp:357:33: error: call to 'HTTPClient::begin' declared with attribute error: obsolete API, use ::begin(WiFiClient, url)
  357 |   int begin_status = http_.begin(url_);
      |                      ~~~~~~~~~~~^~~~~~
*** [.pioenvs/epaper1/src/esphome/components/online_image/online_image.cpp.o] Error 1
========================== [FAILED] Took 3.09 seconds ==========================

Add the possibility of freeing the memory used by an online image
 - Add follow_redirects option

Allow enabling/disabling HTTP follow redirect option.
Automatically end the download loop when the image has been fully downloaded.

 - Add ETag support

Avoid re-downloading an image, if the server says that the image hasn't changed.
Also add an action to release the used buffer.

 - Allow setting UserAgent
 - Allow setting HTTP timeout value
Force esp8266 >= 2.7.0 or esp32 framework
Add callbacks that will be called when an image is successfully downloaded
or an error happened during download.
@guillempages
Copy link
Contributor Author

@DohmenICT Your problem is a different one. You are using an ESP8266, and I have actually never tested the code there. So it does not even compile 😕
There was some half-hearted intent to support it (because I originally based my code on the http_request component; which does support it).

I have updated the code now to support (at least compiling) on ESP8266, but I am not really sure that it will work as you expect; probably you will run into issues because of low performance or low memory.

Please, do test it with the latest version, and let me know whether it works at all.

@guillempages guillempages force-pushed the online_image_buffer branch 2 times, most recently from 68e9470 to 093648b Compare June 5, 2024 22:32
@DohmenICT
Copy link

Hi, thanks for your feedback. The code compiles now and I can upload it. But I get a weird error where the complete url of the image isn't shown in the logging:

[08:42:32][I][online_image:375]: Updating image
[08:42:37][E][online_image:408]: Could not download image from Ԛ\xff?,. Error code: -5
[08:43:00][D][main:100]: Could not download the image

This is my code:

esphome:
  name: epaper1
  friendly_name: epaper1
  on_boot:
    priority: -10       
    then:
      - delay: 10s
      - component.update: energypricesimage
      - component.update: epaperdisplay


interval:
  - interval: 1min
    then:
    - component.update: energypricesimage
    - component.update: epaperdisplay

external_components:
  - source: github://guillempages/esphome@online_image_buffer
    components: [online_image]
    refresh: 0s

online_image:
  - url: "https://office.xxx.nl/EnergyPrices.png"
    id: energypricesimage
    on_download_finished:
      - component.update: epaperdisplay
    on_error:
      - logger.log: "Could not download the image"

esp8266:
  board: esp01_1m

# Enable logging
logger:
    level: DEBUG

api:

ota:
  password: !secret otaPassword

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  use_address: 192.168.169.54
  ap:
    ssid: "Epaper1 Fallback Hotspot"
    password: !secret fallbackHotSpot

captive_portal:
    
spi:
  clk_pin: 14
  mosi_pin: 13  #DIN
 
display:
  - platform: waveshare_epaper
    id: epaperdisplay
    cs_pin: 15
    dc_pin: 12
    update_interval: 300s
    busy_pin: 16
    reset_pin: 4
    model: 4.20in-bV2
    lambda: |-
      it.image(0,0,id(energypricesimage));

and this is the image that is hosted:
image

@DohmenICT
Copy link

saw the new commit; testing....

@DohmenICT
Copy link

Still something wrong with the url in the logging...

[I][online_image:375]: Updating image
[E][online_image:408]: Could not download image from $��?/. Error code: -4
[D][main:098]: Could not download the image

# Online Image specific options
#
cv.Optional(CONF_FORMAT, default="PNG"): cv.enum(IMAGE_FORMAT, upper=True),
cv.Optional(CONF_BUFFER_SIZE, default=2048): cv.int_range(256, 65536),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My screen is a 13.3 inch eink, which are 960x680 which is 81600 bytes. :(

can we increase this to 98304 maybe ?

PR for adding the support for 13.3inch screen size to esphome: #6443

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to increase that? This is not the buffer to hold the decoded image; this is the temporary buffer that is used to download the image chunks before decoding.
The image is downloaded in chunks of this size and decoded on the fly.

Do you have an actual problem with using the default size of the buffer? This option was actually added at some point to debug an issue someone else was having, but it is not intended to be used except in some very special cases, where the chunk downloading does not work.

Copy link
Contributor

@pgericson pgericson Jun 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might have tried to increase the buffer size to get past the error i got, but that might just have been me not having enough free Heap Size for storing and resizing the image. Let me resolve this, there is not resolve button on this repo.

[17:28:10][D][main:456]: downloaded image
[17:28:10][W][component:237]: Component esphome.coroutine took a long time for an operation (107 ms).
[17:28:10][W][component:238]: Components should block for at most 30 ms.
[17:28:10][D][online_image:175]: Allocating new buffer of 81600 Bytes...
[17:28:10][E][online_image:190]: allocation failed. Biggest block in heap: 48116 Bytes
[17:28:10][E][online_image.png:061]: Error decoding image: No IHDR chunk is found
[17:28:10][E][online_image:225]: Error when decoding image.
[17:28:10][D][main:449]: Error downloading image
[17:28:12][D][sensor:094]: 'Heap Free': Sending state 145268.00000 B with 0 decimals of accuracy
[17:28:12][D][sensor:094]: 'Heap Max Block': Sending state 98292.00000 B with 0 decimals of accuracy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "there is no resolve button"? Do you mean that this conversation should be resolved and you don't see the button? I see it, but maybe because I am the owner of the PR ;-)

Or do you mean that the issue is not resolved? Your log looks strange; the biggest reported block is different for the online_image and for the debug sensor, but I actually copied the max block reading code from the debug sensor 🤔

Does your device have PSRAM and if so, is it enabled and working? Images do need a lot of memory, so PSRAM is almost mandatory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "there is no resolve button"? Do you mean that this conversation should be resolved and you don't see the button? I see it, but maybe because I am the owner of the PR ;-)

yeah, its just my view and this thread I started should probably be resolved, as it might not be an issue with your code.

Or do you mean that the issue is not resolved? Your log looks strange; the biggest reported block is different for the online_image and for the debug sensor, but I actually copied the max block reading code from the debug sensor 🤔

This might be a product of the debug code running every X seconds, and thereby not reading the memory at the time of running the pngle_feed function. So don't read too much into it.

auto fed = pngle_feed(pngle, buffer, size);
if (fed < 0) {
ESP_LOGE(TAG, "Error decoding image: %s", pngle_error(pngle));

Does your device have PSRAM and if so, is it enabled and working? Images do need a lot of memory, so PSRAM is almost mandatory.

I don't have one with PSRAM that i can get to work right now.

and thanks for all the work on this PR!

but you can resolve this thread, as I won't be able to debug any more for the time being.

ESP8266 was not compilint. Fixed it and added a test for it.
@guillempages
Copy link
Contributor Author

guillempages commented Jun 9, 2024

Still something wrong with the url in the logging...

[I][online_image:375]: Updating image
[E][online_image:408]: Could not download image from $��?/. Error code: -4
[D][main:098]: Could not download the image

Oops, my bad. Should be fixed now. And I've changed the error code to a string as well, to make it better understandable.

@DohmenICT
Copy link

Well! The URL now shows in the error message.... But the error message doesn't :-D

[19:53:52][I][app:100]: ESPHome version 2024.5.5 compiled on Jun 11 2024, 19:52:40
[19:53:52][C][wifi:580]: WiFi:
[19:53:52][C][wifi:408]: Local MAC: 3C:61:05:DC:7F:E0
[19:53:52][C][wifi:413]: SSID: [redacted]
[19:53:52][C][wifi:416]: IP Address: 192.168.168.234
[19:53:52][C][wifi:419]: BSSID: [redacted]
[19:53:52][C][wifi:421]: Hostname: 'epaper1'
[19:53:52][C][wifi:423]: Signal strength: -76 dB ▂▄▆█
[19:53:52][C][wifi:427]: Channel: 6
[19:53:52][C][wifi:428]: Subnet: 255.255.255.0
[19:53:52][C][wifi:429]: Gateway: 192.168.168.1
[19:53:52][C][wifi:430]: DNS1: 192.168.168.1
[19:53:52][C][wifi:431]: DNS2: 0.0.0.0
[19:53:52][C][logger:185]: Logger:
[19:53:52][C][logger:186]: Level: DEBUG
[19:53:52][C][logger:188]: Log Baud Rate: 115200
[19:53:52][C][logger:189]: Hardware UART: UART0
[19:53:52][C][spi:068]: SPI bus:
[19:53:52][C][spi:069]: CLK Pin: GPIO14
[19:53:52][C][spi:070]: SDI Pin:
[19:53:52][C][spi:071]: SDO Pin: GPIO13
[19:53:52][C][spi:076]: Using HW SPI: SPI
[19:53:52][C][waveshare_epaper:1904]: Waveshare E-Paper
[19:53:52][C][waveshare_epaper:1904]: Rotations: 0 °
[19:53:52][C][waveshare_epaper:1904]: Dimensions: 400px x 300px
[19:53:52][C][waveshare_epaper:1905]: Model: 4.2in (B V2)
[19:53:52][C][waveshare_epaper:1906]: Reset Pin: GPIO4
[19:53:52][C][waveshare_epaper:1907]: DC Pin: GPIO12
[19:53:52][C][waveshare_epaper:1908]: Busy Pin: GPIO16
[19:53:52][C][waveshare_epaper:1909]: Update Interval: 300.0s
[19:53:52][C][captive_portal:088]: Captive Portal:
[19:53:52][C][mdns:115]: mDNS:
[19:53:52][C][mdns:116]: Hostname: epaper1
[19:53:52][C][ota:096]: Over-The-Air Updates:
[19:53:52][C][ota:097]: Address: epaper1.local:8266
[19:53:52][C][ota:100]: Using Password.
[19:53:52][C][ota:103]: OTA version: 2.
[19:53:52][W][ota:106]: Last Boot was an unhandled reset, will proceed to safe mode in 8 restarts
[19:53:52][C][api:139]: API Server:
[19:53:52][C][api:140]: Address: epaper1.local:6053
[19:53:52][C][api:144]: Using noise encryption: NO
[19:54:00][I][online_image:375]: Updating image
[19:54:04][E][online_image:409]: Could not download image from https://office.xxxxx.nl/EnergyPrices.png. Error: \xb2�
[19:54:04][D][main:098]: Could not download the image
[19:54:04][W][component:237]: Component esphome.coroutine took a long time for an operation (4474 ms).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow esphome to load image from URL