Show frame from another Fugleramme instance

[mtlewis]

First of all thank you for putting together this absolutely fantastic project. It's been such a pleasure working on putting my frame together.

I'm running birdnet-go on a Pi 5, with a separate Pi Zero 2 W connected to an Inky Impression 13.3". I'm finding that the render pipeline is slower than I'd like running on the lower-powered device, so with that in mind I'm thinking it'd be cool to let the beefier Pi 5 handle rendering, with the Zero just managing polling, writing to the display, and maybe allowing for button pushes to update the config on the main frame.

At a high level, I think this will require:

• A new frame.png endpoint (which the remote frame requests from the main frame). Similar role to frame.e6, but intended for consumption by fugleramme itself.
• A new alternate run loop which just polls the remote frame.png and manages the panel
• Updates to the install script to allow selecting this mode of operation, in a separate PR that lands after the new mode is released.
• Ideally, an ability to route button presses on the remote frame back to the main frame. A side benefit of this is it also allows for home automations which manage the config - I'm thinking it'd maybe be nice to have a way to switch modes via home assistant, for example.

I've got most of this working in a fork, with a little bit more testing to do - naturally I'd love to get it back upstream to the main repo! Let me know if you're open to a PR and I'll get it up once I've finished testing.

[arnegiacomo]

Would it be possible to use the already exisiting endpoint for fetching the browser image? /collage.png and dither and draw that on the display? Running two instances seems unecesarrily heavy - I'd rather match the client/server pattern from the https://arnegiacomo.dev/fugleramme/screens/#on-an-external-e-ink-panel

In this case it would probably be a small python script using the inky library polling your remote fugleramme instance. Do you think that would work? Otherwise maybe reusing the e6 endpoint (or creating a sibling) might also work (unsure about the technical details on the client side though).

We should be able to get this to work, so we don't need a beefy pi to drive the Inky displays. I've got 5 zero 2WH coming in the mail - so that I have more hardware to test on.

What do you think?

[mtlewis]

Yep, lifting this out into a sibling example script like the one for ee02s definitely seems like another decent option. I actually started there, but ended up going further down the path of a more deeply-integrated solution. I definitely don't have strong opinions.

What I do think is nice about the more deeply integrated option is that consumers get to use the existing installation solutions for both client and server. Ultimately the client instance ends up super lightweight anyway, since the specialised run loop skips everything that isn't needed. It does mean we're cloning more than we need to, but perhaps a bit of sparse checkout to skip the image assets can make that a bit less painful.

As far as the image itself goes - best I understand it, collage.png isn't an option because it's got some kiosk specific characteristics like the inclusion of the paper texture. We could use frame.e6, but it means a bunch of extra decoding ceremony in the client to get back to an image we can push into inky, and also missing out on compression (in my testing a ~1mb frame.e6 compresses down to about 130kb when delivered as png).

If you're convinced that an example script approach is preferable here, I'm happy to aim in that direction! I'll probably finish testing what I have though, and I'm happy to share the branch for feedback once that's done in case you're interested.

[arnegiacomo]

Yeah you're right about collage.png not working, I didn't think about the details here.

If you have the time and capacity, please do try and see if some kind of example script + endpoint works, I'm fine with us adding an additional endpoint for this - as long as its configurable like the e6 or config driven somehow. I'd like to start here and rule it out if it doesnt work that well, then we could look at the dual instance option and if its possible to do it in a clean way.

I haven't thought much about this problem, so bear with me as I try to carve out a path in the solution-space. Any feedback, observations, findings and inputs highly appreciated!

(I'm not strong on the technical details of these e-Ink displays, so bear with me on the more technical stuff. Being easy to use is one of the main reasons I chose the Inky Impressions)

[mtlewis]

Sure thing, I can take a look in that direction. What are your thoughts about opening up an API to allow for remote button pushes?

[arnegiacomo]

That would be fine. Good expansion to have anyways. Should be generic somehow so it could work on any kind of external display and home made buttons. However I'd reccomend splitting these up as two issues.

Please do create two issues for this, so we can track the work

[mtlewis]

I know you've seen them already, but linking here for reference: #332, #333.

[mtlewis]

PRs are up for the two issues created earlier (#334, #336), along with a draft PR for the thin example client itself in #337. As and when the first two land, I'll retest things end-to-end on a fresh installation and then open #337 for review.

[arnegiacomo]

I've merged 2 of the PRs, beware that these will be included in the next release, and I'm not planning to cut one in the coming days. Maybe when we reach the weekend - unsure

You can ofc check out latest main but an autoupdate wont pull this unless its released, and if youre running this in docker you need to build a new image locally

[mtlewis]

Amazing, thank you so much for the quick turnaround @arnegiacomo. Appreciate the heads up about the release timing.

Just rebased #337 and updated it to reflect the changes made to the previous PRs along the way. It's now open for review. Thanks again for the awesome project.

[arnegiacomo]

And thank you @mtlewis for your kind words, interest and contributions!

[mtlewis]

Was just thinking about the opening up of the /press endpoint - I’m wondering if you considered the case where a frame is set up with an external display but also open to the internet. That seems like a fairly natural thing to do, but it’s probably not desirable to end up in a state where anyone on the internet can trigger button push actions remotely.

[arnegiacomo]

Yeah I get your point @mtlewis. Would be pretty funny if someone online were suddenly rotating your frame while you're looking at it in your living room. (Not dangerous, but maybe pretty unwanted behaviour). Maybe I didn't fully think through my previous choice here.

Sending Authorization: Bearer with every req from the external frame would do the trick right? THen we don't have to do any session management. And the envvar can be fully optional ofc.

An alternative, if you're e.g. behind a reverse proxy or CF tunnel or something similar is blocking the endpoint.

What do you think and prefer? May'be it would be better to move it back behind auth, and re-add the envvar and it would be optional?

Whatever we conclude, I appreciate you taking this up with me. I'd rather people speak up, than be silent and unhappy. There isn't always a perfect reason for the choices and decisions I make and I think that its when we work together we find the best solutions.

[arnegiacomo]

Update: if you think its better to add it back behind auth, feel free to update #337 with both server side and client side changes and just let me know. Much faster response than a back-and-forth, I'll approve either way

[mtlewis]

I don't have super strong feelings here - I agree both your proposals would work, and I definitely agree that having a shared token is preferable to having the external display client impersonate a web session.

If I'm picking from the two options (shared admin token and endpoint blocking), I'd go for the first - mostly because it means users are gonna get the right behavior by default.

I'm happy to update #337 with this change, should have a bit of time to look into that this evening.

[mtlewis]

Oh - just to make sure we're aligned on one thing first: I'm thinking a key benefit of the first option is that it puts users in a state where the right thing happens by default for this somewhat esoteric feature. That means I think we need to aim for a solution where if the admin password is turned on, you have to also be using the token when making requests to the /press endpoint. We don't get the same benefits if the two authentication mechanisms are totally separate. Any reservations about that detail?

Concretely I'm thinking that the server can autogenerate and store a token, and display it on the admin page when an admin token is set.

[arnegiacomo]

I was thinking we could reuse the password for the token tbh, like Authorization: Bearer <password> right? So we dont need an additional API-key. (Most importantly we don't want session management and cookie handling on the non-browser clients ofc)

Would that make sense?

[mtlewis]

Ah, I've got you. Yep, that seems reasonable to me in this context. I've updated #337, more details over there!

[arnegiacomo]

Sweet! I'll check it out on a spare frame in a bit. If everythings checks out, I'll merge and close this discussion before I go to bed 😴

After merge note: This will be included in the next release right, and I'm not planning on cutting one immediately - maybe this weekend? You can ofc checkout main locally if you want to actually deploy this (and depending on how you run this build a new docker image yourself)

[arnegiacomo]

Closing discussion after #337 merge, thank you so much for bringing this up and for the work on implementation and good ideas!