-
-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker support #211
Comments
What synchronicity!! I just posted this discussion for needs, and at the top is . . . I want to help however I can!!! I know that this thing is not put together the way it would were I to be anything other than a hobbyist when it comes to developing a web application, so I will likely be of most value just by fixing things I've done so that they're not bad and to make things as easy for you as I can -- please just let me know how to help and I will! |
At this point the simplest thing to do is emulate an entire Raspberry Pi image and then throw that into a container registry, but it's quite bloaty (so far my build is taking more than 20 mins 😿) If you're interested and you want to rewrite the entire web-end in Laravel, and then just querying the existing 'server.py' service internally, you'd get a lot of support from me ;) |
So far the initial issues are as you described in #104, the pre-built TFLite binaries are only for aarch64. This part of the stack I'm not too familiar with. How feasible is it to get similar binaries for x86_64 architecture? If so we can dump a lot of the 'build' work to GitHub Actions. |
I've managed to get a Docker image build running, https://github.com/olipayne/BirdNET-Pi-Docker/runs/6085934891 will leave that for tonight and get back to it tomorrow 🥳 |
Pretty easy, but I think maybe the easiest thing to do would be to try adjusting the |
Hey this is just a comment but imo, the cleanest way to "dockerize" this would be to refactor the whole thing into docker-compose and have each service (db, ui, server, analysis, recording, etc) running in a different container. There are lots of things happening during BirdNET-Pi installation (new files in /usr/local/bin, scripts to add custom shebangs, scripts writing templates, etc). These hacks complicate the maintenance and updates. All of this would be much simpler when using docker images. This is not an easy task if you've never worked with containers. I am not saying dockerizing is the way to go. But if you want to do it properly, I would suggest to refactor at the same time instead of installing into docker image based on raspberry-pi image. By the way, BalenaOS might be worth looking into to deploy docker containers on a raspberry-pi. |
@olipayne I found an x86_64 whl online https://github.com/maxogden/BirdNET-Pi/blob/main/tflite_runtime-2.6.0-cp39-cp39-linux_x86_64.whl but I am not sure if it works... I tried building your Dockerfile locally and I think because |
@lpasselin @olipayne former balena employee here happy to help with the containerization push! I actually tried to deploy this today before realizing it was more complicated than I originally thought. It can certainly be done without much balena lockin (generic docker-compose and service definitions), but the open fleets stuff makes it stupid simple to deploy as well. Again I'm no longer affiliated with balena, just happy to help to make this easier to get up and running! |
Since this entire project is built on the premise that it's running on a Raspberry Pi of some sort, it might be a bigger job than initially expected. I'm looking into seeing what can be chopped up so that in theory, this entire project could run in separate places (e.g., recording on a Pi somewhere, sending the files to a more powerful machine for analysis and webpage hosting) |
Quick update, I'm separating out the Tensorflow stuff first, it seems like the easiest route for now is to build a dedicated image per CPU arch, since there are a lot of hardware level optimisations you miss out on if you build an all-in-one docker image. I've got the Feel free to post to it as much as you like, it'll just slow down but it wont crash anything else. Naturally, this comes with a privacy warning, you'd be sending raw audio to a third party (me). Once I've got a bit more running, I'll stitch together a docker-compose file so you can host this yourself. |
Getting this running on a container platform + RTSP support would mean that people with home NAS devices etc could run the solution with existing security cameras without having to try and source an RPi for sale. Container support brings a whole new set of deployment options to the table and I'm pretty stoked that it's being worked out! |
I'm going to give this an attempt as it will give me the opportunity to learn about docker. |
@supajason is on the right track, using https://github.com/kahst/BirdNET-Analyzer/blob/main/Dockerfile (slightly modified to run server.py though) and then forking everything else over to a normal 'web app' seems like the right way forward. |
If this could be modularized further, I could see an end result that you could plug in many different models, with applications outside of bird identification. I've always wanted to be able to identify sounds & trilaterate them in 3D space using an array of microphones. Similar to gunshot detection that cities use. |
Hi @mgottholsen! I just wanted to show you this since you might be interested in their findings. My best, |
@mcguirepr89 This is really awesome stuff! IT conceptually overlaps with some other stuff I've been looking at Rhasspy voice assistant satellite devices that run on similarly specc'ed hardware. The multilateration is a killer feature as well. I noticed you helped them with their setup, were there large differences in their implementation that would make sense here? |
@mgottholsen My contributions to that DTU project were not around any of the cool engineering, unfortunately -- I just made myself available to them, helped get the installation working, answered questions, etc., and was able to make little adjustments here and there to make things easier for their needs. The system can be noded without too much hassle so long as you know a little bit about networking. Also, since the system can run on the $15 Raspberry Pi 0W2, a triangle of nodes could be ~$45 + $cost of (3)three USB mics, and (3)three USB-A to micro-USB adapters. Do you have any recommendations on some 101, introductory type sources for getting started in being able to gather the info needed to start tinkering with this? |
Unfortunately I'm not the best person to get you started on something like this, I have limited experience in implementation for multilateration, although I did work on a pilot to dispatch drones based off the X/Y results from trilateration/multilateration. There is a good Github reference in that whitepaper you linked that I think can show how the technique can work - https://github.com/jurasofish/multilateration |
+1 for breaking out the different services into different containers. The way I would refactor this (I am not a PHP person so probably not much help in actually doing it):
I would love to run this as I have a lot of Docker/server space and a lot of security cameras, but no RPIs. If you need a testing subject and want some help testing things out, let me know. I think this would be a really neat project to throw up for the fun. |
Audio capture is a bit of sticking point. I don't know enough about docker for USB/device recording. |
I think most people using Docker/containers will likely be using RTSP sources. For the RPI instance, that is likely something you would want to keep on the host machine. The only service on the host machine to record off audio to a folder that docker can then read from. You can try to mount the audio device into the container, but that is likely more trouble than it is worth. |
So I have the analyse and stream_capture containers running. |
@supajason that sounds great! If on another repo, please let us know how to find it and I will do my best to spotlight the repo for folks interested in what you have working. 👍 |
Has anyone used the docker version? I've got it up and running but a 10 second clip is taking 12 seconds to analyse which isn't going to work. |
I am going to give it a go over the next weekend if my Pi0W2 stabilises. I am trying it out on a Rockchip A55 machine but have poor detection results on that device. |
Could you please point me to a guide for your docker idea, I'll give it a whirl on my nuc / home assistant? |
Sorry been busy. I'll tidy what I have and pop the link in. Conscious that this isn't strictly on topic with this repo. (But appreciation it links in) |
Theres interest on the home assistant side, if this could become an integration. Could really up the game. |
FYI just tried installing in a clean simple debian bulleye arm64 docker container. Was going fin till it hit the systemD part which, I gather, is blasphemy on docker. Might see If I run the services under supervisord but I don't really know where to start.
What are you using or are you going the docker compose route? |
How about a .sh entrypoint script for the Dockerfile which just runs the commands from the service init scripts? |
The issue is with systemD, on the classic debian container systemD doesn't work, you need to use supervisorD. Or separate each service into a separate container and use docker compose. Which is the 'right' way of doing it. |
One of RaspiOS's non-Debian-isms is that the default user (
Naturally, the 🤔 not sure if that is the issue you're encountering since the systemd services are run as the Let me know if having those sorts of permissions fixes the systemd shortcomings. (In theory, you could probably move the " My very best regards, |
putting the commands from the execstart lines in the systemd files into a .sh with an & at the end of each command (additionally we can output to a /var/log like this too so docker logs works), then setting the .sh as the entrypoint for the Dockerfile (hence starting everything when the container starts). this doesnt need systemd or initd. I agree in theory that separating the services out into their own containers appears to be the right way, however this is pretty close to working in its current form and could be a good starting point - would bring in some new contributors and sets of eyes to tidy it up. |
If I understand correctly we are going to move all the services away from systemd and into a script that starts when the container is run? All the services will need to be installed beforehand so it doesn't really change the installation process, we just need to create a new entrypoint after the container has finished doing its business the first time. I'll clone the repo myside when I have some time and give it a shot. Unless someone else whats to give it a go too and I clone their repo? |
I'll gove this a go next week, on holiday then. Cheers |
Hey @matthew73210, did you end up trying this out? If yes, did you get anywhere? |
Hey, got completely outpaced by work. On holiday soon. Shall we move this to discussions as it isn't an issue per say? |
Could this be a replacement for a docker variant? https://github.com/mmcc-xx/BirdCAGE |
Did everyone settle on a location to discuss docker and other container approaches? I haven't really found anywhere active that might be relevant except #251. I run my own homebrew container system and have managed to get BirdNET-Pi running in it OK, it should be fairly easy to translate the build script into a Dockerfile. |
Birdcage looks promising: https://github.com/mmcc-xx/BirdCAGE |
Thanks for the heads-up, @evanbills! |
I think BirdCAGE includes a modified |
It does, it's because the docker container was restarting too often.
|
Hi, I've made a working aarch64 and amd64 docker container based on https://github.com/Nachtzuster/BirdNET-Pi from @Nachtzuster. It is a homeassistant addon, that can be installed using my repo here : https://github.com/alexbelgium/hassio-addons If you don't use homeassistant, the containers can be assessed directly from the packages : https://github.com/alexbelgium/hassio-addons/pkgs/container/birdnet-pi-aarch64 and https://github.com/alexbelgium/hassio-addons/pkgs/container/birdnet-pi-amd64 |
Hey @alexbelgium! Wow, thank you very much! Your hassio-addons collection is a must-have now even more :) |
I'm making this issue and working on it myself, just in case anyone else here is interested in following the progress :)
The text was updated successfully, but these errors were encountered: