Docker image for ARM/Raspberry Pi #67
Comments
|
I just tried (rather naively I must admit) to install photonix on an RP4 of mine and failed (well, installationion went smooth I believe but running it fails). Thought photonix seemed like an awesome choice to use for my RP4 based image collection, especially since I'm going to use some ML on that image collection to experiment. :) DId not find any good contact info nor fixes for the issue I ran into so I'll just post this comment to say I believe Photonix seem very promising. |
|
Thanks for having a try @jensbrak. I hope to have a play with getting it running on a Raspberry Pi in the next few months but I just have to crack on with features right now. If you have an error log, please do share it here. I wouldn't expect it to do much at the moment tbh as each Docker image has to be compiled for each CPU architecture, I believe. Hopefully this project can suit your needs in the not-to-distant future. If you want to be notified when v1.0 is available please sign up to the mailing list on https://photonix.org . |
|
Well, I believe the errors I got where related to architecture yes. I
thought I'd make a note here to indicate you have at least one potential
user on RPI. :)
If I have the time, I can try to make an image/PR for the RPI. I'll add the
log here too, for future reference.
…On Thu, Mar 19, 2020 at 8:00 PM Damian Moore ***@***.***> wrote:
Thanks for having a try @jensbrak <https://github.com/jensbrak>. I hope
to have a play with getting it running on a Raspberry Pi in the next few
months but I just have to crack on with features right now.
If you have an error log, please do share it here. I wouldn't expect it to
do much at the moment tbh as each Docker image has to be compiled for each
CPU architecture, I believe.
Hopefully this project can suit your needs in the not-to-distant future.
If you want to be notified when v1.0 is available please sign up to the
mailing list on https://photonix.org .
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#67 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMRYUW573KEIL2RMD5CRZ53RIJTVPANCNFSM4G4P4UKQ>
.
|
|
The log in question: |
|
And this is how far I got building it. Stopper seems to be Tensorflow not supporting Arm architecture (or if it does, the references are wrong). FSEvents also seem to lack adequate platform support but did only warn about it. |
|
Thanks for sending the logs and trying to build, @jensbrak. Seems like the build got pretty far. You might want to try removing the line in |
|
@jensbrak I've made some good progress so far but am not quite there. See this branch for latest developments: https://github.com/damianmoore/photonix/tree/raspberry-pi I managed to build Tensorflow in my related project here: https://github.com/damianmoore/tensorflow-builder Currently I'm using a new Dockerfile and an experimental feature of Docker called Installing Python dependencies on Arm is very slow because there are not many binary Wheel packages available. I've sped up my Docker build time by storing cached Seems like an error building matplotlib/numpy with Fortran compilter but I made sure to install Any suggestions very much appreciated as I'm a bit stuck right now. Thanks. |
|
@damianmoore Thank's for making an effort, appreciated! No spontanious suggestions, but I'll give it a go at some point, for sure! Since it's a lovely piece of software it would be awesome to get it up and running but I am by no means dependent on it. |
|
A small update - mainly for future me. The problem is with building Numpy. I tried using the Debian package |
|
Fixed the errors building Numpy. The problem was in my script Next problems to solve:
|
|
I haven't really tried the Raspberry-Pi branch of Photonix, but I tried the Master branch today. While it fails with the wrong platform for Tensorflow it is possible to go beyond that by using community builds and manually install it. This way I managed to go beyond the Tensorflow parts as well (I used a version from here: https://github.com/bitsy-ai/tensorflow-arm-bin) Currently I'm getting stuck at a SciPy dependency as far as I can see: |
|
I have an image that's generally working. You can try it if you have an ARMv7 Raspberry Pi (such as my Pi2 Model B). It starts up, imports demo photos. The main downsides are:
I should be able to disable certain analysis jobs when memory is tight but allow then to be switched on again through the UI. Fixing the Freetype dependency should just be a case of adding Anyway this is the version: '3'
services:
postgres:
container_name: photonix-postgres
image: postgres:11.1-alpine
environment:
POSTGRES_DB: photonix
POSTGRES_PASSWORD: password
volumes:
- ./data/db:/var/lib/postgresql/data
redis:
container_name: photonix-redis
image: arm32v7/redis:3.2.12
photonix:
container_name: photonix
image: damianmoore/photonix-multiarch:latest
ports:
- '8888:80'
environment:
DEMO: 1
ENV: prd
POSTGRES_HOST: postgres
POSTGRES_DB: photonix
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
REDIS_HOST: redis
ALLOWED_HOSTS: '*'
volumes:
- ./data/photos:/data/photos
- ./data/raw-photos-processed:/data/raw-photos-processed
- ./data/cache:/data/cache
- ./data/models:/data/models
links:
- postgres
- redis |
|
I tested it on the Raspberry pi 4 and it worked just fine, besides the import. |
|
Thanks for the update @Felix-Franz. I actually just got a Raspberry Pi 4 myself and yesterday I confirmed that the armhf/ARMv7 32-bit image works fine on it. For extra performance we should get the ARMv8 64 bit version working but the 32 bit version imports and runs the demo gallery. I think you have discovered a separate bug in Photonix though. You must have a photo that has 'Camera Model Name' in it's metadata but no 'Make'. The import code assumes that there must be a camera make if there's a camera model. I'll create a bug ticket for that. For now maybe you can try some different test images. |
This was referenced Feb 21, 2021
|
I have used the docker-compose for Pi. |
|
Hi @darrepac. Could it be a networking or firewall problem? Are you using a browser on the Raspberry Pi or across the network? If over a network then the URL won't be localhost but the IP/hostname of the Raspberry PI. Sorry if this is obvious - just trying to eliminate any simple fixes. If possible can you run this on the Pi itself?: |
|
I think my issue is on the performance side.... too much things running, too much photo to crawl... will try to step down |
|
@darrepac OK, sounds possible. With the |
|
I started again with only the demo files... took quite some time to have something on the web side and got quite some errors in the terminal. Normal? photonix | utility.execute() |
|
Hi @darrepac thanks for the update. That is about what I would expect. The |
|
photos seems to display slowly (get 2 for the moment)... |
|
Which Raspberry Pi version are you running @darrepac? |
|
Raspberry 3B |
|
@darrepac Would be interesting to know if it's pegged at 100% CPU or if RAM is full or if disk IO is the bottleneck. I find that which SD card I use affects things drastically. You can create a swap file on disk if you are out of RAM. |
|
I get a similar error as @darrepac. |
|
Thanks for the comment @Felix-Franz. I am at a similar point from my side. This I think is the key error: I found out that pip compiles each package it needs to in an isolated environment which means Matplotlib gets built against a different version of Numpy to that in our So far I've tried the suggestion of |
|
This issue is now complete as of 08fb9ba (arm32v7 and arm64v8). Builds are also automated through GitHub Actions whenever a new version is tagged and pushed. I've also moved to a new Docker Hub organisation at https://hub.docker.com/r/photonixapp/photonix/ . @jensbrak, @Felix-Franz, @darrepac it would be good if you could have a little test (with a small number of photos to start with). |
|
@damianmoore - This is exciting news! I'll have a go at it as soon as I can, for sure! Great work! |
|
@damianmoore it just worked find with a smaller number of photos, so I tried a larger number. |
|
Thanks @Felix-Franz for giving it a go and reporting back. I'm merging another feature at the moment to give control over which analysers are running which should be useful for scaling down all that CPU usage from the UI. |
|
Well, I almost made it. Not as successful as @Felix-Franz though. Which leads me back to some messages I briefly noted when launching Photonix (which worked "to the end" now, thanks to all efforts made by you @damianmoore . Some snippets from installation messages: (And then Photonix continued with the onboarding process) Please note that I have NOT tried to do any kind of troubleshooting, I just went through the photonix instalation instructions after checking that I had docker and docker-compose properly installed. So no need to go trying to find issues in Photonix based on this post - It's just me reporting back my first run of the new version. I will have to look at it myself, but must stop now for other activites. I do have a couple of notes just by looking at the logs:
|
|
Well, now I've experimented some more. It works mostly, so no problem with the overall functionality. Again, great work and thanks for the effort. Photonix seems awesome now that I have it running - finally! I have to dig into the classification parts more though. That's where I have my main interest tbh. |
|
OK @jensbrak, thanks for testing. Good to know that it's generally working. Tensorflow failures (or any classification failures shouldn't affect thumbnails so I'm a bit confused about that. |
|
@damianmoore Nah, don't believe classification/tensorflow issues are related to thumbnail issues here. More likely it's due to me interrupting execution of docker at a bad timing or something, when it comes to the thumbnails. Classification works in general (the errors noted may or may not be relevant, haven't got a clue yet) but gives some hillarious results depending on what kind of pictures I feed it. TL;DR: Photonix works 'just fine' on RPi4 now. Some notable errors/warnings when installing and launching that might or might not impact functionality. Some weird thumbnail issue of oncertain origin and some classification issues likely connected to the training data. |
I am having the same error, photonix said during the installation that no photos were detected but they are in the /data/photos folder and visible in my container. Did you get it working in the end ? |
|
Ok, to follow up, I have about 5000 photos in the mounted directory. After about 10 minutes of completing the photonix installation my pi 4 8gb froze. I waited about 30 minutes and it came back to life and photonix was scanning. Both CPU and RAM are maxed out. So it seems that patience is key! Thanks for the great project here! |
|
Well it froze again unfortunatly. Seems that I am running out of RAM. |
|
Would there be a way to limit the number of workers so as to avoid eating up all of my RAM ? |
|
Hi @gomme600, glad it's getting a bit further for you. I have some optimisation work I'm planning at the moment to make sure it doesn't run out of RAM. The analysers are set to only one worker each so can only really scale down to 'off'. For now I would recommend going to settings in the top-right of the UI and turning off most of the analysers off to start with. Jobs will still be queued for photos that are imported so you can turn them on one-at-a-time once importing has completed. Sorry it's a bit hacky on low memory machine's right now but not for much longer, hopefully. |
|
Thanks for the tips! Also, quick question. Whenever I turn on the option to watch the library for new photos it gets turned back off again, is that normal ? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: