[Support]: No EdgeTPU was detected, ValueError: Failed to load delegate from libedgetpu.so.1.0. Was working previously. #3116
Comments
|
Have you try redownloading the drivers? |
|
Drivers are up to date and the device shows up. |
If you set the config to use cpu (so it doesn't die immediately) and you run the same inside the frigate container, is it visible? |
|
|
|
Having the same exact issue running PCIE Coral with Frigate. It works for a day or two and then starts crashing with this error above. If you reboot the server it works until it starts crashing again. When it starts crashing, I’ve tried to run from the host an example image classification script that uses tensorflow as well and it fails with the same error. When these crashes start, the tpu is still findable in pci devices list, as well as under “apex”. |
|
I have the same exact issue here as well! out of the blue it stopped working (was working great for the last 6 months). |
|
Has anyone had luck fixing this issue? I tried rolling back to 0.9.4 and have the same issue, I'm at a loss. |
|
I have seen this same issue with two different systems, at some point with ”docker-compose restart frigate”, it lost Coral. Reboot of system has not fixed it, and I have had to fall back to cpu. One system is Ubuntu with pci-e coral, other is debian and usb. Both running last stable frigate with docker compose. Later lost with Debian host happened today when I added one more camera, Ubuntu has been with cpu for months as I have had no time to further debug and those three cameras on that system are with very few motion. |
|
Log output from boot after problem started:
on docker host: Bus 001 Device 004: ID 18d1:9302 Google Inc. |
|
Somehow I made a progress... seeing this in the logs:
|
|
After few boots of host computers and docker etc (not because of this issue), I tried to add USB-TPU again in config and it works again. |
|
And almost after I wrote above, after yet another "docker compose restart frigate" no Coral again and had to switch CPU-detection. Could this be something about stick left to some wrong state on shutdown? |
Why it looks like everything is okay but now events are generated? Any ideas? |
|
And I can also verify that the TPU is working properly with the example test: `root@frigate:~/coral/pycoral/tflite/python/examples/classification# python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg but still it doesn't work through Frigate itself, no events are being generated :/ |
|
In my case, when the issue appears and I start getting failed to load delegate in frigate, if I try to run the above test from the host, I get the same errors unless I reboot the system. |
|
I've got the same problem. Problem started today or yesterday. I've reinstalled Google's drivers but Frigate still doesn't detect it. Ubuntu detects it fine. lspci shows the tpu, and ls /dev/apex shows the device. |
|
I tried with an older version (0.9.3-5E692AC) and still has the same issue. |
|
Does this coincide with some host os update? Maybe a new kernel update isn't compatible with the apex driver? |
|
I thought that this might be an issue so I tried with an older kernel version asl well. root@frigate:~# uname -r root@frigate:~# ls -l /dev/apex_0 |
|
I downgraded the kernel now, nothing changed: root@frigate:~# uname -r |
|
Are any of you running this in docker? There was a recent docker upgrade. |
|
@phammichael yes. I'm using docker. please be my savior :) btw, I just tried to reinstall all the drivers and follow the instructions on the Coral website, I thought that after a kernel change this was required, but no change. |
|
I'm running in docker, just pulled the latest, no change in behavior. |
|
Sorry my last comment wasn't clear. What I meant: Docker was recently updated, and the new version of Docker might be the cause of the problem. |
|
Is there a way to revert to a previous version? Which version are you running currently? ` Server: Docker Engine - Community |
|
@phammichael I'm running |
|
I also run this in Docker, debian host with USB-coral : Linux testirata 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux Client: Docker Engine - Community Server: Docker Engine - Community Ubuntu with PCI-Coral: Linux sahara 5.13.0-40-generic #45~20.04.1-Ubuntu SMP Mon Apr 4 09:38:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux Client: Server: Havent done any active upgrade to either system just before this started, but it still could be something happened weeks before actual frigate restart that drops Coral undetectable. But at least with that debian-system I have been tweaking a lot recently, it wasn't something like next restart after host reboot, but I had restarted container many times in few hours, and then it just stopped to run when there was no detectors online. |
|
I was able to fix my problem with Coral. I think it was a problem with my python installation. Coral was previously working correctly with Frigate, but recently stopped working. I have a dual TPU Coral using a PCIe adapter, Ubuntu system, and Frigate running in Docker. ls /dev showed apex0 and lspci -nn | grep 089a showed the correct device, so that meant the Coral PCIe driver and the Edge TPU runtime were installed correctly. But when I tested pycoral with the example from Google, it failed. This suggested something was wrong with either my pycoral installation or python installation on my system. Listed below are the commands I ran. Some of it might not be required, but that's what I ran this morning and now my Coral is working again with Frigate. Remove python2 if it's installedsudo apt remove --purge python2 Set python3 versionupdate-alternatives --config python3 Note: I set mine to 3.9. If you're using a different version, the below commands will need to be modified accordingly.Reinstall python3.9sudo apt reinstall python3.9 Reboot systemsudo reboot Reinstall pycoral and numpysudo apt reinstall python3-pycoral Install Pillow if missingpip install -U Pillow Remove existing example for pycoral and then reinstall itsudo rm -fr coral Test pycoral with Google's examplepython3 examples/classify_image.py If the test with Google's example was successful, then Coral TPU should now also work in Frigate. |
|
@phammichael I'm not sure if I'm doing this right, but this was the output I got after following your instructions |
tehniemer
changed the title
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
Found my solution here google-coral/edgetpu#611 |
|
phammichael solution has now worked for me twice. I might add those commands into a bash script and automate it when docker logs show the coral acting up. Sucks this happens but glad to have a working solution |
Describe the problem you are having
Frigate has stopped detecting the EdgeTPU I've been using successfully for quite some time. Everything was working until recently, but I don't have an exact timeframe when it stopped. I'm using a M.2 Accelerator with Dual Edge TPU with a couple adapters to get it into a pcie slot. I haven't had any luck troubleshooting this on my own. Looking for some next steps I could try.
Version
0.10.1-83481AF
Frigate config file
Relevant log output
FFprobe output from your camera
Frigate stats
No response
Operating system
Debian
Install method
Docker Compose
Coral version
M.2
Network connection
Wired
Camera make and model
AMCREST IP3M-HX2
Any other information that may be helpful
No response
The text was updated successfully, but these errors were encountered: