Failed to load delegate from libedgetpu.so.1 on PCIe EdgeTPU [SOLVED]

[asw-v4]

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  ++ Linux Ubuntu 18.04.3
• TensorFlow installed from (source or binary):
  ++ pip install tflite-runtime...whl
• TensorFlow version:
  ++ tflite-runtime-1.14.0-cp37...
• Python version:
  ++ 3.7.5
• Installed using virtualenv? pip? conda?:
  ++ conda
• CUDA/cuDNN version:
  ++ CPU Only with EdgeTPU
• GPU model and memory:
  ++ PCIe EdgeTPU

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               60
Model name:          Intel(R) Core(TM) i7-4700EQ CPU @ 2.40GHz
Stepping:            3
CPU MHz:             1097.578
CPU max MHz:         3400.0000
CPU min MHz:         800.0000
BogoMIPS:            4789.32
Virtualisation:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            6144K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts md_clear flush_l1d

Describe the problem

When running classify_image.py, I receive the following error

(Coral) v4@v4-TPU:~/coral/tflite/python/examples/classification$ python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
Traceback (most recent call last):
  File "/home/v4/anaconda3/envs/Coral/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 165, in load_delegate
    delegate = Delegate(library, options)
  File "/home/v4/anaconda3/envs/Coral/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 119, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "classify_image.py", line 118, in <module>
    main()
  File "classify_image.py", line 95, in main
    interpreter = make_interpreter(args.model)
  File "classify_image.py", line 69, in make_interpreter
    {'device': device[0]} if device else {})
  File "/home/v4/anaconda3/envs/Coral/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 168, in load_delegate
    library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1

Any other info / logs
I have seen various solutions, suggesting I add the user to plugdev group (This was on a solution regarding the USB Accelerator, but i tried it anyway)

Another point was that it wasnt plugged in (again for the usb accelerator), which it is:

v4@v4-TPU:~$ ls /dev/apex_0
/dev/apex_0

v4@v4-TPU:~$ lspci -x | grep 089a
04:00.0 Non-VGA unclassified device: Device 1ac1:089a

And dmesg:

v4@v4-TPU:~$ dmesg | grep apex
[    7.830468] apex 0000:04:00.0: Apex performance not throttled due to temperature

v4@v4-TPU:~$ dmesg | grep 089a
[    0.274318] pci 0000:04:00.0: [1ac1:089a] type 00 class 0x0000ff

so the device is connected properly.

The Libraries are appropriately installed:

(Coral) v4@v4-TPU:~/coral/tflite/python/examples/classification$ ls /usr/lib/x86_64-linux-gnu/libedgetpu.so.1*
/usr/lib/x86_64-linux-gnu/libedgetpu.so.1  /usr/lib/x86_64-linux-gnu/libedgetpu.so.1.0

and these are the permissions of the libraries:

(Coral) v4@v4-TPU:~/coral/tflite/python/examples/classification$ ll /usr/lib/x86_64-linux-gnu/libedgetpu.so.1*
lrwxrwxrwx 1 root root     17 Sep 16 21:27 /usr/lib/x86_64-linux-gnu/libedgetpu.so.1 -> libedgetpu.so.1.0
-rwxr-xr-x 1 root root 956392 Sep 16 21:27 /usr/lib/x86_64-linux-gnu/libedgetpu.so.1.0

NOTE: I did have to add the 2nd and 3rd execute permissions on libedgetpu.so.1.0

[asw-v4]

Having gone through this with Coral technical, it was narrowed down to not properly adding the user to the apex group, causing the error in accessing the libs.

Marked Closed and Solved

[batulrangwala]

I am also facing a similar issue.
The demo API gives error at tflite.load_delegate.

pi@bpi-iot-ros-ai:~/coral/tflite/python/examples/classification$ python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
E :248] HIB Error. hib_error_status = 0000000000000001, hib_first_error_status = 0000000000000001
E :248] HIB Error. hib_error_status = 0000000000000001, hib_first_error_status = 0000000000000001
INFO: Initialized TensorFlow Lite runtime.
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.



pi@bpi-iot-ros-ai:~$ uname -a
Linux bpi-iot-ros-ai 5.4.0-bpi-r64 #1 SMP PREEMPT Mon Dec 16 16:00:08 IST 2019 aarch64 aarch64 aarch64 GNU/Linux
pi@bpi-iot-ros-ai:~$ lscpu
Architecture:          aarch64
Byte Order:            Little Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    1
Core(s) per socket:    2
Socket(s):             1
CPU max MHz:           1350.0000
CPU min MHz:           30.0000
pi@bpi-iot-ros-ai:~$ ls -l /usr/lib/aarch64-linux-gnu/libedge*
lrwxrwxrwx 1 root root     17 Sep 17 04:27 /usr/lib/aarch64-linux-gnu/libedgetpu.so.1 -> libedgetpu.so.1.0
-rwxrwxrwx 1 root root 792376 Sep 17 04:27 /usr/lib/aarch64-linux-gnu/libedgetpu.so.1.0
pi@bpi-iot-ros-ai:~$ lspci
00:00.0 PCI bridge: MEDIATEK Corp. Device 3258
01:00.0 System peripheral: Device 1ac1:089a
pi@bpi-iot-ros-ai:~$ ls /dev/apex_0 
/dev/apex_0
pi@bpi-iot-ros-ai:~$ sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"
pi@bpi-iot-ros-ai:~$ sudo groupadd apex
groupadd: group 'apex' already exists
pi@bpi-iot-ros-ai:~$ sudo adduser $USER apex
The user `pi' is already a member of `apex'.

[batulrangwala]

Coral Tech Support had indicated from the dmesg logs that the memory was not assigned for BAR 0

[ 1.604440] pci 0000:01:00.0: [1ac1:089a] type 00 class 0x0000ff 
[ 1.610743] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit pref] 
[ 1.618078] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x000fffff 64bit pref] 
[ 1.626285] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5 GT/s x1 link) 
[ 1.641429] pci_bus 0000:01: fixups for bus 
[ 1.645617] pci_bus 0000:01: bus scan returning with max=01 
[ 1.651195] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 
[ 1.657816] pci_bus 0000:00: bus scan returning with max=01 
[ 1.663407] pci 0000:00:00.0: BAR 0: no space for [mem size 0x200000000 64bit pref] 
[ 1.671071] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x200000000 64bit pref] 
[ 1.679079] pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x201fffff] 
[ 1.685872] pci 0000:00:00.0: PCI bridge to [bus 01] 
[ 1.690846] pci 0000:00:00.0: bridge window [mem 0x20000000-0x201fffff] 

The BAR0 for pcie bridge didn't get memory range assigned, is is an issue with the driver in the SoC. This "BAR 0 failed to assign" is preventing our device from accessing the host's memory. This is exactly why you get this issue right during load delegate. Unfortunately, this is a little outside of our hand. On another note, the swiotlb=512 from your kernel command line is very low, I suggest increasing to swiotlb=65536 for possible unrelated issues.

I have now update the swiotlb=65536 and gasket.dma_bit_mask=32 in the kernel command line and also managed to resolve the BAR0 memory assignment issue.
Following is the log.

pi@bpi-iot-ros-ai:~$ lsmod
Module                  Size  Used by
mt7622                 45056  -2
mt76                   45056  -2
mac80211              364544  -2
libarc4                16384  -2
apex                   24576  -2
gasket                 98304  -2
cfg80211              258048  -2
btmtkuart              24576  -2
pi@bpi-iot-ros-ai:~$ dmesg | grep apex
[    8.381572] apex 0000:01:00.0: assign IRQ: got 140
[    8.381703]  apex_pci_probe+0x38/0x468 [apex]
[    8.381762]  apex_init+0x44/0x1000 [apex]
[    8.381906]  apex_pci_probe+0x38/0x468 [apex]
[    8.381960]  apex_init+0x44/0x1000 [apex]
[    8.382002] apex 0000:01:00.0: Assigned....BAR 0 [mem 0x20100000-0x20103fff 64bit pref]........
[    8.382008] apex 0000:01:00.0: Assigned and claimed....BAR 0 [mem 0x20100000-0x20103fff 64bit pref]........
[    8.382014] apex 0000:01:00.0: Assigned....BAR 2 [mem 0x20000000-0x200fffff 64bit pref]........
[    8.382020] apex 0000:01:00.0: Assigned and claimed....BAR 2 [mem 0x20000000-0x200fffff 64bit pref]........
[    8.382025] apex 0000:01:00.0: enabling device (0000 -> 0002)
[    8.382078] apex 0000:01:00.0: enabling bus mastering
[   13.538841] apex 0000:01:00.0: Apex performance not throttled due to temperature
pi@bpi-iot-ros-ai:~$ dmesg | grep pci
[    1.494119] mtk-pcie 1a143000.pcie: host bridge /pcie@1a143000 ranges:
[    1.505642] mtk-pcie 1a143000.pcie: Parsing ranges property...
[    1.515992] mtk-pcie 1a143000.pcie:   MEM 0x20000000..0x2fffffff -> 0x20000000
[    1.556439] mtk-pcie 1a143000.pcie: PCI host bridge to bus 0000:00
[    1.568448] pci_bus 0000:00: root bus resource [bus 00-ff]
[    1.573946] pci_bus 0000:00: root bus resource [mem 0x20000000-0x2fffffff]
[    1.580841] pci_bus 0000:00: scanning bus
[    1.584897] pci 0000:00:00.0: [14c3:3258] type 01 class 0x060400
[    1.590957] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x1ffffffff 64bit pref]
[    1.599829] pci_bus 0000:00: fixups for bus
[    1.604022] pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 0
[    1.610729] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    1.618747] pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 1
[    1.625563] pci_bus 0000:01: scanning bus
[    1.629683] pci 0000:01:00.0: [1ac1:089a] type 00 class 0x0000ff
[    1.635984] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit pref]
[    1.643318] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x000fffff 64bit pref]
[    1.651524] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
[    1.666709] pci_bus 0000:01: fixups for bus
[    1.670896] pci_bus 0000:01: bus scan returning with max=01
[    1.676472] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.683093] pci_bus 0000:00: bus scan returning with max=01
[    1.688684] pci 0000:00:00.0: BAR 0: no space for [mem size 0x200000000 64bit pref]
[    1.696347] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x200000000 64bit pref]
[    1.704355] pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x201fffff]
[    1.711151] pci 0000:01:00.0: BAR 2: assigned [mem 0x20000000-0x200fffff 64bit pref]
[    1.718983] pci 0000:01:00.0: BAR 0: assigned [mem 0x20100000-0x20103fff 64bit pref]
[    1.726814] pci 0000:00:00.0: PCI bridge to [bus 01]
[    1.731787] pci 0000:00:00.0:   bridge window [mem 0x20000000-0x201fffff]
[    1.738888] mtk-pcie 1a145000.pcie: host bridge /pcie@1a145000 ranges:
[    1.745425] mtk-pcie 1a145000.pcie: Parsing ranges property...
[    1.751269] mtk-pcie 1a145000.pcie:   MEM 0x28000000..0x2fffffff -> 0x28000000
[    1.758508] mtk-pcie 1a145000.pcie: resource collision: [mem 0x28000000-0x2fffffff] conflicts with pcie@1a143000 [mem 0x20000000-0x2fffffff]
[    1.771141] mtk-pcie: probe of 1a145000.pcie failed with error -16
[    8.381659]  pci_enable_resources+0x68/0x18c
[    8.381665]  pcibios_enable_device+0xc/0x14
[    8.381670]  do_pci_enable_device+0x50/0xd8
[    8.381675]  pci_enable_device_flags+0x100/0x15c
[    8.381680]  pci_enable_device+0x10/0x18
[    8.381684]  pci_enable_bridge+0x50/0x78
[    8.381689]  pci_enable_device_flags+0x98/0x15c
[    8.381694]  pci_enable_device+0x10/0x18
[    8.381703]  apex_pci_probe+0x38/0x468 [apex]
[    8.381709]  pci_device_probe+0xa0/0x144
[    8.381755]  __pci_register_driver+0x40/0x48
[    8.381811] pci 0000:00:00.0: Assigned....BAR 8 [mem 0x20000000-0x201fffff]........
[    8.381817] pci 0000:00:00.0: Assigned and claimed....BAR 8 [mem 0x20000000-0x201fffff]........
[    8.381823] pci 0000:00:00.0: enabling device (0000 -> 0002)
[    8.381837] pci 0000:00:00.0: enabling bus mastering
[    8.381880]  pci_enable_resources+0x68/0x18c
[    8.381884]  pcibios_enable_device+0xc/0x14
[    8.381889]  do_pci_enable_device+0x50/0xd8
[    8.381893]  pci_enable_device_flags+0x100/0x15c
[    8.381898]  pci_enable_device+0x10/0x18
[    8.381906]  apex_pci_probe+0x38/0x468 [apex]
[    8.381911]  pci_device_probe+0xa0/0x144
[    8.381953]  __pci_register_driver+0x40/0x48
pi@bpi-iot-ros-ai:~$ 
pi@bpi-iot-ros-ai:~$ dmesg | grep gasket
[    0.000000] Kernel command line: board=bpi-r64 console=ttyS0,115200n1 earlyprintk root=/dev/mmcblk1p2 rootfstype=ext4 rootwait service=linux debug=7 initcall_debug=0 androidboot.hardware=mt7622 swiotlb=65536 gasket.dma_bit_mask=32
[    8.360957] gasket: loading out-of-tree module taints kernel.
pi@bpi-iot-ros-ai:~$ cd coral/tflite/python/examples/classification/
pi@bpi-iot-ros-ai:~/coral/tflite/python/examples/classification$ python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
E :237] HIB Error. hib_error_status = 0000000000000001, hib_first_error_status = 0000000000000001
E :237] HIB Error. hib_error_status = 0000000000000001, hib_first_error_status = 0000000000000001
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.

Inspite of this I am still getting this error

Any advice to resolve this

[toxicantidote]

As a note for anybody who finds this issue from Google (like me) after installing the mini PCIe accelerator and following the Coral getting started guide (from coral.ai) and you hit this error.

Check the permissions on your /dev/apex_0 device.

Mine was owned by user root, group root with permissions 600.

As root, change the owner of /dev/apex_0 to be user root, group apex, and permissions to 660:
chown root:apex /dev/apex_0 && chmod 660 /dev/apex_0