Coral mini pcie, will adapter work

[Henke19822]

I need to switch HW for my Home Assistant installation, and not many HWs offer mini pcie, which is the version I use for my Google Coral.
A lot of HWs offer M.2 instead, would it be possible to use a M.2 to mini PCIe adapter, or even a USB to mini PCIe adapter? Or would this fail?

[richardkuw]

Personally I would try a PCIe to mini-PCIe adapter if you have standard PCIe slots. Amazon lists several.

I see Mouser has thousands of mini-PCIe Coral TPUs in stock so I expect this will be a popular question.
https://www.mouser.com/ProductDetail/Coral/G650-04528-01?qs=sGAEpiMZZMu3sxpa5v1qrr6044Dk0q4qkDM1TzdftPE%3D

[Henke19822]

Sorry, me again. I don't really get my head around this (actually, not that familiar with PCI Express, except the one time I connected a GPU to it).
Several discussion forums say that I should be able to connect my mini PCIe to a bigger lane, for instance a PCIe X4. But, that doesn't fit, the space in my card is too narrow.

When I search Amazon for "PCIe to mini-PCIe adapter" I only get odd results, like M.2 adapters and stuff. Could you send me a link of one good item and I know what to look for (it can be x16 as well, the new HW should have that).

Thank you so much in advance

[git-coder988]

This one from Amazon worked for me:

PCI Express to Mini PCI Express Card Adapter Wireless Network Ethernet LAN Mini PCI-E Riser PC to Laptop Network Card Adapter Converter Antenna WiFi
https://www.amazon.com/gp/product/B07N2X62LQ/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Good luck!

[hmakmur]

This one from Amazon worked for me:

PCI Express to Mini PCI Express Card Adapter Wireless Network Ethernet LAN Mini PCI-E Riser PC to Laptop Network Card Adapter Converter Antenna WiFi https://www.amazon.com/gp/product/B07N2X62LQ/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Good luck!

Did you have to worry about Voltage requirement with this adapter? I am planning to use it on my dell desktop.
thanks!

[rpm5099]

This one from Amazon worked for me:

PCI Express to Mini PCI Express Card Adapter Wireless Network Ethernet LAN Mini PCI-E Riser PC to Laptop Network Card Adapter Converter Antenna WiFi https://www.amazon.com/gp/product/B07N2X62LQ/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Good luck!

I tried a similar product and the machine wouldn't post at all with it inserted - it did while empty but not with the TPU inserted. I'm left wondering if the TPU is bad or if it was the adapter. I'll try this one and post back.

I see Mouser has thousands of mini-PCIe Coral TPUs in stock so I expect this will be a popular question.
yes

[Henke19822]

Ah, that is actually a good suggestion. Didn't think of that, but I do have an empty PCI Express x16 expansion slot, finding a PCIe to mini PCIe is probably a good solution.
I guess Frigate wouldn't mind to use that instead of the mini PCIe slot?

[richardkuw]

It should just show up as a PCI device if you run lspci.
01:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU

This is configured as device: pci:0 in my frigate.yml.

The TPU only needs one PCIe lane but there's no harm in using a x16 slot as long as you don't need the wide slot for something else. Usually x16 would be for graphics, or high speed storage. Something that requires a lot of bandwidth. But of course it's using zero bandwidth sitting there empty.

[Henke19822]

Thanks a lot for your support. Will look into this as soon as I have the new HW

[richardkuw]

You’re trying to plug a mini-PCIe card directly into a full size PCIe slot. You need an adapter. Unfortunately it appears that the Ableconn PEX-MP117 adapter that was previously reported to work with Coral isn’t available (except I see one on Ebay). I would probably try this one and return if it doesn’t work. It has a switch to control the power supply voltage for the mini card, the Coral needs 3.3 volts. https://www.amazon.com/BQZYX-Mini-PCI-Express-Adapter/dp/B07ZKHWVQV Or, you could try an M.2 adapter as you first suggested, but it needs to be the correct type for your motherboard’s M.2 slot. A/E or M, usually. This one is A/E slot: https://www.amazon.com/Mini-PCI-NGFF-Adapter-Module/dp/B08B3WF67J I suspect the reason Mouser has so many mini-PCIe TPUs is that it’s a 20 year old interface and not very popular. So options for using them are a bit limited. Good luck! On Apr 9, 2023, at 4:15 AM, Henke19822 ***@***.***> wrote: Sorry, me again. I don't really get my head around this (actually, not that familiar with PCI Express, except the one time I connected a GPU to it). Several discussion forums say that I should be able to connect my mini PCIe to a bigger lane, for instance a PCIe X4. But, that doesn't fit, the space in my card is too narrow. [20230409_110605]<https://user-images.githubusercontent.com/48420440/230764682-055bd365-c540-41db-aac8-a302a2a28de9.jpg> When I search Amazon for "PCIe to mini-PCIe adapter" I only get odd results, like M.2 adapters and stuff. Could you send me a link of one good item and I know what to look for (it can be x16 as well, the new HW should have that). Thank you so much in advance — Reply to this email directly, view it on GitHub<#5932 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALWES74IR6WECLYUKKEDVC3XAJ43NANCNFSM6AAAAAAWXKZT6A>. You are receiving this because you commented.Message ID: ***@***.***>

[moskovskiy82]

Just get an adapter from aliexpress or other marketplaces. Works like a charm. Just type "mini pcie adapter"

[Henke19822]

Wow, what awesome support. Deeply grateful.
I wouldn't mind to actually buy a coral with a different (more modern) format, but they are really hard to come by.
This help was awesome

[gcormier]

I designed an adapter myself if anyone is interested - open source hardware. https://github.com/gcormier/minicoralbridge

[haideramn]

Hi would this work to take coral Mini PCIe into PCIe 3.0 on a Dell micro 7050?

[gcormier]

Yes, ignoring any unique physical constraints you may or may not have.

[glossyio]

Adding my experience trying to fit the M.2 Coral chip into my desktop PC (running UNRAID). Mini-PCIe to PCIe adapter was the correct route. I now have it up and running:

• WORKED: GLOTRENDS card (https://www.amazon.com/dp/B0B6HYW2CB). Worked like a charm in my PCIe 1x slot 3 (MSI board). Just needed to install the Unraid Coral Driver App then passthru to Frigate Docker container.
• DID NOT WORK: Zerone card (https://www.amazon.com/dp/B07N2X62LQ) This one "grounded" my board - as soon as I plugged it in it lit up my CPU cooler lights and had a 00 BIOS Debug LED code (without even starting). And hitting the power button spun up my fans but it didn't even POST.

[Henke19822]

Nice to hear, but I have the opposite challenge. I have the mini pcie coral card and would like to fit it in a M.2-slot (I have ordered stuff to enable the PCIe X4 slot, if such is existing on my next HW)

[zombielinux]

I'm seeing the Zerone card failing as well in the same way on a Supermicro X9DRI-LN4F+.

Do you happen to have schematics or high quality photos of that board? I wonder what the difference is.

The Zerone card doesn't have the CLKREQ signal routed, but even bodging it on doesn't get it out of the post.

Other cards work in this adapter, so there's something weird with the Coral + Adapter + Motherboard.

EDIT: The Issue is as follows:

The mPCIe Coral device doesn't conform to PCIe spec. It shorts the 3.3V rail to the 3.3VAUX rail. Smart motherboards recognize this as a bad idea, but not so smart ones don't care or have a unified 3.3v rail.

Slicing the trace coming off the B10 pin is enough to get the board working. Also, the datasheet shows the SMBUS traces are unpopulated, but they're present and routed on the Coral itself.

[Leon-Sam]

I had the same issue with it not posting. You can fix it by taping pin 10 on B side. I did this and it works!
More details here. google-coral/edgetpu#232

[jvrobert]

FYI the makerfabs adapter works great and enables both of the TPUs on the dual TPU card. I had mine in a normal slot prior to getting it and was only able to use one of the TPUs, which had absolutely no impact on my life or ability to use Frigate but highly annoyed me, lol. So I got the adapter and was a bit nervous but it works perfectly.

[LockeCJ]

Here's a link to makerfabs if anyone wants to buy one:
https://www.makerfabs.com/dual-edge-tpu-adapter.html
They're in stock as of this comment (2023-08).
There are other adapters available as well, and an unreleased 2x adapter which would make 4 TPUs available in a single slot:
https://github.com/magic-blue-smoke/Dual-Edge-TPU-Adapter

[mastan30]

Hi I have got the adapter and Coral TPU mini pice version, Can someone help me in setting up the drivers in Proxmox, I searched google a lot but it's confusing. I am trying to run Home assistant and Frigate in Docker and passthrough the Coral in a Proxmox server. Any help is much appreciated!

Also this the adapter I got : https://www.amazon.com/gp/product/B07N2X62LQ/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

[richardkuw]

I made these notes when I set up Home Assistant and Frigate on Proxmox a couple years ago. I used a KVM VM called VM 101, change to match your VM.

The objective is to disable the apex driver on the Proxmox host so the guest can use the TPU. When I set this up I only had a single TPU. If you have a dual they should both show up in lspci.

Edit /etc/modules on Proxmox host:
add:
vfio
vfio_iommu_type1
vfio_pci vfio_virqfd

Edit /etc/modprobe.d/pve-blacklist.conf on Proxmox host:
add:
blacklist apex

Update initramfs and reboot the host:
#update-initramfs -u -k all
#reboot

Check result:
#lspci -nnk (should show no apex driver in use for TPU on the Proxmox host)

Turn on passthrough for VM 101 using pcie address of TPU. Get that from lspci:
#qm set 101 -hostpci0 01:00.0,pcie=on

Create file to add intel_iommu=on boot parameter (or amd_iommu=on if appropriate for your CPU). VM WILL NOT BOOT with passthrough without IOMMU.
Created: /etc/default/grub.d/iommu.cfg:
GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX intel_iommu=on"

Apply changes:
#proxmox-boot-tool refresh
#reboot

Results:
#lspci -nnk (on Proxmox host) shows no apex driver in use:
01:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a] Subsystem: Global Unichip Corp. Coral Edge TPU [1ac1:089a] Kernel driver in use: vfio-pci Kernel modules: apex

#lspci -nnk (in VM) shows apex driver being used:
01:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a] Subsystem: Global Unichip Corp. Coral Edge TPU [1ac1:089a] Kernel driver in use: apex Kernel modules: apex

[kwanty]

Edge TPU on frigate on docker on virtual machine on True NAS core working :-)

Here is my "journey" with minipci Coral :-) Below complete solution for the spec:

• True NAS core (not scale)
• CPU: Xeon E3-1230 v2 (VT-d extension is necessary)
• frigate: True NAS (FreeBSD) -> Virtual Machine (Ubuntu 22.04) -> docker (official frigate image: ghcr.io/blakeblackshear/frigate:stable)

PCI -> miniPCI adapter

First you need buy an adapter. It is only pin-mapping adapter + one 1v regulator. I've tryed two different from aliexpress (3-4 Euro each): link1, link2, they could be unavailable in future, the looks like this with mounted Coral TPU, both works perfect:

Be extremely careful when connecting to PCI-e. The position in slot is different than you (probably) think! Longer part of adapter (with 11 gold pins) goes to shorter part of PCI-e slot, before "key". If you connect it in other way you short-cut and demage motherboard/card.

True NAS Core

True NAS Core is based on FreeBSD, you need allow for pci throughput in virtual machine (Ubuntu) governed by bhyve. First, enable VT-d in a BIOS (Virtual Machine Manager - Virtualization Technology for Directed I/O) - this is Intel stuff, AMD has similar feature.

After restart open True NAS console check if You see pci device, it should looks like:

> lspci -v
0e:00.0
coral: vendor specific information: ID=1556, Rev=1 Len=008

or

>pciconv -lv
none0@pci0:14:0:0:      class=0x0000ff rev=0x00 hdr=0x00 vendor=0x1ac1 device=0x089a subvendor=0x1ac1 subdevice=0x089a
    vendor     = 'Global Unichip Corp.'
    device     = 'Coral Edge TPU'
    class      = old
    subclass   = non-VGA display device

Next, according to: https://www.bsdstore.ru/en/13.0.x/wf_bhyveppt_ssi.html and https://wiki.freebsd.org/bhyve/pci_passthru you need setup system variables, it is essential to pass-through device to virtual machine.

Open System/Tunables in a True NAS and add the following variables:

vmm_loads=YES
pptdevs=14/0/0

The second variable depends on your PCI-e slot, check numbers in lspci 0e:00:0 Non-VGA unclassified device: Global Unichip Corp. Coral Edge TPU 0e:00:0 means 14/0/0 (0x0e in hex converts to 14 decimal)

It will looks like:

Create Virtual Machine

After TrueNAS restart you can finally add PCI device to your VirtualMachine. Before the list with PCI devices was empty. Now it looks like:

Virtual Machine

Assume, you create virtual machine Ubuntu 22.04 (or similar), boot it, login, sudo su and type:

lspci:
00:05.0 System peripheral: Global Unichip Corp. Coral Edge TPU

You sould see Coral Edge TPU inside your Virtual Machine (Ubuntu)...

Now you need to install all stuff related with TPU Coral, according to doc:

sudo echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
apt-get update
sudo apt-get install gasket-dkms libedgetpu1-std

sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"

restart virtual machine and check for apex module:

lsmod | grep apex
apex                   28672  0
gasket                135168  1 apex

and apex device:

> ls -al /dev/apex_0 
crw-rw---- 1 root root 120, 0 Sep 25  2023 /dev/apex_0

frigate docker

Assume you are using frigate in a docker. My docker-compose.yml looks like:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "64mb"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/__user__/frigate/config/config.yml:/config/config.yml
      - /home/__user__/frigate/database:/db
      - type: tmpfs # 1GB of memory
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "10.xx.xx.x:5000:5000" # Port used by the Web UI
      - "10.xx.xx.x:8554:8554" # RTSP feeds
      - "10.xx.xx.x:8555:8555/tcp" # WebRTC over tcp
      - "10.xx.xx.x:8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "xD ;-)"

I notice, some folks pass /dev/apex to the continer but i didn't and it works :-) It means you don't need any pci pass through inside virtual machine. Just start firgate docker and it will see coral TPU (two level of virtualization above ;-)).

frigate TPU

In order to use TPU as a detector you have to configure detectors and model in your frigate config. My looks like:

mqtt:
  enabled: false
cameras:
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://...............................................
          roles:
            - detect
            - record
    objects:
      track:
        - person
        - dog
        - cat
        - bicycle
        - car
    detect:
      width: 1920
      height: 1080
    record:
      enabled: True
      retain:
        days: 14
        mode: all   # possible all->motion (only perserve object with motion content)
      events:
        retain:
          default: 30
          mode: active_objects

database:
  path: /db/frigate.db

detectors:
  coral:
    type: edgetpu
    device: pci

model:
  width: 320
  height: 320
  path: /edgetpu_model.tflite

frigate logs:

2023-09-25 11:13:31.493684599  [INFO] Starting Frigate...
2023-09-25 11:13:32.888125253  [2023-09-25 11:13:32] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-09-25 11:13:32.910007524  [2023-09-25 11:13:32] peewee_migrate                 INFO    : Starting migrations
2023-09-25 11:13:32.915797842  [2023-09-25 11:13:32] peewee_migrate                 INFO    : There is nothing to migrate
2023-09-25 11:13:32.949654710  [2023-09-25 11:13:32] frigate.app                    INFO    : Output process started: 290
2023-09-25 11:13:32.960485295  [2023-09-25 11:13:32] frigate.app                    INFO    : Camera processor started for garage: 297
2023-09-25 11:13:32.971107311  [2023-09-25 11:13:32] frigate.app                    INFO    : Capture process started for garage: 299
2023-09-25 11:13:33.033638620  [2023-09-25 11:13:32] detector.coral                 INFO    : Starting detection process: 289
2023-09-25 11:13:33.034431749  [2023-09-25 11:13:32] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci
2023-09-25 11:13:33.042754705  [2023-09-25 11:13:33] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

final thoughts

It was... difficult and annoying... All because True NAS core is based on FreeBSD. I recommend True NAS scale which is free as core but based on Linux. I didn't switch to scale because of many "jails" on core. The migration will be difficult for me but I plan it in future.

On Linux based Free NAS this device it should work without all this difficult configuration.

[kymocode]

Great guide, thankyou.
I have the adapter from link 1 you posted. Works fine, as long as it is in my only 16x pci port. I have 2 spare pci x1 and an x4 but they prevent truenas from booting. The console spams
pCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
so fast its illegible. I managed to get this shell the only time it booted but my cpu was nearly 100% so I killed it.

Id like to move it to a 1x port so I can move my LSI card back to the 16x for cooling purposes.
Any ideas how to proceed? Am considering a fresh install. I can boot gparted and ubuntu live images just fine with the coral in any port.

[kwanty]

I have no idea :( In my case it works fine on 4x PCIe. Double check pptdevs parameter.

[eagle239]

(y) @kwanty I'm one of those that destroyed a motherboard, so your instructions are spot-on:

" If you connect it in other way you short-cut and demage motherboard/card."

if someone has skills to recover some use for such a damaged motherboard, please do reply. I now have a spare MB which doesnt work. I bought a new one so my NAS is now functional & up.

[kwanty]

It seams you short-cut 5V on PCI-x board. It is probably some kind of fuse on the mother board. Probably easy to fix :)

[DougLorenz]

I've finally got a setup on TrueNAS Scale working comfortably well. It's relatively easy to play around with the console and force TrueNAS to load the necessary Apex drivers, but future TrueNAS updates might just wipe those drivers out.

I'm using one of the Magic Blue Smoke adapters and a dual edge TPU. With an 8 camera setup, I'm getting 6.5ms inference times.

[eagle239]

Hi Doug,
I use the Magic Blue's dual edge TPU adapter also. How are you running frigate on TrueNAS SCALE ? I run the charts available on the truenas library, but that specifically doesnt work on the m.2 dual edge TPUs.
( https://github.com/truenas/charts/tree/master/library/ix-dev/community/frigate:
Note: m.2 Coral TPU devices is not supported. )

At least, I see both TPUs now: ( lspci -nn | grep 089a )
04:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
05:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

Please explain if you created your own helm chart, or just run frigate in a container. will be great if you can share your compose file and/or frigate config detector section.

[kwanty]

I'm using TrueNAS Core (FreeBSD). On TrueNAS Scale (Linux) it should be easier - no need to install Linux@VM on TrueNAS. I don't known how to install frigate on TrueNAS Scale. My docker-compose.yml config is above as well as Frigate configuration.

[eagle239]

on TrueNAS SCALE one easy option is to use predefined charts from the community....BUT the community chart for frigate does not support dual edge TPU :( I'd like to know how others have solved this, either create an edited/updated chart, or some way to run frigate in a container.

[DougLorenz]

The method I used to install a Coral dual TPU and Frigate on TrueNAS Scale is a bit sketchy. The Coral drivers, aka "Apex" drivers need to be installed in the OS. The directions to do this on a regular linux installation are here:

https://coral.ai/docs/m2/get-started/#2a-on-linux

The developers of TrueNAS don't want you to be able to do this, so they have disabled the APT command. To re-enable APT:

chmod +x /usr/bin/apt*

This has worked for me, but there is a major concern. If you ever update your TrueNAS Scale OS, it will likely erase these changes.

[eagle239]

Yes thanks, that's what I've done also so far. My OS can see 2x TPUs under lspci. But I'm not home so can't check which specific chart I used. I had posted my chart link on the GitHub site. I plan on getting back in 2 days and to get the chart you mentioned. Thanks & Regards Ash

…

On Sat, Dec 30, 2023, 6:28 PM Douglas Lorenz ***@***.***> wrote: The method I used to install a Coral dual TPU and Frigate on TrueNAS Scale is a bit sketchy. The Coral drivers, aka "Apex" drivers need to be installed in the OS. The directions to do this on a regular linux installation are here: https://coral.ai/docs/m2/get-started/#2a-on-linux The developers of TrueNAS don't want you to be able to do this, so they have disabled the APT command. To re-enable APT: chmod +x /usr/bin/apt* This has worked for me, but there is a major concern. If you ever update your TrueNAS Scale OS, it will likely erase these changes. — Reply to this email directly, view it on GitHub <#5932 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXX6RJMANCJVU3RZFD3MZPTYMCPTDAVCNFSM6AAAAAAWXKZT6CVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TSOBQHE2TG> . You are receiving this because you commented.Message ID: ***@***.*** com>