Hi everyone,
After adding an NVMe to my Seeed R2000, I started running into an issue. The Hailo device failed to initialize.
Initially, I tried running the following command to identify the Hailo device.
eli@raspberrypi:~ $ hailortcli fw-control identify
eli@raspberrypi:~ $
But it returned nothing. After checking the dmesg logs, I found this error in the screenshot related to the Hailo device.
After encountering this problem, I ran sudo apt update && sudo apt full-upgrade, which resolved the issue with the Hailo device. However, now the NVMe no longer shows up with the lsblk command.
eli@raspberrypi:~ $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
mmcblk0 179:0 0 58.3G 0 disk
├─mmcblk0p1 179:1 0 512M 0 part /boot/firmware
└─mmcblk0p2 179:2 0 57.8G 0 part /
eli@raspberrypi:~ $
Has anyone faced a similar issue after adding an NVMe ? Any suggestions on how to resolve this?
Thanks in advance for your help!
It looks like you ran into a device conflict issue after adding the NVMe SSD to your Seeed R2000, which caused the Hailo accelerator to fail with:
hailo 0000:06:00.0: Failed to enable MSI -28
hailo 0000:06:00.0: Failed enabling interrupts -28
The -28 error typically maps to ENOSPC (No space left on device), which often means that system resources like interrupt lines or MSI (Message Signalled Interrupts) are exhausted or misconfigured—likely due to the addition of the NVMe device.
Then, after you did a full upgrade, the Hailo device started working—but now the NVMe is not detected by lsblk.
1 Like
Hi there,
Just remove the Hailo device , and go again, Adding it LAST!
Have a look at the WIKI also, for more information.
Do follow the steps and only use images from there.
HTH
GL 
PJ 
I have the same or similar problem with the Seeed Studio reComputer AI R2000 model R2130-12. I use the Seeed Studio Raspberry PI 27 watt AC/DC adapter with the R2000. The Hailo-8 accelerator works fine, that is the Hailo-8 driver always loads if the Hailo-8 accelerator is the only card in the Seeed Studio PCIe3.0 to dual M.2 hat for Raspberry PI 5. The Hailo-8 driver also loads regardless of which M.2 key the Hailo-8 accelerator is installed.
Note: The PI OS and eeprom is always the latest version (updated/upgraded with sudo apt update && sudo apt upgradeable and with the sudo raspi-config command), checked many times.
The problem is the Hailo-8 driver never loads if any type of NVME SSD driver is in either M.2 key (blank, boots from, etc… For the Hailo-8 driver, I find the message "Failed to enable MSI -28 " listed in the startup log (dmesg | grep -i hailo). I have tried several versions of the Hailo driver. Reinstallation of the Hailo-8 accelerator and its driver is not the solution, although I have tried this multiple times. If I power down the R2000, remove the NVME SSD, leaving only the Hailoi-8 accelerator in the dual M.2 hat, restart the rR2000, the Hailo-8 driver loads and the Hailo-8 accelerator works fine - same Hailo-8 software, ex. the rpi examples tests from Hailo run correctly. When I put any NVME SSD back in the R2000 (power down, etc) and start up the R2000, the Hailo-8 driver does not load. I have also used the lowest power, Raspberry PI certified dramless NVME SSD I have, a 512 GB 2230 NVMD SSD (Biwin Storage, uses 3.3V at 2 amps). Note. The NVME SSD always works.
This started out when I loaded with loading the Raspberry PI OS with a network installation on a SD card (press shift while the R2000 is powering on for the first time and selected the network installation option for a blank SD card). The Hailo-8 accelerator worked fine after the initial installation. I then moved the PI OS image to a NVME SSD and booted from the NVME SSD. I find that using a SD card for AI type work is not a viable solution (too many writes to the SD card and it is too slow).
The Hailo-8 accelerator is always displayed with the lspci command, without a NVME SSD installed and also with a NVME SSD installed.
0001:06:00.0 Co-processor: Hailo Technologies Ltd. Hailo-8 AI Processor (rev 01)
When the NVME SSD is also in the M.2 hat, the Hailo-8 driver starts to load and then fails. regardless of the version of the Hailo-8 driver (version 4.19 displayed below on a fresh installation of the PI OS; I also used version 4.20 and 4.21 in other tests).
[ 4.367572] hailo_pci: loading out-of-tree module taints kernel.
[ 4.376702] hailo: Init module. driver version 4.19.0
[ 4.390804] hailo 0001:06:00.0: Probing on: 1e60:2864…
[ 4.390811] hailo 0001:06:00.0: Probing: Allocate memory for device extension, 11632
[ 4.390830] hailo 0001:06:00.0: enabling device (0000 → 0002)
[ 4.390836] hailo 0001:06:00.0: Probing: Device enabled
[ 4.390854] hailo 0001:06:00.0: Probing: mapped bar 0 - 0000000064485481 16384
[ 4.390858] hailo 0001:06:00.0: Probing: mapped bar 2 - 00000000e06a8bbc 4096
[ 4.390860] hailo 0001:06:00.0: Probing: mapped bar 4 - 00000000ec0a1d16 16384
[ 4.390863] hailo 0001:06:00.0: Probing: Force setting max_desc_page_size to 4096 (recommended value is 16384)
[ 4.390873] hailo 0001:06:00.0: Probing: Enabled 64 bit dma
[ 4.390875] hailo 0001:06:00.0: Probing: Using userspace allocated vdma buffers
[ 4.390879] hailo 0001:06:00.0: Disabling ASPM L0s
[ 4.390882] hailo 0001:06:00.0: Successfully disabled ASPM L0s
[ 4.390945] hailo 0001:06:00.0: Failed to enable MSI -28
[ 4.390947] hailo 0001:06:00.0: Failed Enabling interrupts -28
[ 4.390949] hailo 0001:06:00.0: Failed activating board -28
[ 4.390957] hailo 0001:06:00.0: probe with driver hailo failed with error -28
The NVME SSD always works. The R2000 boots fine from the NVME SSD, with the Hailo-8 accelerator installed or without it, and regardless of which M.2 key is used. The led light on the Hailo-8 accelerator is always red when the power is on.
As insufficient power could be a cause, on the PCIe 3.0 to dual M.2 hat for Raspberry PI 5, I checked if the power was also being received at the pogo pins. The M.2 hat gets power from the pci cable and also from the 40 pin connector via pogo pins. From the pogo ground pin, and another pogo pin, I measured 5 volts DC. I measured 5 volts DC with a third pogo pin and the ground pin. On a fourth pogo pin and the ground pin, I measured 3.3 volts DC. On a fifth pogo pin and the ground pin, I measured 0 volts. Power is getting from the 40 pin connector to the dual M.2 hat.
I have yet to find a fix for this. I have been able to use a USB 3.0 connected SSD drive with the Hailo-8 accelerator on the R2000. This setup would work for AI work. I still prefer to be able to use the second M.2 slot for a NVME SSD as this is more compact and faster, but so far without any luck.
