Xiao BLE nRF52840 flaky scanning

We’re considering switching our project over to using the Xiao BLE nRF52840 instead of the Arduino Nano IoT, but when porting over the BLE functionality (using ArduinoBLE) the Seeed module fails to scan reliably after 30 minutes or so. The scan is using localName matching

BLE.scanForName(PERIPHERAL_NAME, true);
unsigned long startTime = millis();
while(millis() < startTime + 5000) {
  BLEDevice scannedDevice = BLE.available();
  bool isPeripheral = scannedDevice.localName() == PERIPHERAL_NAME;
  if (!isPeripheral) continue;
  Serial.print("Found possible sensor");

The scan goes for 5 seconds, then there’s a 5 second break. When I initially start running this code the device is scanned at least once every scan cycle, but after 30 - 60 minutes it barely can pick up the peripheral once every 5 scan cycles. Using the nRF Connect app shows the peripheral advertising when it should be with 100ms advertising interval.

Reliable BLE scans are the most important part of our project so is there some way to fix this? There are no issues here currently with the Nano 33 IoT

The entire module also freezes up after several hours, the serial communication crashes and the GPIO pins continue outputting whatever values they had before the freeze occurred