nRF52840 Sense - hardware UART not working

Products & Technology XIAO
#1

With my nRF52840 I’m experiencing issues with the hardware serial functionality for UART. It just doesn’t seem to connect whereas in Software Serial things work perfectly fine.
E.g. when communicating with an Adafruit Soundboard I have a code like that:

#include <Adafruit_Soundboard.h>
#include <HardwareSerial.h>

#define SFX_RST 8

Adafruit_Soundboard sfx = Adafruit_Soundboard(&Serial1, NULL, SFX_RST);

void setup() {
  Serial.begin(9600); 
  Serial1.begin(9600);

  if (!sfx.reset()) {
    Serial.println("Not found");
    while (!Serial1) { /* wait until it is connected*/ }
  }
  Serial.println("SFX board found");
  uint8_t files = sfx.listFiles();
  Serial.print("Found "); Serial.print(files); Serial.println(" Files");
}

It can’t communicate with the device (finding 0 files). However when I’m switching to SoftwareSerial the same code looks fine.
E.g.

#include <Adafruit_Soundboard.h>
#include <SoftwareSerial.h>

#define SFX_TX 6
#define SFX_RX 7
#define SFX_RST 8

SoftwareSerial ss = SoftwareSerial(SFX_TX, SFX_RX);   // <---- this worked, hardware did not?!
Adafruit_Soundboard sfx = Adafruit_Soundboard(&ss, NULL, SFX_RST);

void setup() {
Serial.begin(115200); 
ss.begin(9600);
if (!sfx.reset()) {
    Serial.println("Not found. Waiting...");
    while (!ss) { /* wait until it is connected*/ }
}
Serial.println("SFX board found");
uint8_t files = sfx.listFiles();
Serial.print("Found "); Serial.print(files); Serial.println(" Files");
}

Any idea what is going on? Do I have to “enable” hardware UART ports somehow and I missed that?

Thanks a lot!