I had a similar issue and found no fix so far.
See Xiao ESP32-C3 and Xiao Grove shield compatiblity? - #5 by reivilo