Me too! Lots of problems with the XIAO esp32s3.
I just assembled about 15 battery driven low power climatic sensors. I had a few dying onboard LEDs. Killing them just by switching them on to show that the configuration website of the sensor is active. And even worse five out of 15 sensors are refusing to run from a 18350 battery which I carefully soldered to the +/- pads. The sensors are all build exactly the same and I can measure the 3V3 at the according pin - no matter if connect them to USB or battery only. So I think there is no problem with the battery and the onboard voltage regulator but even a simple three line blinky program (using those sensors with a still working LED and without use of Serial.something()) will only run if the device is powerd by USB. When I pull the USB cable the device stops blinking immediately and goes dead. Ok, I may connect the battery to the VBus-pin (5V pin) and the device will start running. According to the wiring diagram the VBus-path and the VBat-path both are using the same 3v3 voltage regulator. But I just don’t get it why the USB power can drive the sensor but the battery can not using exactly the same regulator and providing 3V3 as well. At least is that what I can measure at the 3V3 pin. Crazy!
May be I should use another board.