XIAO ESP32C3 two AAA batteries (3V each) connected to the Li-Po terminal

You would still need a buck booster from on the VIN/GND from 3.5 to 5v to keep the voltage stable I would assume as we lock out the C port.