For me, when doing a board recovery, I take it out of board carriers, and connect using the built in usb CDC port.
The CMSIS-DAP usb interface is actually provided by a second mcu on the same xiao, so try that way first, as the onboard debugger is virtually impossible to kill via software.