That is NORMAL operation, The Boot mode com port is Different from the Runtime USB port.
I must select port and device EACH and EVERY time in the IDE for the flow to work properly.
Note: After a succefull programming and reset . It works every time.
Once in a BluMoon it won’t OR if I flash KAKA code 
A,K,A, Garbage it won’t Auto find the programming USB port and I’ll have to Play a Game by 'Clicking Upload" and pressing reset on the Xiao at the right time and it catches it" Takes a few attempts but has eventually worked (happened a few times) I have over 20 Nrf52840 Sense Xiao’s and 3-4 BLE only Xiao’s
Once a good upload is achieved the problem goes away and subsequent port enumeration, uploads and flash work normal.
HTH
GL 
PJ
p.s I have removed all ghosted USB ports from Device manager (show Hidden) when installing or Starting IDE on a New or different Windows Machine, YMMV 