Status Update:
I flashed the system with the guide provided by Seeed here.
It allowed both CSI ports to work correctly. However, it is missing all the libraries that normally come with JetPack, like Cuda, VPI, TensorRT etc. and they do not appear easy to actually install. So I cannot run or build jetson_inference, which I need.
I flashed again using the nVidia SDK Manager and it now has all the libraries correctly installed, so jetson_inference works. However, the CSI CAM1 stopped working again, with the same error as before.
Ideas to diagnose issue?