Based on the tests, I have traced it down to that something crazy happens as soon as I enable the TIM1 timer. I’m using it to drive the DMA that transfers data from FPGA to memory. Right before I enable the timer, I write the text “Enabling capture TIM1” - and as soon as I get the DMA interrupt, I draw a bunch of red pixels. I really don’t have a good theory where on earth it magically jumps in the between, because obviously it does not execute the main program but it doesn’t execute the interrupt routine either.
Here is a yet another version. This time the HardFault handler can survive a corrupted stack pointer. I’m hoping so much that it would just give the red screen 
Almost like using Windows, hoping all the time to see the blue screen, right?
http://kapsi.fi/~jpa/stuff/other/LOGIC005.HEX
And regarding the differences; I realized that we may have different bootloader versions because it is not upgradable through USB. Mine is “Device Firmware Update V3.10”. Using this application (APP3 slot) you can dump your whole flash to flash.bin. Then I can load the file to my device using the programming port on the PCB and then our devices should be exactly identical by code:
http://kapsi.fi/~jpa/stuff/other/BOOT_APP.HEX
A quite probable theory is that we have different revisions of the STM32F103VCT6 processor. Unfortunately, one of the bugs in that processor is that it is impossible to read processor revision (DBGMCU->IDCODE) by software!