Thanks for the feedback.
This is not an issue with the ADC. This is noise picked up by the preamp. Noise from the ADC would show up the same on the screen regardless of the vertical range used.
I had a similar problem with my unit, causing thickening of the trace on ChA in full speed mode similar to yours. Turned out to be a bad ground. The metal frame that holds the LCD is grounded by a stick-on metalized tape to one of the input or generator output jacks and it was making an intermittent connection. You could move the plugs around when inserted into the jacks and the noise would cut in and out.
In any case, this is definitively noise picked up by the front end in the input section. The device switches in a different op-amp when set to ranges of 1V/div and above. Looks to me like noise from the rest of the circuitry is being leaked into that section, possibly from a bad ground, capacitive coupling, power supply issues, etc… ChA seems more susceptible to this for some reason, probably because of the location of the circuit traces on the board. The LCD is also a prime cause of radiation into the circuitry if it’s frame is not properly grounded, since it covers the entire board. The grounding method is rather poor, relying on simple pressure of part of the frame onto the metalized stick-on tape.
As I mentioned previously, the full speed mode will, by it’s more sensitive nature more readily reveal any noise, whether generated by the circuit you are monitoring or generated inside the scope itself. This is in fact part of what makes it useful. Also, loose or defective probe plugs making a poor ground connection where inserted into the input jacks can cause problems like this.
You could try reverting to an earlier FPGA, it’s possible the extra circuitry for the full speed mode is causing an increase of noise on the supply lines, and an increase in this noise, but since it’s only appearing on some of the ranges, I suspect that some hardware issue in the preamp is causing this, and that it would also show up with earlier FPGA’s, though possibly to a lesser extent. Notice that the “spike” part of the noise shows up in your screenshots even when not in full speed mode.