Assuming these are mechanical encoders like the common KY-040 style devices:
Do NOT connect the rotary encoder 5V pin to 5 Volts!
This is connected to pullup resistors to the data and clock (and switch if the encoder has one). You do NOT want the signals to the CPU to go up to 5 Volts. Connect it to 3.3V
If this doesn’t help, post the Specification/DataSheet for your encoders.
Regards,
Dave