Respeaker 2mic hat pop sound when start/stop, change track

So I have Reaspeaker 2-mic hat connected to raspberry pi zero and using it as a little mp3 player. All works almost perfectly, the only problem is when the music starts/stops and I am changing the tracks there is a distinctive “pop sound”. I have been searching the internet and trying to find a working solution but could not find any satisfying answer to remove this issue. I am using mpg123 and ALSA to play the mp3s. I presume it is related to not keeping the driver open and the voltage difference creates that sound on the speakers. I need your help as this drives me crazy and it is probably destroying my speakers.