The iPhone 4s, like the recent iPads, seems to deliver less power to the headphone power. As with the iPads, supplying external power solves the problem nicely.
Of course, be sure you have the sound volume turned all the way up before resorting to an external power source. 