Thanks for your replies, and you are right that with 3.7V, the battery will not be able to power the GPRS. But when the battery is fully charged (4.1V), then I have found that it is able to drive the GPRS shield for some time.
Right now, as I have it, I have the GPRS shield, Grove shield, battery and solar panels, mounted as per the Seeduino stalker configuration.
I have removed the USB5V external supply (removed the UARTsBee), and when I solder a wire connection between the BAT (bottom of the Arduino board) and the USB5V, it was able to power on the GPRS shield and work for some time (till the battery level fell below 4 V). The small time was enough for me to upload the Arduino values to the remote server and serve my purpose! And when the solar panel charges the battery again to its max, I can program the GPRS shield to start working again and send its update.
I took a look at the circuit diagrams, and apparently, it seems to be ok if I connect the BAT output back to USB5V. But I just wanted to make sure in the forum that this was correct and it would not cause damage to the board.