Is it safe to use juice in a 4 week olds bottle for....?

constipation? He's on enfamil prosobee due to a milk protein allergy and it makes his stools pebble like and difficult for him to get out. We've tried Karo Syrup but had to use so much to get a result that it would make him gag and spit up after each bottle. What kind of juice should I use? Is apple okay? And how much? He drinks 4 oz of formula every 3 to 4 hours. Thanks