Does anyone know if I would be able to get a passport for my baby while DH is deployed if I have a POA?
No, you can't use a POA. There is a specific form you need from the state department. He will have to sign it and have it notarized.
ETA: it's on state.gov in the passport section. There's a menu on the right side and one option is "forms". The last choice is "minor consent". That's what you need.