No, you can't use a POA. There is a specific form you need from the state department. He will have to sign it and have it notarized.
ETA: it's on state.gov in the passport section. There's a menu on the right side and one option is "forms". The last choice is "minor consent". That's what you need.
Talk to your base passport agent. My DS's father was overseas when I had to get his, there was just a separate form I had to fill out stating that it wasn't possible to get the other parent's signature. I believe I may have had to provide deployment orders as proof, but still no biggie.
Re: Passport for Baby while DH Deployed
No, you can't use a POA. There is a specific form you need from the state department. He will have to sign it and have it notarized.
ETA: it's on state.gov in the passport section. There's a menu on the right side and one option is "forms". The last choice is "minor consent". That's what you need.