Questioning Pediatrician

My 5mo DD came down with a virus, the lovely reality of having two kids, everyone told me it was inevitable that my older DD would give something to my baby, well she did.

I took the baby in yesterday to just be on the safe side. My pediatrician is out all week so they had me see one of her partners, who I've never seen before. When they put me in one of the rooms there was a woman and boy accross from us that I could hear was very sick so I closed the door over as they were not practisiting safe coughing and I didn't want my LO to be exposed to anything else. I could hear the doctor order a test for the boy testing for influenza B which I the heard came back positive. The doctor finished explaining how to take the medicine to the woman and came into our room to examine my DD. The doctor came in and immediately began to examine my baby. He was listening to her heart and she placed her mouth on his hand to which I freaked out and started questioning that he NEVER WASHED HIS HANDS WHEN HE CAME IN. So I said to him you just treated a child and his mother who have influenza B and you didn't wash your hands before touching my child. He brushed it off and said I washed them before I came I here. I've been thinking about this all day and night and I could swear that my ped washes her hands before and after treating my daughter. I don't know what to do. Influenza is deadly for babies and that's just nasty in my opinion. I want to report it. I'm not certain he exposed my baby to anything but what about another child....? Am I being paranoid??