Return to breastfeeding after exclusive pumping

my baby is 4 weeks old and has been bottle fed expressed milk basically from the beginning. We had painful latch issues and neither one of us was sleeping in the hospital. I had surgery for retained placenta immediately after delivery and he was on antibiotics for an infection as well. We were both exhausted after trying sns with a nipple shield so he got a couple bottles of donor milk. When we got home it was still incredibly painful and he was losing a lot of weight. Lactation consultant had us nursing for 15 minutes from each side, then pumping 15 minutes and offering whatever came out. He would always eat all of it. After a day of that exhausting schedule I stopped trying to nurse at all. We had one more consult and it was determined that, despite having a strong suck and no visible tie issues, baby wasn't pulling anything from my breast, despite the fact that I had plenty of milk available. He'd be latched and sucking the whole time, but end up dozing off since it was so much work and no food. So I've been exclusively pumping since, and he's gaining weight like a champ. I'd like to return to breastfeeding now that he's on track with weight gain. We've been trying today and the latch is no longer painful. This morning he nursed for 15 minutes from my right breast and then I pumped both for 10 minutes. My left breast gave me almost 3 oz while my right only gave me 1. It seems like he must be pulling enough out, but he was still fussy after he let go of the breast (refused the left breast as well) so I gave him a bottle and he ate another oz. Later this morning he nursed for about ten minutes and fell asleep, which was part of the issue we had before. I'm going to go tomorrow to try and do a weighted feed at the lactation clinic but in the meantime I'd love suggestions for avoiding the bottle and making sure he's fed enough. I'm worried that it's been too long and he won't eat from the breast easily now. Thanks in advance!