should I call dr?

I was pretty sure I had a breast infection. Two nights ago I had a fever of 100.4. My left breast was EXTREMELY sore in one spot. Yesterday the fever went down to 99.5, but my breast had red marks where it was sore. I called my dr but they were not in so I planned on calling today. Today, I don't have a fever anymore, and the spot is less sore but still red. Could this be getting better?