Baby on separate floor?

I live in a row home and our bedroom situation is unique with one large bedroom on each the second and third floors. Each has a bathroom, one with shower one with tub. Main living area and basement are below and will be our living areas. We are currently on floor 2 with the tub and my hubby wants to stay there and put the baby upstairs so it is quietest for them. I cannot decide if we should stay or move up. Any opinions or similar situations anyone can share to help!?? Baby will prob sleep in our room directly for first few months regardless.