Well, we did it! Five families agreed on a house and it is reserved! Of course we have yet to travel....so we'll see how it all works out. But I am feeling optimistic!
Thanks again for all your advice. In the end I got creative and figured since we wanted to stay 7 nights and everyone else wanted to stay only 4, we just invited friends that live nearby to join us for the other 3. It is a big added bonus to get to spend a few days with them as well! And I was able to negotiate a discounted rate for those 3 days since only two families (4 adults and 2 kids) are staying in a house that easily holds 14. I also realized that my DH really needed to step in and deal with his family. I get along with my in laws but this is so far out of their comfort zone that I felt I was making them more uncomfortable and DH was able to smooth things over.
So we'll see how it goes! This will either become a fun family tradition or a "never again". Honestly, I'm fine either way but I'm excited that everyone is willing to give it a try.
Re: family vacation @ beach - update