My husband and I have been to Cape May the last two summers and have loved it. We plan to go back next year. We stayed at a hotel so I can't give you any advice renting a house, but there are tons to rent there. There are things to do with younger kids, I see them there all the time. There is a cute lighthouse near Cape May which I would recommend checking out. Also there are horse and carriage rides which might be fun. If you're looking to save a little money there is a good grocery store where you can get most everything, I think it's called Acme. Check out this website: https://www.capemay.com/ is has a lot of information.
Re: cape may or LBI?