A couple suggestions:
1.) Meetup.com ... you might find a group that meets your desires... A lot of them do get togethers fairly often.
2.) Library Story time... this is the link to events at Fulton County Libraries. You can search for what county/area you are in. https://www.afpls.org/events/events-calendar
3.) We really enjoy Gymboree Classes. They do cost though... You can try a class for free though to see if it is for you guys!
I am sure some others will have more ideas on here!
I did it with both my boys. Ridge started at 3 months and Colton 6 months. It was fabulous.
You can also do The Music Class. They loved it at that age.
When they get a little older there are more things to do.
A girl I went to highschool recently opened up a really neat kids place in Buckhead. I haven't been yet but am planning to in the next few weeks. They have free-play time (which is $8, free for babies under 10mos) as well as classes.