Ones like the pp pictured are worth it, yes. Both of my kids really liked laying under it from basically the beginning (on a boppy) then as they got older they learned to reach for the toys and after that it would encourage them to roll over and reach for things. In fact, we'll be getting a new one for this baby because our last one is all but worn out/broken/missing, etc. We had one that had some lights and songs, it was definitely a hit.
Are you referring to the mat that lays on the floor with the bars over it with the toys hanging down?
We had the above one for DD and she loved it - she used it on her back from about 2/3 months all the way through sitting up, finally put it away around 8/9 months maybe. I would recommend it if you have the space (not necessarily THAT particular one, although it's good - really, they're all pretty similar IMHO and it doesn't much matter which one you get).
If this is what you are referring to OP then it is totally worth it. We used the activity mat for tummy time. The mat coupled with the Fisher Price 1-2-3 Crawl Along Snail were so helpful for keeping DS entertained during tummy time.
Re: Baby gym