Sudden massive lag

Hi!
I have a really odd issue and can't figure out what it is. Everytime there are too many prefabs on my screen, my game lags so much (i go straight from green to red) that i simply can't move. I play with lag compensation off, but if i turn it on i'll be able to walk but i get massive rubberbanding.
I've died so many times to this that i even avoid going to caves now (earthquakes are terrible!) or passing through many killer bee hives. Funny thing is this doesnt happen when i'm alone (when no one else or few people are on the same shard as me, caves or surface). As soon as 5/6 people join in (i'm assuming this number, i'm not sure how many people it takes) i'll start getting that huge lag whenever i have too many prefabs on my screen.  Like i said, it happens with quakes and killer bees but it can happen to basically everything else such as a massive spider field.
Note that, if i'm playing in a private server by myself, i NEVER get this issue, so it's not a hardware problem (i hope).
I may be wrong, but as far as i remember, this didnt use to happen when i started playing the game, around May this year. I tried disabling all the mods one by one but sadly it didn't help.