Don't Starve Together (Not Responding)

Any time I start up the game, try to go into a server, go into caves, or leave the server the game freezes up and I get (Not Responding). This happens with or without mods. On the Normal branch or beta. It happens no matter what I do. On friends or my own server, Dedicated too. After it freezes up I have to wait about 2 mins or a max of 5 minutes before it responds. I can miss a all game day trying to load into a server.