Tested hosting on two local servers, adjoining biome type doesn't have an effect, issue appears with and without mods. Approaching the swamp biome causes the game to crash.
Loading the game and hosting the server, and walking away from the swamp allows it to continue running.
Load server (with caves), walk towards any swamp biome. Game crashes when approaching (not reaching, crashes before you can enter). A save with a character close enough will never load (game crashes) and has to be rolled back. Same game will run fine until process of walking towards the swamp is repeated.