Can place walls close to structures but not vice versa

Trying to place a wall next to a structure (such as a crafting station) works fine; the walls can be placed very close. However, trying to place a crafting station near a wall has a 3.2 tile keep-out radius. This is extremely frustrating when building and the walls have already been placed. It really sucks having to smash a bunch of walls just to build them again in the "correct order".

Here's what I'm trying to do, and you can see just how many walls I'll have to smash (a lot!) just to place the structure.