This has been happening to both me (the host), and my friends. Basically, we try to press X (PS4) on something, we get stuck trying to interact with it. If we try to walk away, the game forces us to walk back to it. For example, it makes us open and close Chester repeatedly, but it also happens on sticks, grass, players, ect. Leaving the game and rejoining fixes the issue.
Interacting with things, anything it seems.