If there's more than 2 Willow around, Bernie changes his owner if the other Willow has active Bernie! for her.
When I experienced this situation I mastered all 9 Bernie skills, and the other Willow didn't have any of the Bernie skills, while being not insane so she couldn't have active Bernie! for her. Interestingly, I still can lit up her Bernie, and the appearance of lunar affinity my Bernie had was remained.
1. There's 2 Willows, one has her active Bernie! and other doesn't.
2. Suddenly the Bernie changes his owner, following other Willow.