I built a slanty shanty with the key to the city. I put some manure on the floor and went outside. I went back inside and picked up the manure, and the fly sounds didn't stop. I got rid of the manure altogether, and the fly sounds didn't stop. I could hear the fly sounds inside other buildings as well, where there was never any manure.
Put manure on floor in slanty shanty, go outside, go back in, pick it up. I don't know if it happens every time. But that's what seemed to do it for me.