Ambiguous references happen in the real world all the time. For example, the name Big Ben is used to refer to the clock tower and to the bell, but strictly speaking only one of those is right.
Most of the time these misplaced names don’t matter - listeners make assumptions about the intended referent and correct those assumptions as more information comes to light.
A lot of the time the possible referents share a lot of characteristics and can be used interchangeably. When giving directions you can say “head towards Big Ben” without fear of confusing people. You could even get away with “listen to the sound of Big Ben” because sound is perceived to come from a general location. The information a speaker produces Big Ben can be inconsistent for quite a long time before it becomes a problem.
But what happens when the speaker says something that crystallizes that inconsistency for the listener?
Suppose the speaker believes that Big Ben refers to the bell and the listener assumes it refers to the tower. The speaker suddenly says:
“Big Ben was cast in 1856 in Stockton-on-Tees”
The listener realizes now that they have been talking about different things.
“Oh, I thought you were talking about the tower”, the listener might exclaim.
“No, you ignoramous, the tower is called Victoria Tower. Big Ben is the bell inside it! How can a tower possibly be cast in a northern town?” retorts the speaker.
“Well, you learn something new every day,” says the listener ruefully.
Now the listener backtracks and applies all the information they had learned from the speaker about Bug Ben to the bell instead of the tower.
However, throughout this conversation neither the speaker nor the listener assumed the name Big Ben applied to two things at once. They both assumed it applied to a single referent, but they happened to be different ones. Once more information was shared then they managed to reach agreement on the actual referent for Big Ben.