While Borel was lecturing to an audience of several hundred mathematicians at the famous Bourbaki seminars, a baby began screaming at the back of the room. Borel paused until the noise had died down, and then looked out at the audience and asked: "Question?"
In the early sixties, Grothendieck visited Harvard while Zariski was still a faculty member. Once, while Zariski was lecturing in a seminar, Grothendieck kept asking him why he didn't prove his result for all schemes, not just varieties, but Zariski simply responded that it didn't work. Eventually, Grothendieck could stand it no longer and went to the blackboard and began writing down a proof for schemes. While he did so, Zariski wrote down a counter-example. When Grothendieck realized he was wrong, Zariski said (in his heavily accented Russo-Italian English) "In my time, I have had to learn many languages." At this, Grothendieck turned bright red from embarrassment.
Another time Zariski was lecturing and Grothendieck again asked him why he didn't generalize his work to schemes. This time Zariski merely said "Now now Alexander, we must show some self control."
In a discussion with Grothendieck, Messing mentioned the formula expressing the integral of exp(-x2) in terms of pi, which is proved in every calculus course. Not only did Grothendieck not know the formula, but he thought that he had never seen it in his life.
A mathematician was explaining his work to Tate, who looked bored. Eventually the mathematician asked "You don't find this interesting?" "No, no" said Tate, "I think it is very interesting, but I don't have time to be interested in everything that's interesting".
As a thesis topic, Tate gave me the problem of proving a formula that he and Mike Artin had conjectured concerning algebraic surfaces over finite fields. One day he ran into me in the corridors of 2 Divinity Avenue and asked how it was going. "Not well" I said, "In one example, I computed the left hand side and got p13; for the other side, I got p17; 13 is not equal to 17, and so the conjecture is false." For a moment, Tate was taken aback, but then he broke into a grin and said "That's great! That's really great! Mike and I must have overlooked some small factor which you have discovered." He took me off to his office to show him. In writing it out in front of him, I discovered a mistake in my work, which in fact proved that the conjecture was correct in the example I considered. So I apologized to Tate for my carelessness. But Tate responded: "Your error was not that you made a mistake -- we all make mistakes. Your error was not realizing that you must have made a mistake. This stuff is too beautiful not to be true."
During a seminar at Harvard, a conjecture of Lichtenbaum's was mentioned. Someone scornfully said that for the only case that anyone had been able to test it, the powers of 2 occurring in the conjectured formula had been computed and they turned out to be wrong; thus the conjecture is false. "Only for 2" responded Tate from the audience. [And, in fact, I think the conjecture turned out to be correct except for the power of 2.]
Finally, a story to keep in mind the next time you ask a totally stupid question at a major lecture. During a Bourbaki seminar on the status of the classification problem for simple finite groups, the speaker mentioned that it was not known whether a simple group (the monster) existed of a certain order. "Could there be more than one simple group of that order?" asked Weil from the audience. "Yes, there could" replied the speaker. "Well, could there be infinitely many?" asked Weil.
Once upon a time, when André Weil was still editor of the Annals of Mathematics, he wandered down to the basement of Fine Hall and found there a stranger with a weird machine. When asked what it was, the stranger replied that it was a time machine. Of course Weil was deeply sceptical, but he nevertheless allowed himself to be given a demonstration... and sure enough, he found himself in Fine Hall ten years in the future. He went to the library and was delighted to discover that the latest issue of the Annals contained an elegant proof of the Riemann hypothesis. Weil returned to the present, and the stranger and his machine vanished, perhaps off to a more pleasant age. As the time approached for the issue of the Annals, Weil waited anxiously for the article, and to discover who the author was, since it had been signed xxx. However, no article came, and the publisher began pressing Weil for the copy for the issue. Eventually, it was not possible to wait any longer, and so Weil sat down and wrote the article himself --- he was able to recall it perfectly --- and he signed it xxx.
Sources: (2,6,7) I was there; (1) told to me by Borel; (5) told to me by Messing; (3,4,8) told to me by other students when I was a graduate student; (9) hearsay; (10) folklore among the residents of Fine Hall.