You shouldn't be worried about anything at all. Guys have this sort of thing happen occasionally, and it's not abnormal. In the meantime, you, knowing he was having a problem really pushed him, too. And every time it failed, he felt worse & worse.
He did complain about being hot and sweaty......that could mean he's got a cold, or the flu, or he's just not feeling well. If that's the case, again, you pushed too much. We all have libidos, but we should also be in control of ourselves if there's a problem.
The first time he failed you should have just left him alone, at least for that evening. You're right, for some guys it's a very sensitive subject, and the more it happens, the more he worries about it, and it becomes a "self fulfilling prophecy". He's so busy thinking about it and worrying about it, that it happens again. The more you try, the worse it becomes. Maybe you're just pushing him too hard......in a couple of days, you went at it 8 or 10 times. It could be that he's just not THAT interested, and he's worried about "keeping up" with you.
There's a lot more to a good relationship than spending a week-end in bed having sex. Maybe you need to get out of the house and DO some thing.......go to a movie, go to the park, get together with friends...... anything to take the pressure off him.
I would worry more about him trying to keep up with you, and causing psychological anxiety. You getting upset does't help, either. He knows your upset, and he apologizes, but he really can't help it. IF it happens again, then you need to discuss it OUT of the bedroom, and ask him what HE thinks the problem is.
