What should i do in these kinds of situations?

Ok, heres the thing, some of these situations i didnt know what to do in and theyve been bothering the heck out of me, like this one i was playing racquet ball with this fat guy, not too fat like he had a belly, but during the game i started getting nervous over loud noises that he was making when he served the ball, and i mean real nervous where my legs were shaking, i dont know why that happenned, then i lost but i never played this guy before but my dad did and he kicked that fat guys a**! And after that he was talking to my dad (i was right by the fat guy) and they were gonna play 2v2 and the fat guy hecka screamed out his friends name right by me which made me shrugged my shoulders and i said "dang man!" and his friend was just on the couch not even that far he was like 5 to 6 feet away and i sear to god i wanted to punch that fat a** right in the face so bad when he did that, im serious, i was so angry, but then again violence wasnt the answer probably i dont know maybe and then another time when my bros friend was given me a hair cut and the guy was cutting my hair was saying "so you want to be as big as james huh?" but james was like mostly fat and a little bit of muscle and i said "well thats too big" my bro and his friend laughed except james did and he said " well mike has the jokes today, then i couldnt hear him that much cuz of the buzzering noise the shaver was making and he said something "beat his a**" then my bro said to him "hey shut the f*** up man" and james said " you man!" so immiediatly in my head i was thinking was he talking about me? I couldnt really decide so i just didnt really think about it till later on the week that happenned and now its been like a year since that happened, now im thinking, was he saying it to me because i said that joke? Its like everytime i make a joke, people take it so offensively, it annoys me so much, And when other people do it the other knows that theyre kidding around, i dont understand it, please help