Don't touch me!

So I'm working at my second job. It is a house party for some rich fuck in Peacock Gap just outside of San Rafael. Standard gig. Some guy doesn't want his guests to have to walk the hundred yards to his house so he wastes a bunch of money on valets. Good for me, dumb for him.
I'm working for the first time with this guy who I had just met that night. So, we get all of the guests into the house and all of the cars are parked. This produces the down time of the night when valets generally talk about nothing and everything. This guy starts telling me this story about how he got sold into doing something he didn't want to do. No big deal except he starts showing me how he flet like the person was verbally caressing him and to show he starts rubbing my arm. WTF?
Now, for those who know me, you know that I do not do human contact unless we are very close. No hugs with acquaintances, no rubbing of arms, none of it. Am I wrong feeling this way? Does American society not suggest that not only touching but standing too close is at least uncomfortable if not unnacceptable? I do believe that I take it further than most but that is my comfort level and there is nothing short of therapy in the form of "A Clockwork Orange" that will fix me.
Back to this guy rubbing my arm. I know he was saying something while he was touching me but all I could here was my inside voice saying, "This mother fucker is touching me," over and over in my head. So I take a step back and he steps also and continues. At this point I am about to snap. My first urge is to just swing with my free hand and give this guy an instant nose job. I even go so far as to ball up my fist but I don't swwing. I am at work and I am a proffesional so I unclinch my fist and don't say anything. Am I wrong here? I think if I would have dropped this guy it would have been morally justifiable. Leave a comment if your name is not Tsuji Eriku and you think I'm wrong.