It probably has less to do with the fact that you're a man, and more to do with the fact that you're just stronger than them. It's more efficient for one person to do a job, than two.
I'm 5'11" and 280+ lbs...its annoying but I understand why everyone asks me to lift their crap
