Above all, as we always say, "every attitude has a special say", I am just a curious what the "trivial thing" are. They might be very important in another side of view.
Is that man not very religious?
Is he not very rich?
Does he have an ugly face?
Does he work in a "low position" as some people say?
Is there a social difference between you and that man?
Please clarify, and take in the consideration the order of the prophet Mohammed peace be upon him that we shouldn't refuse a man at all if he has a good religion and good manners, otherwise it will be a great corruption in the earth.
Allah guides you to the right action and I am waiting for your reply.