The 'asl is that you cannot look at women. The evidence is the following Verse of the Qur'aan
Tell the believing men to reduce [some] of their vision and guard their private parts. That is purer for them. Indeed, Allah is Acquainted with what they do.
[Qur'aan 24:30]
Jariir bin 'Abdullah reported:
I asked Allah's Messenger (sallallaahu alayhi wa sallam) about the sudden glance (that is cast) on the face (of a non-Mahram). He commanded me that I should turn away my eyes.
[Sahiih Muslim Hadiith No. 2159]
Narrated Buraidah ibn al-Hasib:
The Prophet (sallallaahu alayhi wa sallam) said: to Ali: Do not give a second look, Ali, (because) while you are not to blame for the first, you have no right to the second.
[Sunan Abi Dawud Hadiith No. 2149]
The scholars made exceptions to this 'asl.
They said there are certain circumstances where you can look at the woman. You can look at her when she is giving testimony or is being testified against. You can look at her when you have to deal with her in sale and trade or matters pertaining to rent. Also, they said, a doctor can look when necessity dictates. A man who wants to marry a woman can look at her as well.
They stipulated that none of this should be with desire and lust. It should not invite to al-munkar. If a doctor knows he will lust and desire if he looks then he cannot. The same applies to the other cases.
Ahmad has been reported to have said: If a person deals with a woman in sale and trade or matters of rent he has the right to look at her face so that he knows who she specifically is.
As for your case, it is best you not look at her because you have no real need. Feeling unconformable and impolite is not a need. Do not worry about what the creation thinks. If she thinks it was impolite, so be it. You are only to worry about Allaah.
And, to be honest, I don't think a woman who wears a niqaab will think it is impolite. If you are so worried just tell her I do not look for Islamic reasons so don't mind. It is that simple.
Reference(s):
Ruling regarding looking at women
Ruling of looking at the eyes of a woman wearing niqaab