is there any potential attraction that may grow over time?
Depend on which attraction?
Might be you ever had an experience, a handsome man passed in front of you, you feel awed, you talked to yourself "wow! he's handsome!", and you got a feeling you wanted to be with him. It's called attracted by physical attraction.
Attraction like this cannot be grown by time. Even this will go in the not too long.
If you marry a man who is not attractive in your eyes, you will never have a feeling to your husband like when you saw that handsome man. But emotional bond that built from your closeness with your husband, can make you feel comfortable with your husband. It will makes you always want together with your husband. And in the time when you are, excuse me, on the bed with your husband, you will enjoy it although you are realize, your husband is not physically attractive. It's called attracted by emotional closeness.
This is the attraction that can be grown by time, and will continue to survive as long as husband and wife still have an emotional bond that unites them. Emotional bond could be built if husband and wife respect each other, appreciate each other, and treat each other nicely. And husband and wife could respect each other if they like each other personality.