Just because we can't understand it doesn't make it untrue.
It is perfectly possible to understand the various magnitudes of infinite. The problem with infinite time is that it would be contradictory. It would be inconsistent with the rules governing infinite.
Consider the old joke "Can god make a rock so massive even he can't lift it", which digs at the concept of infinite power.
The universe is not infinite. Hence, a rock inside the universe would not be of infinite size. Furthermore, within the framework of Newtonian thinking, a rock and the earth mutually attract each other. if the rock were substantially larger than the earth, it would be seen attracting the earth and not the other way around. Next, the Eddington limits impose a maximum size on celestial bodies. Beyond that size, we only have black holes. So, it would amount to lifting a black hole. And so on. The question just falls apart when reaching sizes that are beyond the tolerances involved. The earth cannot attract objects beyond a particular size. It would be attracted by it instead. For example, you cannot prevent the earth from being attracted by the sun just by lifting objects from the surface of the earth. Hence "countering this attraction" aka "lifting the object" becomes meaningless.
Indeed arguing for first cause or for a God only becomes more troublesome when you attack the concept of infinity. If infinity can't exist, then can God? He couldn't be infinitely powerful or infinitely knowledgable or anything else infinite.
The universe as it expanded out of a singularity, is finite. Hence, nothing contained inside the universe can be infinite. However, its first cause must necessarily pre-exist it. Therefore, it cannot be a part of the universe or be contained within it. Since "understanding" itself requires causality, which in turn rests on the existence of time, which did not exist before the universe and its timespace existed, we cannot truly understand anything that would have pre-existed the universe and its time. Furthermore, what infinite dimensions would this be about? Without timespace, dimensions have no meaning. You can safely assume that anything prior to the existence of time is fundamentally incomprehensible to us. All that we could truly understand from the first cause is its initial effect on the universe, because this effect is part of the universe and its initial state.
You also have to look at the alternative. If there hasn't always been something (ie, the universe or pre-universe, God, etc), then something had to come from nothing.... and I'm sure you've seen a plethora of theists insisting that something can't come from nothing.
"Nothing" routinely falls apart into a particle and its anti-particle. -1 + 1 = 0. With zero representing nothing, you can see that there is absolutely no problem with something coming from nothing, as long as it also symmetrically creates the required anti-something. The idea is that something cannot come from nothing is utterly simplistic. Theories in physics are replete with virtual particles. That is really 19th century material.
Special pleading that God can have always existed or can have come from nothing doesn't really get you anywhere, as much as you may wish it did.
The first cause of the universe can impossibly be part of what it has caused. That is certainly not "special pleading". Distinguishing between the entity causing and the entity caused is necessary. Therefore, the idea is that nothing inside the universe could possibly have caused its existence, cannot be understood as a form of "special pleading".
What's more, "special pleading" is not a sound counterargument in math. Recursion always requires at least one initial, special case. For example, the Fibonacci sequence is defined as following: for n larger than two, F

=F(n-1)+F(n-2), while F(1)=1 and F(2)=1. There are two "special pleadings" in this definition, because anybody with even the slightest intuition in math can see that this is obviously a requirement. The entire "special pleading" terminology does not come from math of physics, but clearly from some other, but then rather ineffective and inferior way of thinking. We do not use that term. It is simplistically dumb. You cannot produce counterarguments against math of physics from that kind of non-methodological way of thinking. Nobody would accept that.
Not even the most atheist of scientists believes that time would be infinite. Nobody would ever propose that as a counter-argument. Your considerations are not of the same level as Aristotle's work. In fact, they rather reflect a lack of knowledge of the existing body of math and physics around. A universe expanding out of singularity could impossibly be infinite or contain infinitely large objects. Aristotle's work has been around for over 2500 years now, and if you had used existing solid counter-arguments, which certainly exist, it would be a draw, but you didn't. Why did you try to invent your own on the fly? You fail to realize that arguments invented on the fly will lack the quality necessary to attack Aristotle's work.