Another clarification: a lot of religious people think science says time is "infinite."
This is not the case. Time, like space, is not infinite. The universe has a limited area, and it has a limited age. In fact, time and space are part of the same sort of fabric, called "spacetime," which is what Einstein proved.
So the question is not whether spacetime is infinite. The question is whether spacetime has any sort of "edge" or "boundary." When people say the Big Bang is the "beginning" of time, they often have in mind a sort of drop-off or edge, where at one moment there was no universe, and then the next moment there was.
But this doesn't make any sense. Because "moments" are all part of spacetime, and thus part of the universe.
A better way to think about the Big Bang, the "beginning," is like the North Pole on Earth. The North Pole is the "northernmost" point on earth, just like the Big Bang is the "earliest" point in spacetime. But the North Pole isn't an edge or a boundary. The surface of the earth doesn't drop off at the North Pole—it remains smooth. You can't go "north" of the North Pole, just like you can't go "before" the Big Bang. But this is because the Big Bang is simply a point on the structure of the universe, like the North Pole is a point on Earth's surface.
The upshot of all this is as follows: If there is no time outside of the Universe, then the Universe has by definition always existed. Just like you believe your God has always existed. It may be finite—it may have an "earliest point" or "beginning"—but its finite time is all the time there is.