The majority of the people in Makkah before the birth of the Messenger were pagans, those who worshipped idols. Jews and Chrisitans did exist, but they were the minority. The Jews were, if I am correct in Madinah (known at that time as Yathrib) and Khaibar. The Chrisitans were in the north, the Byzantine Empire. And there were the magians, the Persian empire. However the area of arabia which we are concerned with were mostly pagans and idol worshippers. The time before the Messenger is known as the time of Jahiliyyah, i.e. Ignorance.
Before the Messenger, at the time of Isma'il a.s. the people were upon Tawheed and that continued until some generations before the Messenger where a person called ‘Amr b. Luhay brought an idol to the people and from that idol worhsip and paganism spread amongst the arabs. He was the reason that shirk spread amongst them and people started making new idols after pious people such as Al-Lat etc.
Now you asked how did they (the arabs of Jahilliyah) know about Allaah. You see, they did not believe that their idols could create, destroy, benefit or cause harm. They know only Allaah could do that and they knew that Allaah was the Rabb. However, their shirk came in when they took their idols as intercessors with Allaah and approached Allaah by the means of these idols. For example, Al-Lat was a pious man who used to give a type of food to the pilgrims of Makkah, and when he died the arabs of Jahilliyah turned him into an idol and started seeking intercession from it. The Qur'an testifies to the fact that they knew that Allaah was the creator, yet they commited shirk in their worship of Allaah:
10: 18 And they worship besides Allah things that hurt them not, nor profit them, and they say: "These are our intercessors with Allah." Say: "Do you inform Allah of that which He knows not in the heavens and on the earth?" Glorified and Exalted be He above all that which they associate as partners with Him!
39: 3 Surely, the religion is for Allah only. And those who take Auliya' (protectors and helpers) besides Him (say): "We worship them only that they may bring us near to Allah." Verily, Allah will judge between them concerning that wherein they differ. Truly, Allah guides not him who is a liar, and a disbeliever.
39:38 And verily, if you ask them: "Who created the heavens and the earth?" Surely, they will say: "Allah." Say: "Tell me then, the things that you invoke besides Allah, if Allah intended some harm for me, could they remove His harm, or if He (Allah) intended some mercy for me, could they withhold His Mercy?" Say : "Sufficient for me is Allah; in Him those who trust (i.e. believers) must put their trust."