"God is described in every faith as something beyond human comprehension, an all-know all-powerful spiritual force intertwined with everything physical and spiritual, yet is nothing like His creation. It seems rather ungodly for Him to demand worship and submission as if He is human king ruling an absolute monarchy....."
I would like to take a stab at answering the original question----this is my opinion only......(The part about Judaism comes from a conversation with a Jewish person on the concept of soul)
If we accept that God is Just, Compassionate, and Merciful. And, we also accept that God has no needs/wants---Then any assumption that God "demands" that we worship him out of any "egoic' desires/wants such as arrogance, pride, anger, jealousy...etc have to be ruled out.
The Quran says we have been created for worship of God, and that this worship strenghtens our souls. Soul in Quran is Nafs, and in Judaism it is Nefesh. In Judaism, Nefesh is further divided into 2 parts/levels, Yezter Ha Ra,(egoic/animal soul) and Yezter Ha Tov, (transegoic soul) which would correspond somewhat to Nafs Ammara (egoic/animal soul) and Nafs Lawwama (transegoic soul).
(Nafs Mutmainna (Peaceful soul) would correspond to Neshama (the highest level of soul)in Judaism)
God created the soul to inhabit the body. As Human beings, (body and soul) we need to have shelter, to eat, to have companionship, and to protect the family and territory in order to survive on earth. Therefore, our "animal desires" are a neccessary part of our survival mechanism. However, we have also been created inherently good. When God animated us, he blew in his breath/spirit (Ruh= Quran, Ruach=Judaism) We have spiritual natures. When we build up God awareness (Taqwa) through worship of God, we strengthen our spiritual natures/soul---which leads us to strengthen in goodness. Weakness of our transegoic soul leads us to strenghten our egoic soul--which leads to all kinds of problems such as pride, anger, hate, arrogance, greed...etc.
Thus, a Compassionate, Merciful and Just God, guides us to the right path so that we may enter Paradise. To worship God is for the benefit of Human beings. Therefore, "submission" as demaded in Islam is a path that leads to "Nafs mutmainna"--or peaceful soul because we achieve harmony/balance with the noble nature we were created with.
So why give us free-will---why not just create us to obey God's will? ----Because God is compassionate and Merciful----It is our scope of free-will that enables us to connect with the Divine, to experience spirituality. without it, we would never have the oportunity---we would simply be like any other animal.
God has given us conditional free-will. We do not have a choice as to where we are born, when we die or some of the circumstances of our life---but we do have some choices. With free-will, coupled with God-given intelligence, we can choose to make choices that would benefit all of God's creations in the best possible way. As his representatives on earth, it is our responsibility to do so.---and we can best accomplish our duty through Islam (submission to God)