Good questions and wonderful that you asked, if we don't ponder over questions, we won't know what to answer an interested or curious infidel who asks.
The thread sister insaanah pointed you to answered the first question to the best of everyone's ability, it was also the first house sanctified for the worship of the Creator and sustainer of the heavens and the earth, and God commanded it.
To an infidel you'd have to explain that and whatever other details, then ask them: if not there then where?
Once they grapple with the circular thinking they'll most likely come 360. Newton saw that everything fell towards the center of the earth and sat and pondered until he discovered the law called gravity (it is a law written by Allah that all things would follow such a law). Which way otherwise should they go?
The black stone, the Prophet pbuh used to kiss it at every circuit despite his revulsion to idols and statues of shirk, indicating that it was nothing to do with shirk and it was halal for any Muslim with him to do so and even Umar (ra) had his psychological wonderings with it but made clear in front of everyone, by Allah i know you have neither power to benefit, nor to harm, and if i hadn't seen the messenger of Allah kissing you i would not have done so.
There are stories that Allah has placed it as a purifier of sins - not within it's own power just as He has placed the ability to wash with water and irrigate fields for crops, though we don't worship water or believe it has powers of it's own. Nor was the ability placed in jesus pbuh to heal the blind and lepers, raise the dead, breath life into a clay bird powers of his own, it was placed in him by Allah.
The rites, they were prescribed to Ibrahim pbuh and are a good test and training of submission to Allah, sometimes (especially in the military) bosses give tasks to employees which make no sense to the employees and they carry them out like clockwork, it also develops a chain of command structure although those are just observations of positive benefits and aren't necessarily the only reasons. The "i am here at your service" call (labbayk) is an indicator to some of the connections i made above though.
Stand at ease! attention!
what does that have to do with combat wonders the green recruit lol. Uniformity, orderliness, unwavering obedience, respect and loyalty are some aspects.
We practice certain rituals every day whether we like it or not, and we do it so much that we forget to question as we did when we were kids, why do i sleep and wake up? Why do i have to eat? Why do i have to make funny movements and brush my teeth? Some parents will explain in depth, some will give the basics some will say just due it because i told you to, but ultimately you have to and there is a good reason. Explaining to a kid why the chicken jumped on another chicken is dificult so we just drag them away or chase away the chickens, but we know there's a reason.
Regarding brother saleem khan's comment on the heavens, it may not be literally but it is in some dimension or other, the slave girl pointed towards the heavens and was freed after the questioning of the prophet pbuh. Allah also says "He who is in the heavens" in the Quran.
it is deeper and more difficult to grasp the aspect of it though since it is reported that His presence comes to the lower heavens in the last part of the night to hear the askers, but since it's always the last part of the night somewhere or other on this orbiting and spinning earth, it can't be grasped in that literal sense, also since the His throne alone encompasses the heavens and the earth, they can't fit Him literally obviously unless He makes His presence and benevolence amplify at certain times and places or whatever it might be Allah - Glorified be He - knows best.
Even the isdue of Qadr is hairsplitting despite us all knowing it exists.
Hope i shed some light on some issues to the best of my ability.
