Think about it, being responsible for the pain and suffering of 100 billion+ humans that have ever existed, is no small responsibility. And to eat the one fruit that was forbidden out of the millions that were offered to you, i mean come one really....
Adam and Eve were not responsible for any pain that the rest of humanity suffers. Just what causes you to conclude this? If it's the fact that they were thrown out of Paradise, then note that there is no agreement on what that "Jannat" means. It could mean the Jannat of the afterlife which they had, or a Jannat on earth they were thrown out of. It could be anything. It may not be either of those two things I mentioned, it could be something else entirely. Point is, no one is agreed.
And bear in mind, no one is responsible for the burden of sin someone else incurs. Adam and Even went through pain and suffering just as we do. The Holy Qur'an says: "
Verily, We have created man in toil." (90:4, Mohsin Khan translation) I do not see how Adam and Eve were both exempt from this when they clearly had to ask Allah for forgiveness when they were deceived. Allah forgave them for what they did, by the way. In fact, Allah taught Adam (as) a du'a for asking Him forgiveness. And Allah did eventually forgive him. This is all in the Holy Qur'an.
So, if Allah forgave him for his trespass, then why shouldn't you? Allah knows more than you do about everything. If Adam (as) was ever responsible for the suffering of 100 billion+ people, then Allah would probably not have forgiven him and he would keep incurring sin for all the suffering each human being has. But this is not the case.
Also, do not get confused with Christian doctrine. I think you might be thinking of the original sin which Christians believe in. This is completely contrary to the teachings of Islam and it has no basis. Original sin is a false doctrine and there is nothing correct about it. Each human being born does not incur the sin of Adam and Eve, they were both forgiven in their lifetimes, and furthermore, every baby is born without sin - even a baby born from a prostitute. A baby born from a prostitute that dies still has funeral prayers said over it, because it is sinless the day it comes into the world until the baby grows into a young person who can distinguish right from wrong.
So, if we are already all made to toil and struggle, born without sin and Adam and Eve were forgiven by Allah - then I think all of these are arguments that clearly exonerate them of any misdeed of causing human suffering thousands of years later. How can someone be guilty of what others go through thousands of years later anyways? There is nothing they directly did which causes us pain and suffering now. Again, being thrown out of Jannat does not constitute any incurring of pain and suffering which would not have been there before.
If you are worried about what they did, and you blame them, then consider that this was all part of the Al-Qadar of Allah. In fact, to truly answer your question, which Musa (as) also once presented to Adam (as), the Holy Prophet (saw) reported what was exchanged between the two of them:
From Jami' At-Tirmidhi, Book of Al-Qadar, Hadith #2134:
Abu Hurairah narrated that the Prophet(S.A.W) said:
"Adam and Musa argued, Musa said: "O Adam! You are the one that Allah created with His Hand, and blew into you of His Spirit, and you misled the people and caused them to be expelled from Paradise.' So Adam said: 'You are Musa, the one Allah selected with His Speech! Are you blaming me for something I did which Allah had decreed for me, before creating the heavens and the earth?'" He said: " So Adam confuted Musa." (Sahih)
So I think that should satisfy you. This Hadith teaches a good lesson on not blaming others for anything. For us Muslims, things don't just happen, they are Decreed by Allah. Everything is part of His Decree. Therefore, don't blame someone for what they did since His Decree determines all. And Allah can't be blamed for anything. Thus, what is there to get riled about? It was all meant to be.
