There are multiple reasons that one might see a new English translation of the Bible being produced:
(I'm assuming that the reasons for new translations in languages in which the Bible is not already available is self-explanatory.)
1) Language continues to evolve. If one could ever produce a perfect English translation, it would quickly become imperfect with the passage of time.
2) There really is no such thing as a perfect translation, not just of the Bible, but of even the most simple sentence used in everyday communication between 2 people who speak different languages. Some people desire a literal equivalents in translations, but few terms actually have literal equivalents in other languages.
For instance, most English speakers are familiar with the Spanish phrase "Mi casa es su casa." But should it be translated as "My house is your house." or "My home is your home."? There is no right or wrong answer to that question without knowing the mind of the original speaker. And let us say that one was an English speaker wanting to welcome a native Spanish speaker to one's home by saying, "My house is your home." You could use "casa" for both nouns, but what is the appropriate personal pronoun "tu" or "su"? Both spanish pronouns mean "your" in English, but "Mi casa es su casa." and "Mi casa es tu casa." are going to convey different degrees of welcoming to your guest.
3) Many suggest that one should give up on the goal of a literal word-for-word translation and that the goal should be that of dynamic equivalence (i.e., translating thoughts and phrases, not individual words). But the presence of figurative language makes this goal just as elusive. For instance how would you translate "Go fly a kite!" into a language that doesn't have a similar type of phrase? Plays on words and puns are actually quite common in the Bible, but they don't translate well into English.
4) Once one determines where one is going to come down on literal versus dynamic equivalence, one still has to select the exact text one is going to translate from. The Bible was never written as a single whole document. Multiple copies of the many various books that together comprise the Bible were made over many years. Given that we don't have the original of any Biblical book against which to compare the existing copies, any difference between the two (or three or more) texts becomes a case of "He said, She said" between the various texts. Just as parents and judges are called on to make such determinations, so the field of textual criticism helps us in this endeavor. And that field is always advancing its science so that we believe we are better able to make those calls today than in the past, leading some people to desire to create a new translation.
5) Textual criticism is just one of the sciences employed in Biblical studies. Literary criticism and better knowledge of ancient languages has also made their marks in helping scholars to better interpret the text we have today. And again, this knowledge leads some people to desire to create a new translation.
6) But by far I think the biggest reasons that one sees an increasing number of new translations today is the recognition that there is no single monolithic English-speaking culture for which one is translating the Bible.
One understands that because of the language skill level of children, that perhaps chldren's Bibles need to be more than just adult Bibles with pictures, they actually need the syntax of the sentences written with a child in mind. But children are not the only unique group in the English speaking world. Clarence Jordan realized that the workers on the cotton and peanut farms of southern Georgia really spoke a different sort of English than that in which the Bible they had access to was written, and decided to create something called the Cotton Patch Bible that re-cast the Bible (or at least the New Testament) into their langugae and culture. That might be at the extreme end of the spectrum, but many new translations are indeed targeting micro-cultures and subsets within the English-speaking world. Perhaps the most successful of those was Kenneth Taylor's "Living Bible", a paraphrase he originally produced for his 5th grade son. Seems that many adults with high school education and better really do read at the 5th-6th grade level, for I've never held a Bible study in which someone didn't bring that with them as their Bible of choice. The "Good News" Bible and "The Message" are more modern attempts at basically the same thing -- and they have the same inherent strengths (easy to read) and weaknesses (the translator makes determination of meaning of complex passages for the reader).
7) Not all new productions are actually new translations. Many "new" Bibles that are being produced today aren't new at all. They are just existing standard translations in a new package. That new packaging might include fancy maps, imbedded concordances, a Bible dictionary, and most popluar nowadays, a running commentary from some particular theological persuasion or a famous writer/preacher. I'm a little cynical of these, wondering how many of them are being published out of audience demand and how many out of observation that there is money to be made? But I admit to having my NT with explanatory notes by John Wesley. And my Lutheran pastor friend has something similar with notes from Martin Luther; so I suppose I shouldn't be so critical of today's preachers and their audiences.
I suppose that there are other reasons that I haven't thought of, but these seem to me to be the most commonly cited reasons.
As for which is best?
Well, readers, just as do translators, have to make decisions with regard to literal versus dynamic equivalence. And then the rest is more about find that which one is most comfortable with. And I think in many ways that becomes a matter of taste. For some, that means the New American Bible Revised edition is going to be just the Bible they have always waited for. While I've grown so accustomed to the NIV, that even though I think the NRSV is probably a better translation, I just try to be aware of the NIV's weaknesses and continue to use it for 90% of my Bible reading.
