I don't think that's true. First and foremost, we avoid music as an act of obedience to what Allaah (swt) and His Final Messenger (peace be upon him) have commanded. This is the bottom line, and it's very simple. Beyond this, there are many wisdoms that have been suggested as reasons for why musical instruments are forbidden. If someone doesn't agree or understand them, it doesn't detract from the fact that wisdom exists behind the prohibition, and thus one is simply to trust the wisdom of Allaah (swt) and obey Him in whatever He has decreed.