We need to realize that there are two brands of Sufism. There is a sufism that many of our early scholars followed, such as Shaykh al-Islam Abu Ismail al-Harawi. He in fact wrote a manual on Tasawwuf called Manazil al-Sa’irin which Ibn Qayyim al-Jawziyyah later explained in his own Madarij as-Salikeen. Ibn Rajab said about him: ‘The Jurist, faqih, the Exegete, mufassir, the Hadith Master, hafidh, the Sufi and exhorter, Shaykh al-Islam Abu Isma`il…’, Ibn Hajr referred to him with the title of Shaykh al-Islam and Ibn Abi Ya`la said about him, ‘He was the Imam of Ahlu’l-Sunnah in Herat.’ al-Dhahabi said, ‘Shaykh al-Islam was a pure and committed Athari who would rebuke the Mutakallimin,’ and, ‘He was a drawn sword against the Mutakallimin.’ al-Harawi used to say:
I am Hanbali so long as I live, and when I die
My legacy to the people is to become Hanbalis
So what we understand from this is that there were amongst our scholars many who were 'sufis' but they weren't upon any innovation or heretical beliefs. Rather, for them 'Tasawwuf' was what it was to be originally, i.e. a way to Tazkiyyah and suluk. Ibn Taymiyyah etc weren't against
this brand of Tasawwuf, rather what they were against was the newly invented philosophical Tasawwuf with it's pure heretical concepts such as Wahdatal Wujud (pantheism), ittihad, etc which incorporated into it the beliefs of the sects that had adopted Kalaam etc. It is this brand of Tasawwuf that our scholars were against.
Personally, I'd avoid using the word 'Tasawwuf' simply because it's such a vague term, it means different things to different people. So I think, and Allaah knows best, when we talk about 'Tasawwuf' we need to be just and realize what exactly it is we're against - i.e. the innovations, the heresies and corrupted beliefs. And we also need to realize that just because some people call themselves 'sufis', especially those amongst the lay people, they are not innovators, they may truly believe what they're doing is correct and not know any better. Some people just happen to use the term to indicate that they are trying to attain tazkiyyah etc. We need to be just and not see these things in black and white, there are shades of gray.