Giving up is not a viable solution here, speaking of giving up we have a certain old tale which speaks of a king (can not remember his name atm) that kept loosing battles, on the eighth loss he sat down and was on the verge of giving up. He noticed a scorpion nearby that kept trying to climb to ceiling, but it kept falling. The king had some pity for it, and told one of his attendants to help the scorpion reach the top and whenever the attendant would try to get near it would try to sting him. So the night they watched the scorpion which kept on falling down, but on the ninth try (or something) it managed to successfully climb.
The king taught to himself, if the scorpion which kept falling down kept on trying and in the end accomplishing its task, why can not I? and the next day he won the battle.
As a muslim, we have to keep trying and not giving up on any tests. At times we will fail keep failing but on the other occasions we will be successful. We as a mere slave of rub 'Azza wa jal alone can not alone succeed in the tests. It is actually the mercy and rahma of Allah rabul 'izza that He allows us to succeed hence we have many duas which continuously were read by the Prophet (sallahu allay wa salam) to show humbleness and know that we alone can not do as we want, it is only the mercy and rahma of Allah rabul 'izza who allows us to succeed.
We to have know well that as you, I or anyone out there travel along life we come along at points that we see something good happening and as a muslim we should do Shukr. And when something bad happens our role is to have Sabr. These two are recipes of life that puts the Imaan of a believer on the state of peace. And what is interesting, for a believer (mu'min) whatever the state falls upon him/her, it is always for the person's benefit and goodness.
As our beloved Prophet (sallahu allay wa salam) has mentioned in a reported hadith:
'Strange is the affair of a mu'min (the believer), verily all his affairs are good for him. If something pleasing befalls him he thanks (Allah) and it becomes better for him. And if something pleasing befalls him he is patient (sabir) and it becomes better for him. And this is only for the mu'min.
We do feel that Zulm (oppression) is being done upon us, this is because we do not know the Islamic History. If we really knew what hardships the Companions (radh), the Prophets ('allay salam), the Salaf Saliheen such as Imam Bukhari (rah), Imam Ahmed bin Hanbal (rah) etc then we would know what troubles we are facing is minor compared to theirs. But SubhanAllah they still remained steadfast upon the deen, never left the Imaan.
If I move just a bit more deeper, generally we translate Sabr has patience in English but this is the third level of Sabr. According to Islamic translation the first two stages of Sabr are linked to steadfastness. Where the highest stage of Sabr is steadfastness in the Ibadah of Rub 'Azza wa Jal and the second level is steadfastness in staying away from Sins while the last stage is having patience in any difficulties.
Getting back to the point.
Speaking of neighbours, the Shaytan never wants the person of Imaan to climb the ladders of reaching closer to Allah rabul 'izza hence one way he does succeeds in his task is by trying to hoard evil thoughts in the heart for others. Or in other words having malice and hatred.
Certainly we are thought to have the rights of the neighbours. And to do well to others even if others are doing badly or wrongly towards you. Do not live with the motto that one does good to only those who do good to them, but rather ensure that one fulfils the rights. And when one executes justice upon everything around us, then we find harmony within the life.
Truly I can understand how disturbing it is to have such neighbours. I remember one of our neighbours were couples who were late night partying type, with loud music on while alcohol being served. But we, as a family, did what was right which was to be good to them. What we did was we sent some dishes that were cooked in our home, and any other gifts that could be sent along. It so did happened, that they started feeling guilty and bad for what they are doing and they knew that we are muslims living right next-door. Until they literally shifted their home somewhere else. There was another family on the other side of neighbour-hood, similar case and they shifted.
The next neighbour that came across were Arabs, they had white huskies which would bark loud. We did the same, subhanAllah they gave away the dogs and in Ramadan they kept on sending their traditional iftaar foods. Even though we barely met, and they also left later onwards. The current neighbour now are from Shia ideology, I guess they are impressed enough that whenever we pass by the dad of the house would wave and he would be seen sitting right nearby the first saff (row) for Juma prayers

P) lol.
On the other side we have neighbours that are Hindus, pretty staunch. At first they were aggressive in their approach to see us muslims living, they would tilt around with the rubbish bin outside. But we did what was right, which was to give things to their house. Even now when my mum passes by them the lady of the house waves and she even visited to ask if we can give any vegetables that grow in our backyard.
Yes, pretty much had different neighbours across.
It is about doing our own part. The power of Sunnah of even smiling to a brother has a large impact on the person. Give it a test, if you know a muslim that approaches harshly to you or is aggressive. Just walk down and say 'Assalamualaikum' with hand out, he will reply and as a matter of fact a time will come when he will be the first giving salaams to you.
When we have neighbours around, we should show our good conducts and manners. I understand it is not easy, and it does takes guts. Knowing that a neighbour maybe hostile or aggressive but you got to do your part. Try sending gifts or some food across, in the end of the day, they also have a heart and they will feel guiltiness in their own part.
It is a shortcoming for us to see the next generation having really bad manners, and I say that as a muslim who is looking around the next generation of muslims growing up. There is no sense of shame or haya on how they treat the elders let other any other person out there. When such as our Akhlaq, then we really can not expect a first good impression on us muslims. But if we truly stick to what Islam teaches us, the Shari'a teaches us that we will realise that people will start respecting and loving us.
I still feel the need to write further, but I really do not want it to make a big whole essay. And truthfully, I have not managed to re-read what I have written above so do excuse my grammatical mistakes. But I will conclude with one incident of Sultan Salahudeen Ayyubi (rah), and this incident I still have not managed to get it across in book or written form but I have heard from one of our teachers. (so not sure about authenticity unless someone knows here)
When Sultan Salahudeen Ayyubi (rah) was fighting the crusades, it so happened that the leader/commander of the crusades fell ill. What Salahudeen Ayyubi (rah) did was he sent his personal physician (who was a jew) to attend for the aid of the commander.
The relations of Christians and Jews were not good at that time, while the leader was fighting Sultan's army. But as a muslim person Sultan Salahudeen Ayyubi (rah) did what was deemed good and right. Even the opponents felt it when Sulahudeen Ayyubi (rah) passed away.
So yes, as a muslim we should do good even if anyone does bad towards us. Be it neighbour, or any family relatives. Its tough but it never goes in vain, because we will see the importance of it when once we go to grave.