No offense, but I get annoyed when cab drivers don't speak English well.
Why would that bother you?
It's really annoying to hear them talk that way, I think they are too lazy to try harder and learn.
If you mean you can't understand them, that's fine. But if they aren't speaking perfectly but you still can understand them, you shouldn't be upset at them. After all they are still learning the language.
They should speak English better if they are gonna live here.
It helps to be a little more patient, as long as you can still understand the gist of what they are saying, then it should be all okay. They are trying their hardest.