I'm not going to let my dog's previous owner take him for a walk out of fear he might try to steal him back
Is there a history that the previous owner likes to steal dogs?
No, but I just don't trust him to not steal him back.
I understand, it's very kind of you to look out for the welfare of your dog. You need to be vigilant because you never know what people will try and do.
I agree, I'm going to make sure he stays safe.
Just tell him no or walk with the dog with the previous owner. Get to know that person better so you can have a dog sitter in the future.