I'm going to date my brothers ex wife
Do you have feelings for each other?
I'm sure she loves me, I like her a lot as well.
If your brother does not mind at all, there is no problem with it. It is good to communicate your intentions to him so there is no confusion.
I don't want to ask him in case he says no.
The right thing to do would be to ask him what he thinks. Doing it behind his bac is not the way to go about it, even if he would be cool with it. He would feel like he cannot trust you if you went behind him and did it.