Should be pretty trivial and could help a bit to filter for pending fixes prior the merge window. It might be more complicated to automatically remove the label and I guess this won't happen often, so I would go with the simplest case for now.