Treat rejectedIdentifier as a permanent failure

[jsha]

We have one cert-manager client in our logs doing 10,000 requests every 15 minutes, trying to issue for this invalid domain name:

400 :: rejectedIdentifier :: Error creating new order :: Cannot issue for \"go-api-boilerplate.local\": Name does not end in a public suffix

cert-manager should internally store rejectedIdentifier errors and treat them as permanent errors, so it never tries to issue for that same identifier again.

[munnerz]

I've opened #2198 to handle this - I think we're safe to treat all 4xx errors as something we should halt operating on (as it usually indicates bad configuration/input).

/kind bug
/area acme

[munnerz]

To clarify here, the issue describe in #2196 I do not think is caused specifically by us not persisting this error somewhere - in the normal case, upon receiving this error we'd apply a regular 'back off' which caps out at retrying once every 30 minutes.

I think in the #2196 case, it is coincidental that they are receiving this error as well as causing a large number of requests.

[jsha]

Yep, agreed that #2196 is a separate issue. I saw a different client that was sending lots of requests for the invalid name go-api-boilerplate.local (which I think comes from some template code somewhere).