If your custom code calls httpError(404) in the "index" action or an earlier stage like the init() method and this URL is an existing SiteTree URL or it existed before (versioned table), it results in an endless redirect loop.
Why would someone call httpError(404) for an existing and published URL?
Sometimes we have pages were more conditions (beside "is published") have to be fulfilled. For example we've implemented a time based publishing and versioning of content where the page is published, but actually we check some other dates in the init() method and if the conditions are not met, we throw a 404 error.
Currently this results in an endless redirect loop by the OldPageRedirector because SilverStripe/OldPageRedirector "thinks" the page is existing and therefore redirects to the URL again.
I think it should be differentiated if the core is throwing a 404 or the developer who is using the framework. In the first case the OldPageRedirector should handle the 404 and this is good, because it is a nice feature!
In the second case I think the developer has a good reason to throw a 404 and probably does not want to get a 301 to another or sometimes the same URL.
For now we are using a custom httpError method with no fallback.
Maybe this has to be split into a feature and a bug ticket?
As long as the problem is not resolved - this is the workaround I found:
throw new SS_HTTPResponse_Exception(ErrorPage::response_for(404), 404);