I'm not sure if this is a bug exactly, but I ran into a NullReferenceException here in the ApiVersionRouteConstraint due to the middleware not yet having been injected.
In my particular case, the issue turned out to be that UseApiVersioning() was registered after UseRouting(), rather than before it. This resulted in the IApiVersioningFeature to not yet have been registered in the middleware chain.
I think perhaps just adding a null check here (and probably all other calls for httpContext.Features.Get<IApiVersioningFeature>()) and throwing a more informative exception would help.