Skip to content
This repository has been archived by the owner on Dec 13, 2018. It is now read-only.

Docker Swarm + nginx + WS-Federation: multiple redirection issue #1923

Closed
rynnova opened this issue Nov 29, 2018 · 33 comments
Closed

Docker Swarm + nginx + WS-Federation: multiple redirection issue #1923

rynnova opened this issue Nov 29, 2018 · 33 comments

Comments

@rynnova
Copy link

rynnova commented Nov 29, 2018

I have a repo as an example for this, but essentially: under Docker Compose/Swarm with Linux Containers and using nginx, I've had trouble getting the ASP .NET Core WS-Federation library to function.

Remove nginx, serve the application from https Kestrel only, and it works fine. Use nginx as the https proxy with Kestrel using http and it redirects 6 times from the auth server before failing entirely.

What could cause this? What else would help identify the cause of this issue?

@terafil

This comment has been minimized.

@rynnova

This comment has been minimized.

@Tratcher
Copy link
Member

Friend of yours?

It's likely a config issue with the x-forwarded-proto or related headers. Start by sharing the troubleshooting output from here and a Fiddler trace file.

@rynnova
Copy link
Author

rynnova commented Nov 29, 2018

Yeah, a co-worker.

I'll check on the headers, but I couldn't output any troubleshooting information before using that same code block. I'll see if that was because of a logging setup.

@Tratcher
Copy link
Member

You can always write it to the response body instead.

@rynnova
Copy link
Author

rynnova commented Nov 29, 2018

I made the changes in that article, and still only see the below in the Compose logs.

The responses from localhost only send back 302s to the ADFS server, so it never has a response body in Chrome. I have updated the repo to include the forwarded headers implementation as well as writing to responses.

app_1_9d5f435a2428 | info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[0]
app_1_9d5f435a2428 |       User profile is available. Using '/root/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
app_1_9d5f435a2428 |       Creating key {a73fb557-4609-476f-9a83-bc4d57f717a7} with creation date 2018-11-29 18:14:49Z, activation date 2018-11-29 18:14:49Z, and expiration date 2019-02-27 18:14:49Z.
app_1_9d5f435a2428 | warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
app_1_9d5f435a2428 |       No XML encryptor configured. Key {a73fb557-4609-476f-9a83-bc4d57f717a7} may be persisted to storage in unencrypted form.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
app_1_9d5f435a2428 |       Writing data to file '/root/.aspnet/DataProtection-Keys/key-a73fb557-4609-476f-9a83-bc4d57f717a7.xml'.
web_1_e8e63ccbdc0e | 2018/11/29 18:14:49 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/run/secrets/wsfed.crt"
web_1_e8e63ccbdc0e | nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/run/secrets/wsfed.crt"
app_1_9d5f435a2428 | Hosting environment: Production
app_1_9d5f435a2428 | Content root path: /app
app_1_9d5f435a2428 | Now listening on: http://[::]:80
app_1_9d5f435a2428 | Application started. Press Ctrl+C to shut down.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 GET http://localhost/
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[7]
app_1_9d5f435a2428 |       Cookies was not authenticated. Failure message: Unprotect ticket failed
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 839.2439ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 1290.2006ms 302
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:55 +0000] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 POST http://localhost/ application/x-www-form-urlencoded 9536
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:56 +0000] "POST / HTTP/1.1" 302 0 "https://auth.org.customer.com/adfs/ls/?wtrealm=https%3A%2F%2Flocalhost%3A44396%2F&wa=wsignin1.0&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed&wctx=FIRSTONE" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 11.8432ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 22.138ms 302
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Server.Kestrel[32]
app_1_9d5f435a2428 |       Connection id "0HLIM4Q7UFGE3", Request id "0HLIM4Q7UFGE3:00000001": the application completed without reading the entire request body.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 POST http://localhost/ application/x-www-form-urlencoded 9544
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:56 +0000] "POST / HTTP/1.1" 302 0 "https://auth.org.customer.com/adfs/ls/?wtrealm=https%3A%2F%2Flocalhost%3A44396%2F&wa=wsignin1.0&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed&wctx=SECONDONE" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 0.2781ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 0.6033ms 302
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Server.Kestrel[32]
app_1_9d5f435a2428 |       Connection id "0HLIM4Q7UFGE4", Request id "0HLIM4Q7UFGE4:00000001": the application completed without reading the entire request body.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 POST http://localhost/ application/x-www-form-urlencoded 9538
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 5.7113ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 8.5493ms 302
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Server.Kestrel[32]
app_1_9d5f435a2428 |       Connection id "0HLIM4Q7UFGE5", Request id "0HLIM4Q7UFGE5:00000001": the application completed without reading the entire request body.
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:57 +0000] "POST / HTTP/1.1" 302 0 "https://auth.org.customer.com/adfs/ls/?wtrealm=https%3A%2F%2Flocalhost%3A44396%2F&wa=wsignin1.0&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed&wctx=THIRDONE" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 POST http://localhost/ application/x-www-form-urlencoded 9548
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 0.4533ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 0.8402ms 302
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Server.Kestrel[32]
app_1_9d5f435a2428 |       Connection id "0HLIM4Q7UFGE6", Request id "0HLIM4Q7UFGE6:00000001": the application completed without reading the entire request body.
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:57 +0000] "POST / HTTP/1.1" 302 0 "https://auth.org.customer.com/adfs/ls/?wtrealm=https%3A%2F%2Flocalhost%3A44396%2F&wa=wsignin1.0&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed&wctx=FOURTHONE" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:57 +0000] "POST / HTTP/1.1" 302 0 "https://auth.org.customer.com/adfs/ls/?wtrealm=https%3A%2F%2Flocalhost%3A44396%2F&wa=wsignin1.0&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed&wctx=FIFTHONE" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 POST http://localhost/ application/x-www-form-urlencoded 9548
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 0.2683ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 0.6077ms 302
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Server.Kestrel[32]
app_1_9d5f435a2428 |       Connection id "0HLIM4Q7UFGE7", Request id "0HLIM4Q7UFGE7:00000001": the application completed without reading the entire request body.
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:14:57 +0000] "POST / HTTP/1.1" 302 0 "https://auth.org.customer.com/adfs/ls/?wtrealm=https%3A%2F%2Flocalhost%3A44396%2F&wa=wsignin1.0&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed&wctx=SIXTHONE" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 POST http://localhost/ application/x-www-form-urlencoded 9536
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 0.2682ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 0.6042ms 302
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Server.Kestrel[32]
app_1_9d5f435a2428 |       Connection id "0HLIM4Q7UFGE8", Request id "0HLIM4Q7UFGE8:00000001": the application completed without reading the entire request body.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
app_1_9d5f435a2428 |       Request starting HTTP/1.0 GET http://localhost/
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler[7]
app_1_9d5f435a2428 |       Cookies was not authenticated. Failure message: Unprotect ticket failed
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
app_1_9d5f435a2428 |       Route matched with {action = "Index", controller = "Home"}. Executing action wsfed_issue.Controllers.HomeController.Index (wsfed-issue)
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
app_1_9d5f435a2428 |       Authorization failed.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[3]
app_1_9d5f435a2428 |       Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.ChallengeResult[1]
app_1_9d5f435a2428 |       Executing ChallengeResult with authentication schemes ().
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Authentication.WsFederation.WsFederationHandler[12]
app_1_9d5f435a2428 |       AuthenticationScheme: WsFederation was challenged.
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
app_1_9d5f435a2428 |       Executed action wsfed_issue.Controllers.HomeController.Index (wsfed-issue) in 0.2562ms
app_1_9d5f435a2428 | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1_9d5f435a2428 |       Request finished in 2.7287ms 302
web_1_e8e63ccbdc0e | 172.21.0.1 - - [29/Nov/2018:18:15:07 +0000] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" "-"

@Tratcher
Copy link
Member

Did you put that troubleshooting middleware at the top of Startup.Configure right after UseForwardedHeaders?

&wreply=http%3A%2F%2Flocalhost%2Fsignin-wsfed in the generated auth url is wrong, it's not getting the correct scheme or port.

@rynnova
Copy link
Author

rynnova commented Nov 29, 2018

I implemented the middleware here: rynnova/docker-swarm-nginx-dotnet-wsfederation-issue@5188e9b

It's in the Configure block, but there's a separate ConfigureServices block.

EDIT: Interesting, I'll see if the auth URL is anything on the ADFS side.

@Tratcher
Copy link
Member

Move app.Use(Middleware); right before app.UseAuthentication();

@rynnova
Copy link
Author

rynnova commented Nov 29, 2018

It did this the moment I moved it:

app_1  | System.InvalidOperationException: OnStarting cannot be set because the response has already started.
app_1  |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ThrowResponseAlreadyStartedException(String value)
app_1  |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.OnStarting(Func`2 callback, Object state)
app_1  |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.Microsoft.AspNetCore.Http.Features.IHttpResponseFeature.OnStarting(Func`2 callback, Object state)
app_1  |    at Microsoft.AspNetCore.Http.Internal.DefaultHttpResponse.OnStarting(Func`2 callback, Object state)
app_1  |    at Microsoft.AspNetCore.Http.HttpResponse.OnStarting(Func`1 callback)
app_1  |    at Microsoft.AspNetCore.Authentication.Cookies.CookieAuthenticationHandler.InitializeHandlerAsync()
app_1  |    at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.InitializeAsync(AuthenticationScheme scheme, HttpContext context)
app_1  |    at Microsoft.AspNetCore.Authentication.AuthenticationHandlerProvider.GetHandlerAsync(HttpContext context, String authenticationScheme)
app_1  |    at Microsoft.AspNetCore.Authentication.AuthenticationService.AuthenticateAsync(HttpContext context, String scheme)
app_1  |    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
app_1  |    at wsfed_issue.Startup.Middleware(HttpContext context, Func`1 next) in /source/Startup.cs:line 140
app_1  |    at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
app_1  |    at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
app_1  |    at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
app_1  |    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
app_1  | info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
app_1  |       Request finished in 14.0072ms 200 text/plain

It makes sense why it happened, but what's the right approach here, then?

By the way, that line is this one: https://github.com/ryakstis/docker-swarm-nginx-dotnet-wsfederation-issue/blob/master/Startup.cs#L140

@Tratcher
Copy link
Member

Ah, I hadn't read the contents of Middleware. Either convert it back to logs or do not call next().

@rynnova
Copy link
Author

rynnova commented Nov 29, 2018

It's no problem, it still posted a response, which actually led to an interesting piece of information:

Looking at the KnownProxies, I assumed that the nginx container would send its own X-Forwarded-For IP address back to ASP .NET Core. However, it sends the Docker network's IPAM gateway IP address instead, 172.21.0.1. KnownProxies has 172.21.0.3 instead for the web service.

Here's a sample response:

Request Method: GET
Request Scheme: http
Request Path: /
Request Headers:
Cache-Control: no-cache
Connection: close
Pragma: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: <removed>
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
X-Real-IP: 172.21.0.1
X-Forwarded-For: 172.21.0.1
X-Forwarded-Proto: https
Upgrade-Insecure-Requests: 1
DNT: 1

Request RemoteIp: ::ffff:172.21.0.3
Proxy: 172.21.0.3

@Tratcher
Copy link
Member

Note the ::ffff: prefix, we didn't add extra support for that until 2.2. You'll need to include it in your KnownProxies registration.

@rynnova
Copy link
Author

rynnova commented Nov 29, 2018

Okay, I tried using:

        private void ConfigureForwardedHeaders(
            ForwardedHeadersOptions options) {
            foreach (var address in Proxies)
                options.KnownProxies.Add(address);
        }

        private IEnumerable<IPAddress> Proxies {
            get {
                IEnumerable<IPAddress> empty = new IPAddress[] { };
                var gateway = new[] {IPAddress.Parse("172.21.0.1")};
                var nginx = Dns.GetHostAddresses("web");
                return nginx
                    .Concat(gateway)
                    .Select(ToMultiProtocolAddresses)
                    .Aggregate(empty, ConcatenateLists);
            }
        }

        private IEnumerable<IPAddress> ToMultiProtocolAddresses(
            IPAddress address) => new[]
            {address.MapToIPv4(), address.MapToIPv6()};

        private IEnumerable<IPAddress> ConcatenateLists(
            IEnumerable<IPAddress> aggregate,
            IEnumerable<IPAddress> current) => aggregate.Concat(current);

...and that hasn't helped to resolve anything yet. Same redirects.

@Tratcher
Copy link
Member

If the proxy is reporting as ::ffff:172.21.0.3 why are you adding 172.21.0.1?

Alternatively you could use the KnownNetworks option and accept the range 127.21.0.0/8.

@rynnova
Copy link
Author

rynnova commented Nov 30, 2018

Just to be safe. I'll try out KnownNetworks.

EDIT: Details. The reason I added 172.21.0.1 is because that's what nginx reports as the forwarded IP: X-Forwarded-For: 172.21.0.1.

@rynnova
Copy link
Author

rynnova commented Nov 30, 2018

I used:

        private void ConfigureForwardedHeaders(
            ForwardedHeadersOptions options) {
            var network = new IPNetwork(IPAddress.Parse("172.21.0.0"), 8);
            options.KnownNetworks.Add(network);
        }

...and it did not change the redirect issue.

@Tratcher
Copy link
Member

X-Forwarded-For is the client, not the proxy. What does the diagnostic output look like now?

@rynnova
Copy link
Author

rynnova commented Dec 3, 2018

Request Method: GET
Request Scheme: http
Request Path: /
Request Headers:
Connection: close
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
X-Real-IP: 172.21.0.1
X-Forwarded-For: 172.21.0.1
X-Forwarded-Proto: https
Upgrade-Insecure-Requests: 1
DNT: 1

Request RemoteIp: ::ffff:172.21.0.3

The Compose logs look the same as before.

@Tratcher
Copy link
Member

Tratcher commented Dec 3, 2018

And if you change your IPNetwork to ::ffff:172.21.0.0, 120, then what happens?

@rynnova
Copy link
Author

rynnova commented Dec 3, 2018

Same thing. 6 redirects, then "an error occurred" served by the ADFS server to the browser, with no valuable information at that page. Same headers.

@Tratcher
Copy link
Member

Tratcher commented Dec 3, 2018

The ForwardedHeadersMiddleware has its own logging, it should tell you what's wrong. You may have to turn the logs up to Debug.
https://github.com/aspnet/BasicMiddleware/blob/1de636cd18309070792b68888ca29c8b85fac98b/src/Microsoft.AspNetCore.HttpOverrides/ForwardedHeadersMiddleware.cs#L236

@rynnova
Copy link
Author

rynnova commented Dec 4, 2018

Aha!

app_1  | dbug: Microsoft.AspNetCore.HttpOverrides.ForwardedHeadersMiddleware[1]
app_1  |       Unknown proxy: [::ffff:172.21.0.3]:56322

So WS-Federation expects that port open in nginx. Unfortunately, that port changes every time I run docker-compose up. It also doesn't help that the moment I removed the Compose network, the second octet in the subnet changed.

So we can't hard-code an IP network because of Docker Swarm/Compose, and we have to have some kind of common port in the nginx configuration, since nginx doesn't support listening on dynamic ports, which would imply configuring the WS-Federation middleware to reply to authentication requests on a static port instead of a dynamic one.

@Tratcher
Copy link
Member

Tratcher commented Dec 4, 2018

It's not checking the port, only the IP. You say the IP keeps changing? If it's the second octet then you can adjust the network range to allow for that. I think that would be 104.

@rynnova
Copy link
Author

rynnova commented Dec 5, 2018

It only changes the IP when I remove the Docker network, like when I run docker-compose down, not every time. Generalised the Known Network:

        private void ConfigureForwardedHeaders(ForwardedHeadersOptions options) {
            options.KnownNetworks.Add(Network);
        }

        private IPNetwork _network;

        private IPNetwork Network => _network ?? (_network = new IPNetwork(LocalAddress, 16));

        private IPAddress LocalAddress => NetworkInterface
            .GetAllNetworkInterfaces()
            .Where(n => n.OperationalStatus == OperationalStatus.Up)
            .Where(n => n.NetworkInterfaceType != NetworkInterfaceType.Loopback)
            .SelectMany(n => n.GetIPProperties()?.GatewayAddresses)
            .Select(ToGatewayAddress)
            .FirstOrDefault(a => a != null);

        private IPAddress ToGatewayAddress(GatewayIPAddressInformation info) {
            var firstThreeOctets = info?.Address.GetAddressBytes().Take(3);
            var octets = firstThreeOctets.Append((byte)0).ToArray();
            return new IPAddress(octets);
        }

And still, multiple redirects with Unknown proxy: [::ffff:172.18.0.3]:33150. 33150 is the part that changes on restart of the Swarm services.

I'll push these changes up just to have the progress made so far.

@Tratcher
Copy link
Member

Tratcher commented Dec 5, 2018

Might I suggest simplifying until you have a working scenario? Even if that means hardcoding addresses. I think your network ranges are wrong (try 8 instead of 16). I also don't see anything in your code that handles the ::ffff: prefix. Note 2.2 released yesterday and it includes handling for this.

@rynnova
Copy link
Author

rynnova commented Dec 5, 2018

I had the :ffff: prefix implemented as of this comment, but removed it since.

I changed return new IPAddress(octets); to return new IPAddress(octets).MapToIPv6();. The proxy network address read ::ffff:172.18.0.0. The error still persisted.

I then changed private IPNetwork Network => _network ?? (_network = new IPNetwork(LocalAddress, 16)); to private IPNetwork Network => _network ?? (_network = new IPNetwork(LocalAddress, 8));. The error still persists.

@rynnova
Copy link
Author

rynnova commented Dec 6, 2018

I have made the following changes:

So far, the multiple redirection issue remains, along with the same debug messages.

@Eilon
Copy link
Member

Eilon commented Dec 6, 2018

Hi. It looks like this is a question about how to use ASP.NET Core. While we do our best to look through all the issues filed here, to get a faster response we suggest posting your questions to StackOverflow using the asp.net-core tag.

@Eilon Eilon closed this as completed Dec 6, 2018
@rynnova
Copy link
Author

rynnova commented Dec 7, 2018

@Eilon I had issues signing up with Stack Overflow this morning, but they finally let me create an account.

I've re-posted the question here at Stack Overflow and will await a response.

@rynnova
Copy link
Author

rynnova commented Dec 10, 2018

Hey @Eilon, I got responses daily here on Github with Chris, but I haven't had a response in two days on Stack Overflow, are you sure I'll get a faster response there?

@Tratcher
Copy link
Member

There's only so much time we can devote to any individual issue.

@rynnova
Copy link
Author

rynnova commented Dec 10, 2018

Understood. Thank you for your time, @Tratcher. You were a great help on this. Hopefully someone will notice the SO question.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants