Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom OIDC (Azure) leads to phoenix crashing #642

Closed
allanhung opened this issue May 24, 2022 · 16 comments
Closed

Custom OIDC (Azure) leads to phoenix crashing #642

allanhung opened this issue May 24, 2022 · 16 comments

Comments

@allanhung
Copy link

allanhung commented May 24, 2022 •
edited

Hi,

I try to config firezone with azure oidc. But I get error when start phonenix. Here are the logs:

13:04:06.636 [info] Access FzHttpWeb.Endpoint at https://firezone.shared.in.quid.com
13:04:14.677 [notice] Application fz_http exited: FzHttp.Application.start(:normal, []) returned an error: shutdown: failed to start child: OpenIDConnect.Worker
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, :update_documents, %HTTPoison.Error{id: nil, reason: :timeout}}
            (openid_connect 0.2.2) lib/openid_connect/worker.ex:55: OpenIDConnect.Worker.update_documents/2
            (openid_connect 0.2.2) lib/openid_connect/worker.ex:23: anonymous fn/1 in OpenIDConnect.Worker.init/1
            (elixir 1.13.4) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
            (elixir 1.13.4) lib/enum.ex:1496: Enum.into/3
            (openid_connect 0.2.2) lib/openid_connect/worker.ex:22: OpenIDConnect.Worker.init/1
            (stdlib 4.0) gen_server.erl:848: :gen_server.init_it/2
            (stdlib 4.0) gen_server.erl:811: :gen_server.init_it/6
            (stdlib 4.0) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
{"Kernel pid terminated",application_controller,"{application_start_failure,fz_http,{{shutdown,{failed_to_start_child,'Elixir.OpenIDConnect.Worker',{{badmatch,{error,update_documents,#{'__exception__' => true,'__struct__' => 'Elixir.HTTPoison.Error',id => nil,reason => timeout}}},[{'Elixir.OpenIDConnect.Worker',update_documents,2,[{file,\"lib/openid_connect/worker.ex\"},{line,55}]},{'Elixir.OpenIDConnect.Worker','-init/1-fun-0-',1,[{file,\"lib/openid_connect/worker.ex\"},{line,23}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,\"lib/enum.ex\"},{line,1593}]},{'Elixir.Enum',into,3,[{file,\"lib/enum.ex\"},{line,1496}]},{'Elixir.OpenIDConnect.Worker',init,1,[{file,\"lib/openid_connect/worker.ex\"},{line,22}]},{gen_server,init_it,2,[{file,\"gen_server.erl\"},{line,848}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,811}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,240}]}]}}},{'Elixir.FzHttp.Application',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,fz_http,{{shutdown,{failed_to_start_child,'Elixir.OpenIDConnect.Worker',{{badmatch,{error,update_documents,#{'__exception__' => true,'__struct__' => 'Elixir.HTTPoison.Error',id => nil,reason => timeout}}},[{'Elixir.OpenIDConnect.Worker',update_documents,2,[{file,"lib/openid_connect/worker.ex"},{line,55}]},{'Elixir.OpenIDConnect.Worker','-init/1-fun-0-',1,[{file,"lib/openid_connect/worker.ex"},{line,23}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1593}]},{'Elixir.Enum',into,3,[{file,"lib/enum.ex"},{line,1496}]},{'Elixir.OpenIDConnect.Worker',init,1,[{file,"lib/openid_connect/worker.ex"},{line,22}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,848}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,811}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}}},{'Elixir.FzHttp.Application',start,[normal,[]]}}})

oidc setting:

default['firezone']['authentication']['oidc'] = {
  azure: {
    discovery_document_uri: "https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/v2.0/.well-known/openid-configuration",
    client_id: "d326b254-954d-414f-a481-01c8737786c3",
    client_secret: "secret",
    response_type: "code",
    redirect_uri: "https://firezone.xxx.com/auth/oidc/azure/callback",
    scope: "openid email profile",
    label: "azure"
  }
}

Many thanks for your help.
@jamilbk
Copy link
Member

jamilbk commented May 24, 2022 •
edited

Hi @allanhung -- are you sure your Azure portal is configured correctly? Looks that discovery_document_uri is returning an error:

{"error":"invalid_tenant","error_description":"AADSTS90002: Tenant '77287fa5-9999-8888-7777-d9d20280f8be' not found. Check to make sure you have the correct tenant ID and are signing into the correct cloud. Check with your subscription administrator, this may happen if there are no active subscriptions for the tenant.\r\nTrace ID: b3848545-01ac-4f3a-b4d9-ac91776df300\r\nCorrelation ID: 46a5f8d3-745d-423d-9ef8-39978e4ff83f\r\nTimestamp: 2022-05-24 18:08:42Z","error_codes":[90002],"timestamp":"2022-05-24 18:08:42Z","trace_id":"b3848545-01ac-4f3a-b4d9-ac91776df300","correlation_id":"46a5f8d3-745d-423d-9ef8-39978e4ff83f","error_uri":"https://login.microsoftonline.com/error?code=90002"}

I'm guessing Phoenix isn't able to fetch the necessary documents because that response isn't valid.

@allanhung
Copy link
Author

Sorry, I mask some info. I have updated the comment with correct discovery_document_uri and the client_id.

@jamilbk
Copy link
Member

jamilbk commented May 24, 2022

@allanhung Ah thanks. So it looks like OpenID Connect is timing out when trying to fetch your OIDC documents. Can you verify your Firezone server can reach https://login.microsoft.com by pasting the output of openssl s_client -connect login.microsoft.com:443? E.g.:

CONNECTED(00000006)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
verify return:1
depth=0 C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = graph.windows.net
verify return:1
---
Certificate chain
 0 s:/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=graph.windows.net
   i:/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
 1 s:/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIROzCCECOgAwIBAgIQBN5UXxys67VM441vzLuCRDANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMjIwNDE5MDAwMDAwWhcN
MjMwNDE5MjM1OTU5WjBwMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3Rv
bjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0
aW9uMRowGAYDVQQDExFncmFwaC53aW5kb3dzLm5ldDCCASIwDQYJKoZIhvcNAQEB
BQADggEPADCCAQoCggEBALsr8Utw2t/0O2lCQojTjDct5xqNTGPY9LtnqFrT4BtC
FpWNIl3nAnMtbTWgIErgsDvBk8NEMzCm97/REGzrmKYmtBb8PxWZv6mrrrYGaJzl
JCGZtldpLlCa3sNhaS3rOK2HFhlS3gnqk0dEmEBIhUDKOFVT1UNMMguVjBBSGpar
2vG7DQtvN4tqvmFIEYXhLSW3lNr411my6ndh59R0aajjww/UjIRFDbFK2Q2SXHGa
RKjEFt4/gCY1DgzPyZGszfSwgqnYazwpX2EKXpmq0m45io7TCa4O6NiCHadx28tt
pYl508zt55s/4XXxBVenldeNg2jn6ZC4NRltMFgEuiUCAwEAAaOCDfIwgg3uMB8G
A1UdIwQYMBaAFA+AYRyCMWHVLyjnjUY4tCzhxtniMB0GA1UdDgQWBBSDAnb5tVwE
mYBvSrMKNvXBuoU4ezCCCp8GA1UdEQSCCpYwggqSghFncmFwaC53aW5kb3dzLm5l
dIISKi5hYWRnLndpbmRvd3MubmV0giUqLmFhZGtkcy5wcGUucmVwb3J0aW5nLm1z
aWRlbnRpdHkuY29tgiUqLmFhZGtkcy5wcmQucmVwb3J0aW5nLm1zaWRlbnRpdHku
Y29tgicqLmFjY2Vzc2NvbnRyb2wuYWFkdHN0My53aW5kb3dzLWludC5uZXSCHyou
YWNjZXNzY29udHJvbC53aW5kb3dzLXBwZS5uZXSCGyouYWNjZXNzY29udHJvbC53
aW5kb3dzLm5ldIIqKi5hZGxzLmFhZGtkcy5wcGUucmVwb3J0aW5nLm1zaWRlbnRp
dHkuY29tgioqLmFkbHMuYWFka2RzLnByZC5yZXBvcnRpbmcubXNpZGVudGl0eS5j
b22CKiouYWR0aS5hYWRrZHMucHBlLnJlcG9ydGluZy5tc2lkZW50aXR5LmNvbYIq
Ki5hZHRpLmFhZGtkcy5wcmQucmVwb3J0aW5nLm1zaWRlbnRpdHkuY29tgg0qLmF1
dGhhcHAubmV0gh0qLmF1dGhvcml6YXRpb24uYXp1cmUtcHBlLm5ldIIZKi5hdXRo
b3JpemF0aW9uLmF6dXJlLm5ldIIOKi5iMmNsb2dpbi5jb22CEiouY3BpbS53aW5k
b3dzLm5ldIIpKi5kMmsuYWFka2RzLnBwZS5yZXBvcnRpbmcubXNpZGVudGl0eS5j
b22CKSouZDJrLmFhZGtkcy5wcmQucmVwb3J0aW5nLm1zaWRlbnRpdHkuY29tghcq
LmZwLm1lYXN1cmUub2ZmaWNlLmNvbYIVKi5nYXRld2F5LndpbmRvd3MubmV0ghgq
LklkZW50aXR5LmF6dXJlLWludC5uZXSCFCouSWRlbnRpdHkuYXp1cmUubmV0ghAq
LmxvZ2luLmxpdmUuY29tghUqLmxvZ2luLm1pY3Jvc29mdC5jb22CGyoubG9naW4u
bWljcm9zb2Z0b25saW5lLmNvbYIXKi5sb2dpbi53aW5kb3dzLXBwZS5uZXSCGSou
bG9naW5jZXJ0Lm1pY3Jvc29mdC5jb22CGyoubG9naW5jZXJ0LndpbmRvd3MtcHBl
Lm5ldIIgKi5taWNyb3NvZnRhaWstaW50LmF6dXJlLWludC5uZXSCGCoubWljcm9z
b2Z0YWlrLmF6dXJlLm5ldIIYKi5wdC5hYWRnLm1zaWRlbnRpdHkuY29tghcqLnIu
bG9naW4ubWljcm9zb2Z0LmNvbYIdKi5yLmxvZ2luLm1pY3Jvc29mdG9ubGluZS5j
b22CGyouci5wcmQuYWFkZy5tc2lkZW50aXR5LmNvbYIRKi53aW5kb3dzLXBwZS5u
ZXSCIWFhZGNkbi5wcml2YXRlbGluay5tc2lkZW50aXR5LmNvbYIQYWFkZy53aW5k
b3dzLm5ldIIXYWFkZ2Nkbi53aW5kb3dzLWludC5uZXSCE2FhZGdjZG4ud2luZG93
cy5uZXSCFmFhZGd2Ni5wcGUud2luZG93cy5uZXSCEmFhZGd2Ni53aW5kb3dzLm5l
dIIlYWNjZXNzY29udHJvbC5hYWR0c3QzLndpbmRvd3MtaW50Lm5ldIIUYWNjb3Vu
dC5saXZlLWludC5jb22CEGFjY291bnQubGl2ZS5jb22CFmFwaS5sb2dpbi5saXZl
LWludC5jb22CHWFwaS5sb2dpbi5taWNyb3NvZnRvbmxpbmUuY29tghdhcGkucGFz
c3dvcmQuY2NzY3RwLmNvbYIlYXBpLnBhc3N3b3JkcmVzZXQubWljcm9zb2Z0b25s
aW5lLmNvbYIiYXV0b2xvZ29uLm1pY3Jvc29mdGF6dXJlYWQtc3NvLmNvbYIQYmVj
d3MuY2NzY3RwLmNvbYImY2xpZW50Y29uZmlnLm1pY3Jvc29mdG9ubGluZS1wLWlu
dC5uZXSCImNsaWVudGNvbmZpZy5taWNyb3NvZnRvbmxpbmUtcC5uZXSCGWNvbXBh
bnltYW5hZ2VyLmNjc2N0cC5jb22CImNvbXBhbnltYW5hZ2VyLm1pY3Jvc29mdG9u
bGluZS5jb22CEGNwaW0ud2luZG93cy5uZXSCIGRldmljZS5sb2dpbi5taWNyb3Nv
ZnRvbmxpbmUuY29tghxkZXZpY2UubG9naW4ud2luZG93cy1wcGUubmV0gh5kaXJl
Y3Rvcnlwcm94eS5wcGUud2luZG93cy5uZXSCGmRpcmVjdG9yeXByb3h5LndpbmRv
d3MubmV0ghpnYXRld2F5Zm9ya2luZy53aW5kb3dzLm5ldIIVZ3JhcGgucHBlLndp
bmRvd3MubmV0ghZncmFwaHN0b3JlLndpbmRvd3MubmV0ghdpcHY2LmxvZ2luLmxp
dmUtaW50LmNvbYIcbG9naW4tdXMubWljcm9zb2Z0b25saW5lLmNvbYISbG9naW4u
bGl2ZS1pbnQuY29tgg5sb2dpbi5saXZlLmNvbYIXbG9naW4ubWljcm9zb2Z0LXBw
ZS5jb22CE2xvZ2luLm1pY3Jvc29mdC5jb22CHWxvZ2luLm1pY3Jvc29mdG9ubGlu
ZS1pbnQuY29tghtsb2dpbi5taWNyb3NvZnRvbmxpbmUtcC5jb22CHWxvZ2luLm1p
Y3Jvc29mdG9ubGluZS1wc3QuY29tghlsb2dpbi5taWNyb3NvZnRvbmxpbmUuY29t
ghZsb2dpbi5wYXNzcG9ydC1pbnQuY29tghFsb2dpbi53aW5kb3dzLm5ldIIhbG9n
aW5jZXJ0Lm1pY3Jvc29mdG9ubGluZS1pbnQuY29tgh1sb2dpbmNlcnQubWljcm9z
b2Z0b25saW5lLmNvbYIZbG9naW5uZXQucGFzc3BvcnQtaW50LmNvbYIebWljcm9z
b2Z0YWlrLWludC5henVyZS1pbnQubmV0ghZtaWNyb3NvZnRhaWsuYXp1cmUubmV0
ghhtc25pYS5sb2dpbi5saXZlLWludC5jb22CG21zbmlhbG9naW4ucGFzc3BvcnQt
aW50LmNvbYIfbmV4dXMubWljcm9zb2Z0b25saW5lLXAtaW50LmNvbYIbbmV4dXMu
bWljcm9zb2Z0b25saW5lLXAuY29tghZuZXh1cy5wYXNzcG9ydC1pbnQuY29tghNw
YXMud2luZG93cy1wcGUubmV0gg9wYXMud2luZG93cy5uZXSCE3Bhc3N3b3JkLmNj
c2N0cC5jb22CLXBhc3N3b3JkcmVzZXQuYWN0aXZlZGlyZWN0b3J5LndpbmRvd3Nh
enVyZS51c4IhcGFzc3dvcmRyZXNldC5taWNyb3NvZnRvbmxpbmUuY29tgiVwcGUu
YWFkY2RuLnByaXZhdGVsaW5rLm1zaWRlbnRpdHkuY29tgiBwcm92aXNpb25pbmcu
bWljcm9zb2Z0b25saW5lLmNvbYITc2lnbnVwLmxpdmUtaW50LmNvbYIPc2lnbnVw
LmxpdmUuY29tgg9zdHMud2luZG93cy5uZXSCGHRvb2xzLmxvZ2luLmxpdmUtaW50
LmNvbYIWeG1sLmxvZ2luLmxpdmUtaW50LmNvbYISeG1sLmxvZ2luLmxpdmUuY29t
MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
gY0GA1UdHwSBhTCBgjA/oD2gO4Y5aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0Rp
Z2ljZXJ0U0hBMlNlY3VyZVNlcnZlckNBLTEuY3JsMD+gPaA7hjlodHRwOi8vY3Js
NC5kaWdpY2VydC5jb20vRGlnaWNlcnRTSEEyU2VjdXJlU2VydmVyQ0EtMS5jcmww
PgYDVR0gBDcwNTAzBgZngQwBAgIwKTAnBggrBgEFBQcCARYbaHR0cDovL3d3dy5k
aWdpY2VydC5jb20vQ1BTMH4GCCsGAQUFBwEBBHIwcDAkBggrBgEFBQcwAYYYaHR0
cDovL29jc3AuZGlnaWNlcnQuY29tMEgGCCsGAQUFBzAChjxodHRwOi8vY2FjZXJ0
cy5kaWdpY2VydC5jb20vRGlnaUNlcnRTSEEyU2VjdXJlU2VydmVyQ0EtMi5jcnQw
CQYDVR0TBAIwADCCAX0GCisGAQQB1nkCBAIEggFtBIIBaQFnAHUA6D7Q2j71BjUy
51covIlryQPTy9ERa+zraeF3fW0GvW4AAAGAQ3SPnAAABAMARjBEAiAZVJsS2K5Z
EIPo+VWKNagnlkgXCdYBEN2zn2/MA5EziwIgfKZJNfQqdJzJFpqe3gNHNcvkjjwI
uF3+ZAemPsqsEpYAdgA1zxkbv7FsV78PrUxtQsu7ticgJlHqP+Eq76gDwzvWTAAA
AYBDdI/mAAAEAwBHMEUCIQCmx5c2lTHAfMt7wdVC4cjDsFeYNTaxRTUhHJiKAzen
nwIgKAbMj07u9grRkpQmEMnutvUVh75Z8hkQjbPPZiMfwHoAdgCzc3cH4YRQ+GOG
1gWp3BEJSnktsWcMC4fc8AMOeTalmgAAAYBDdJAaAAAEAwBHMEUCIQC+vA3ur1Xn
TVXZH+Gpk0Rv9rNIJk9LcBcShGepVvBymAIgA4i0kWrK4Qohz9PRUCGwGTBT9dHF
ga2mnNLMn+i3fg4wDQYJKoZIhvcNAQELBQADggEBAIsaRxZjDIw7ecTsJ7956nPp
upkYcFnBvchJaS4pdcFjSHNv68RgWpxiN7YU8KP/X+lnWB1Xn0Tg7ljFonkC726B
L/ouMSKbUJ3gtrF4Wd/CfUuSO8Iv5Q4R97RiB4Q3ity8ui5+IFgoF9TEI8hebBbJ
EwQrDECxkju9buqV8/Ee1dcT6aIOtqoSNdqRwozFQwKviID6BR13G4m7SwzsCRIA
s4TC5rEPfi6l2k+TPrhWtbpBCoa7/R9ZhZuInbdGpS+EJF5sNmT7lqHv7ERq1s0C
irWT2P1o9Y7dBc9V/CLQakPVpI+XcNgABXw9CEPgdLKEAuC3WhpBdvWT4OGhUhs=
-----END CERTIFICATE-----
subject=/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=graph.windows.net
issuer=/C=US/O=DigiCert Inc/CN=DigiCert SHA2 Secure Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 6194 bytes and written 354 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 6425000010F7B9C5121017C7DD01F86B38AB52A2604278F59841632ADFAC20C2
    Session-ID-ctx:
    Master-Key: FB72F1FF6499C1A121E16336F6E313D0DF3F4F5A0BDCEC27F6267DF84031AA6318DC6017DB96D961ED8CAC4870120BAB
    Start Time: 1653422363
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

@allanhung
Copy link
Author

openssl s_client -connect login.microsoft.com:443

CONNECTED(00000003)
---
Certificate chain
 0 s:C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = graph.windows.net
   i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
 1 s:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
   i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIROzCCECOgAwIBAgIQBN5UXxys67VM441vzLuCRDANBgkqhkiG9w0BAQsFADBN
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E
aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMjIwNDE5MDAwMDAwWhcN
MjMwNDE5MjM1OTU5WjBwMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3Rv
bjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0
aW9uMRowGAYDVQQDExFncmFwaC53aW5kb3dzLm5ldDCCASIwDQYJKoZIhvcNAQEB
BQADggEPADCCAQoCggEBALsr8Utw2t/0O2lCQojTjDct5xqNTGPY9LtnqFrT4BtC
FpWNIl3nAnMtbTWgIErgsDvBk8NEMzCm97/REGzrmKYmtBb8PxWZv6mrrrYGaJzl
JCGZtldpLlCa3sNhaS3rOK2HFhlS3gnqk0dEmEBIhUDKOFVT1UNMMguVjBBSGpar
2vG7DQtvN4tqvmFIEYXhLSW3lNr411my6ndh59R0aajjww/UjIRFDbFK2Q2SXHGa
RKjEFt4/gCY1DgzPyZGszfSwgqnYazwpX2EKXpmq0m45io7TCa4O6NiCHadx28tt
pYl508zt55s/4XXxBVenldeNg2jn6ZC4NRltMFgEuiUCAwEAAaOCDfIwgg3uMB8G
A1UdIwQYMBaAFA+AYRyCMWHVLyjnjUY4tCzhxtniMB0GA1UdDgQWBBSDAnb5tVwE
mYBvSrMKNvXBuoU4ezCCCp8GA1UdEQSCCpYwggqSghFncmFwaC53aW5kb3dzLm5l
dIISKi5hYWRnLndpbmRvd3MubmV0giUqLmFhZGtkcy5wcGUucmVwb3J0aW5nLm1z
aWRlbnRpdHkuY29tgiUqLmFhZGtkcy5wcmQucmVwb3J0aW5nLm1zaWRlbnRpdHku
Y29tgicqLmFjY2Vzc2NvbnRyb2wuYWFkdHN0My53aW5kb3dzLWludC5uZXSCHyou
YWNjZXNzY29udHJvbC53aW5kb3dzLXBwZS5uZXSCGyouYWNjZXNzY29udHJvbC53
aW5kb3dzLm5ldIIqKi5hZGxzLmFhZGtkcy5wcGUucmVwb3J0aW5nLm1zaWRlbnRp
dHkuY29tgioqLmFkbHMuYWFka2RzLnByZC5yZXBvcnRpbmcubXNpZGVudGl0eS5j
b22CKiouYWR0aS5hYWRrZHMucHBlLnJlcG9ydGluZy5tc2lkZW50aXR5LmNvbYIq
Ki5hZHRpLmFhZGtkcy5wcmQucmVwb3J0aW5nLm1zaWRlbnRpdHkuY29tgg0qLmF1
dGhhcHAubmV0gh0qLmF1dGhvcml6YXRpb24uYXp1cmUtcHBlLm5ldIIZKi5hdXRo
b3JpemF0aW9uLmF6dXJlLm5ldIIOKi5iMmNsb2dpbi5jb22CEiouY3BpbS53aW5k
b3dzLm5ldIIpKi5kMmsuYWFka2RzLnBwZS5yZXBvcnRpbmcubXNpZGVudGl0eS5j
b22CKSouZDJrLmFhZGtkcy5wcmQucmVwb3J0aW5nLm1zaWRlbnRpdHkuY29tghcq
LmZwLm1lYXN1cmUub2ZmaWNlLmNvbYIVKi5nYXRld2F5LndpbmRvd3MubmV0ghgq
LklkZW50aXR5LmF6dXJlLWludC5uZXSCFCouSWRlbnRpdHkuYXp1cmUubmV0ghAq
LmxvZ2luLmxpdmUuY29tghUqLmxvZ2luLm1pY3Jvc29mdC5jb22CGyoubG9naW4u
bWljcm9zb2Z0b25saW5lLmNvbYIXKi5sb2dpbi53aW5kb3dzLXBwZS5uZXSCGSou
bG9naW5jZXJ0Lm1pY3Jvc29mdC5jb22CGyoubG9naW5jZXJ0LndpbmRvd3MtcHBl
Lm5ldIIgKi5taWNyb3NvZnRhaWstaW50LmF6dXJlLWludC5uZXSCGCoubWljcm9z
b2Z0YWlrLmF6dXJlLm5ldIIYKi5wdC5hYWRnLm1zaWRlbnRpdHkuY29tghcqLnIu
bG9naW4ubWljcm9zb2Z0LmNvbYIdKi5yLmxvZ2luLm1pY3Jvc29mdG9ubGluZS5j
b22CGyouci5wcmQuYWFkZy5tc2lkZW50aXR5LmNvbYIRKi53aW5kb3dzLXBwZS5u
ZXSCIWFhZGNkbi5wcml2YXRlbGluay5tc2lkZW50aXR5LmNvbYIQYWFkZy53aW5k
b3dzLm5ldIIXYWFkZ2Nkbi53aW5kb3dzLWludC5uZXSCE2FhZGdjZG4ud2luZG93
cy5uZXSCFmFhZGd2Ni5wcGUud2luZG93cy5uZXSCEmFhZGd2Ni53aW5kb3dzLm5l
dIIlYWNjZXNzY29udHJvbC5hYWR0c3QzLndpbmRvd3MtaW50Lm5ldIIUYWNjb3Vu
dC5saXZlLWludC5jb22CEGFjY291bnQubGl2ZS5jb22CFmFwaS5sb2dpbi5saXZl
LWludC5jb22CHWFwaS5sb2dpbi5taWNyb3NvZnRvbmxpbmUuY29tghdhcGkucGFz
c3dvcmQuY2NzY3RwLmNvbYIlYXBpLnBhc3N3b3JkcmVzZXQubWljcm9zb2Z0b25s
aW5lLmNvbYIiYXV0b2xvZ29uLm1pY3Jvc29mdGF6dXJlYWQtc3NvLmNvbYIQYmVj
d3MuY2NzY3RwLmNvbYImY2xpZW50Y29uZmlnLm1pY3Jvc29mdG9ubGluZS1wLWlu
dC5uZXSCImNsaWVudGNvbmZpZy5taWNyb3NvZnRvbmxpbmUtcC5uZXSCGWNvbXBh
bnltYW5hZ2VyLmNjc2N0cC5jb22CImNvbXBhbnltYW5hZ2VyLm1pY3Jvc29mdG9u
bGluZS5jb22CEGNwaW0ud2luZG93cy5uZXSCIGRldmljZS5sb2dpbi5taWNyb3Nv
ZnRvbmxpbmUuY29tghxkZXZpY2UubG9naW4ud2luZG93cy1wcGUubmV0gh5kaXJl
Y3Rvcnlwcm94eS5wcGUud2luZG93cy5uZXSCGmRpcmVjdG9yeXByb3h5LndpbmRv
d3MubmV0ghpnYXRld2F5Zm9ya2luZy53aW5kb3dzLm5ldIIVZ3JhcGgucHBlLndp
bmRvd3MubmV0ghZncmFwaHN0b3JlLndpbmRvd3MubmV0ghdpcHY2LmxvZ2luLmxp
dmUtaW50LmNvbYIcbG9naW4tdXMubWljcm9zb2Z0b25saW5lLmNvbYISbG9naW4u
bGl2ZS1pbnQuY29tgg5sb2dpbi5saXZlLmNvbYIXbG9naW4ubWljcm9zb2Z0LXBw
ZS5jb22CE2xvZ2luLm1pY3Jvc29mdC5jb22CHWxvZ2luLm1pY3Jvc29mdG9ubGlu
ZS1pbnQuY29tghtsb2dpbi5taWNyb3NvZnRvbmxpbmUtcC5jb22CHWxvZ2luLm1p
Y3Jvc29mdG9ubGluZS1wc3QuY29tghlsb2dpbi5taWNyb3NvZnRvbmxpbmUuY29t
ghZsb2dpbi5wYXNzcG9ydC1pbnQuY29tghFsb2dpbi53aW5kb3dzLm5ldIIhbG9n
aW5jZXJ0Lm1pY3Jvc29mdG9ubGluZS1pbnQuY29tgh1sb2dpbmNlcnQubWljcm9z
b2Z0b25saW5lLmNvbYIZbG9naW5uZXQucGFzc3BvcnQtaW50LmNvbYIebWljcm9z
b2Z0YWlrLWludC5henVyZS1pbnQubmV0ghZtaWNyb3NvZnRhaWsuYXp1cmUubmV0
ghhtc25pYS5sb2dpbi5saXZlLWludC5jb22CG21zbmlhbG9naW4ucGFzc3BvcnQt
aW50LmNvbYIfbmV4dXMubWljcm9zb2Z0b25saW5lLXAtaW50LmNvbYIbbmV4dXMu
bWljcm9zb2Z0b25saW5lLXAuY29tghZuZXh1cy5wYXNzcG9ydC1pbnQuY29tghNw
YXMud2luZG93cy1wcGUubmV0gg9wYXMud2luZG93cy5uZXSCE3Bhc3N3b3JkLmNj
c2N0cC5jb22CLXBhc3N3b3JkcmVzZXQuYWN0aXZlZGlyZWN0b3J5LndpbmRvd3Nh
enVyZS51c4IhcGFzc3dvcmRyZXNldC5taWNyb3NvZnRvbmxpbmUuY29tgiVwcGUu
YWFkY2RuLnByaXZhdGVsaW5rLm1zaWRlbnRpdHkuY29tgiBwcm92aXNpb25pbmcu
bWljcm9zb2Z0b25saW5lLmNvbYITc2lnbnVwLmxpdmUtaW50LmNvbYIPc2lnbnVw
LmxpdmUuY29tgg9zdHMud2luZG93cy5uZXSCGHRvb2xzLmxvZ2luLmxpdmUtaW50
LmNvbYIWeG1sLmxvZ2luLmxpdmUtaW50LmNvbYISeG1sLmxvZ2luLmxpdmUuY29t
MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
gY0GA1UdHwSBhTCBgjA/oD2gO4Y5aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0Rp
Z2ljZXJ0U0hBMlNlY3VyZVNlcnZlckNBLTEuY3JsMD+gPaA7hjlodHRwOi8vY3Js
NC5kaWdpY2VydC5jb20vRGlnaWNlcnRTSEEyU2VjdXJlU2VydmVyQ0EtMS5jcmww
PgYDVR0gBDcwNTAzBgZngQwBAgIwKTAnBggrBgEFBQcCARYbaHR0cDovL3d3dy5k
aWdpY2VydC5jb20vQ1BTMH4GCCsGAQUFBwEBBHIwcDAkBggrBgEFBQcwAYYYaHR0
cDovL29jc3AuZGlnaWNlcnQuY29tMEgGCCsGAQUFBzAChjxodHRwOi8vY2FjZXJ0
cy5kaWdpY2VydC5jb20vRGlnaUNlcnRTSEEyU2VjdXJlU2VydmVyQ0EtMi5jcnQw
CQYDVR0TBAIwADCCAX0GCisGAQQB1nkCBAIEggFtBIIBaQFnAHUA6D7Q2j71BjUy
51covIlryQPTy9ERa+zraeF3fW0GvW4AAAGAQ3SPnAAABAMARjBEAiAZVJsS2K5Z
EIPo+VWKNagnlkgXCdYBEN2zn2/MA5EziwIgfKZJNfQqdJzJFpqe3gNHNcvkjjwI
uF3+ZAemPsqsEpYAdgA1zxkbv7FsV78PrUxtQsu7ticgJlHqP+Eq76gDwzvWTAAA
AYBDdI/mAAAEAwBHMEUCIQCmx5c2lTHAfMt7wdVC4cjDsFeYNTaxRTUhHJiKAzen
nwIgKAbMj07u9grRkpQmEMnutvUVh75Z8hkQjbPPZiMfwHoAdgCzc3cH4YRQ+GOG
1gWp3BEJSnktsWcMC4fc8AMOeTalmgAAAYBDdJAaAAAEAwBHMEUCIQC+vA3ur1Xn
TVXZH+Gpk0Rv9rNIJk9LcBcShGepVvBymAIgA4i0kWrK4Qohz9PRUCGwGTBT9dHF
ga2mnNLMn+i3fg4wDQYJKoZIhvcNAQELBQADggEBAIsaRxZjDIw7ecTsJ7956nPp
upkYcFnBvchJaS4pdcFjSHNv68RgWpxiN7YU8KP/X+lnWB1Xn0Tg7ljFonkC726B
L/ouMSKbUJ3gtrF4Wd/CfUuSO8Iv5Q4R97RiB4Q3ity8ui5+IFgoF9TEI8hebBbJ
EwQrDECxkju9buqV8/Ee1dcT6aIOtqoSNdqRwozFQwKviID6BR13G4m7SwzsCRIA
s4TC5rEPfi6l2k+TPrhWtbpBCoa7/R9ZhZuInbdGpS+EJF5sNmT7lqHv7ERq1s0C
irWT2P1o9Y7dBc9V/CLQakPVpI+XcNgABXw9CEPgdLKEAuC3WhpBdvWT4OGhUhs=
-----END CERTIFICATE-----
subject=C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, CN = graph.windows.net

issuer=C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-384, 384 bits
---
SSL handshake has read 6202 bytes and written 475 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: EA110000555828186BBB3B2D86B25FE9068288A55D9F7B67ADC77BFED08F1917
    Session-ID-ctx: 
    Master-Key: E3961F62AE45BB7A87A3983B683C6A8D047C2B8D252D99D39A2D298EC3DAB72C295AD9BFA2395814D8EC222DA02E4262
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1653422938
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---

@allanhung
Copy link
Author

I do the tcpdump. it seems doesn't get the OIDC documents from azure.
Screen Shot 2022-05-25 at 4 49 00 AM
But it's ok with curl command.

$ curl https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/v2.0/.well-known/openid-configuration
{"token_endpoint":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/oauth2/v2.0/token","token_endpoint_auth_methods_supported":["client_secret_post","private_key_jwt","client_secret_basic"],"jwks_uri":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/discovery/v2.0/keys","response_modes_supported":["query","fragment","form_post"],"subject_types_supported":["pairwise"],"id_token_signing_alg_values_supported":["RS256"],"response_types_supported":["code","id_token","code id_token","id_token token"],"scopes_supported":["openid","profile","email","offline_access"],"issuer":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/v2.0","request_uri_parameter_supported":false,"userinfo_endpoint":"https://graph.microsoft.com/oidc/userinfo","authorization_endpoint":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/oauth2/v2.0/authorize","device_authorization_endpoint":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/oauth2/v2.0/devicecode","http_logout_supported":true,"frontchannel_logout_supported":true,"end_session_endpoint":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/oauth2/v2.0/logout","claims_supported":["sub","iss","cloud_instance_name","cloud_instance_host_name","cloud_graph_host_name","msgraph_host","aud","exp","iat","auth_time","acr","nonce","preferred_username","name","tid","ver","at_hash","c_hash","email"],"kerberos_endpoint":"https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/kerberos","tenant_region_scope":"NA","cloud_instance_name":"microsoftonline.com","cloud_graph_host_name":"graph.windows.net","msgraph_host":"graph.microsoft.com","rbac_url":"https://pas.windows.net"}

@jamilbk
Copy link
Member

jamilbk commented May 25, 2022

@allanhung Hmmm. Strange. It's not clear from the crash which URL OpenID Connect is timing out for. Does this run successfully for you from your Firezone instance?

curl https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/discovery/v2.0/keys

@allanhung
Copy link
Author

@jamilbk Yes, it runs successfully.

@allanhung
Copy link
Author

@jamilbk I test with my dex and it works.

    -          "discovery_document_uri": "https://login.microsoftonline.com/77287fa5-1d4b-4908-8dc3-d9d20280f8be/v2.0/.well-known/openid-configuration",
    +          "discovery_document_uri": "https://dex.xxx.com/.well-known/openid-configuration",

Here is the tcpdump.
Screen Shot 2022-05-25 at 11 04 22 AM

@jamilbk
Copy link
Member

jamilbk commented May 25, 2022 •
edited

@allanhung Yeah, I wonder if there's something in your Azure OAuth App settings that could be causing this. For reference, here's our Azure OIDC config we use for development and testing:

{
  "azure": {
    "client_id": "<redacted>",
    "client_secret": "<redacted>",
    "discovery_document_uri": "https://login.microsoftonline.com/c3f15e43-d01d-4e2d-b0b6-b961f6e34239/v2.0/.well-known/openid-configuration",
    "redirect_uri": "http://localhost:4000/auth/oidc/azure/callback/",
    "response_type": "code",
    "scope": "openid email profile",
    "label": "Azure"
  }
}

Another thing we've seen cause intermittent timeouts is the WireGuard or interface MTU -- on some providers (GCP) you need to lower it to 1360 or lower. Is your instance egress traffic configured to route out through a WireGuard tunnel?

If you don't mind sharing the cloud/hosting provider your Firezone instance is running in, I'll see if I can spin up a test instance to try and replicate the issue.

@allanhung
Copy link
Author

@jamilbk I think it's nothing about Azure OAuth App settings since it didn't finish the tls handshake. My instance egress traffic configured to route out through eth0. Here is the route table:

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.20.1.253     0.0.0.0         UG    100    0        0 eth0
10.3.2.0        0.0.0.0         255.255.255.0   U     0      0        0 wg-firezone
10.20.1.0       0.0.0.0         255.255.255.0   U     100    0        0 eth0

Sure. How do i share the instance with you?
Appreciate it for your help.

@jamilbk
Copy link
Member

jamilbk commented May 26, 2022 •
edited

Which cloud / hosting provider is your Firezone instance running in? Azure as well?

It may be faster to debug over Slack -- would be you be able to join our Slack group and DM me there?

https://www.firezone.dev/slack

@allanhung
Copy link
Author

I have joined the slack channel. And my cloud provider is alicloud.

@jamilbk jamilbk mentioned this issue May 26, 2022
Closed
@jamilbk
Copy link
Member

jamilbk commented May 26, 2022

Hey @allanhung This should be fixed in #651 and will make it into the next release coming out sometime next week.

@allanhung
Copy link
Author

@jamilbk Will test it with next release. Thanks.

@jamilbk
Copy link
Member

jamilbk commented Jun 1, 2022

@allanhung So this ended up being a bug with Erlang/OTP 25. We're in the process of reverting to 24 and will cut a new release soon with the fix.

@jamilbk
Copy link
Member

jamilbk commented Jun 1, 2022

Fixed by #664

@jamilbk jamilbk closed this as completed Jun 1, 2022
@jamilbk jamilbk mentioned this issue Jun 9, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jamilbk @allanhung