[data-importer] Incorrect cookies cause OAuth2 authentication error

[tjmv]

Support guidelines

• I've read the support guidelines

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem
• ... it's definitely me, not you

Description

There are many issues about setting up the data importer. Unfortunately, none of them described the errors I was running into: Initial OAuth2 is successful and gets a valid response, but the validation function always fails with the error that it cannot connect to /api/v1/about (connection refused, wrong port on the public URL).

I did some digging and it turns out doValidate retrieves the base url from the secret manager, which first checks for the existence of a cookie named base_url (https://github.com/firefly-iii/data-importer/blob/develop/app/Http/Controllers/TokenController.php#L216). I completely reset everything (the data importer and cleared all cookies), then ran a private window of my browser and the cookie got set again. Deleting the cookie does make everything work.

My setup is basically this: #5381 , but I ran into a different issue.

Expected behaviour

Setting up the importer with a client id, vanity url and base url as described in the docs should get everything going.

Debug information

Docker setup.

FIREFLY_III_URL=http://firefly-iii:8080
VANITY_URL=https://firefly.mydomain
CLIENT_ID=41

OAuth2 callback settings in Firefly III:

https://fidi.mydomain/callback

Error presented on the app page:

The importer could not connect to Firefly III. Please remedy the error below first, and check out the documentation if necessary.

cURL error 7: Failed to connect to firefly.mydomain port 443: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://firefly.mydomain/api/v1/about

Debug trace:

fidi | [2022-01-07 13:01:42] local.INFO: The following configuration information was found:
fidi | [2022-01-07 13:01:42] local.INFO: Personal Access Token: "" (limited to 25 chars if present)
fidi | [2022-01-07 13:01:42] local.INFO: Client ID            : "41"
fidi | [2022-01-07 13:01:42] local.INFO: Base URL             : "http://firefly-iii:8080"
fidi | [2022-01-07 13:01:42] local.INFO: Vanity URL           : "https://firefly.mydomain"
fidi | [2022-01-07 13:01:42] local.DEBUG: Found client ID "41" + URL "http://firefly-iii:8080" in config, redirect to Firefly III for permission.
fidi | [2022-01-07 13:01:42] local.DEBUG: Now in App\Http\Controllers\TokenController::redirectForPermission(request, "http://firefly-iii:8080", "https://firefly.mydomain", 41)
fidi | [2022-01-07 13:01:42] local.DEBUG: Query parameters are {"client_id":41,"redirect_uri":"https://fidi.mydomain/callback","response_type":"code","scope":"","state":"-snip-","code_challenge":"-snip-","code_challenge_method":"S256"}
fidi | [2022-01-07 13:01:42] local.DEBUG: Now redirecting to "https://firefly.mydomain/oauth/authorize?" (params omitted)
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:42 +0100] "GET /token HTTP/1.1" 302 2809 "https://fidi.mydomain/" "browser identifier"
fidi | [2022-01-07 13:01:43] local.DEBUG: Now at App\Http\Controllers\TokenController::callback
fidi | [2022-01-07 13:01:43] local.DEBUG: State is valid!
fidi | [2022-01-07 13:01:43] local.DEBUG: Params for access token {"form_params":{"grant_type":"authorization_code","client_id":41,"redirect_uri":"https://fidi.mydomain/callback","code_verifier":"-snip-","code":"-snip-"}}
fidi | [2022-01-07 13:01:43] local.DEBUG: Will contact "http://firefly-iii:8080/oauth/token" for a token.
fidi | [2022-01-07 13:01:45] local.DEBUG: Response {"token_type":"Bearer","expires_in":1209599,"access_token":"-snip-"}
fidi | [2022-01-07 13:01:45] local.DEBUG: Saving base url: http://firefly-iii:8080    // debug stuff I inserted to make sure it does what I think it does
fidi | [2022-01-07 13:01:45] local.DEBUG: Saving vanity url: https://firefly.mydomain // same thing here, it actually saves this value as the base_url cookie value
fidi | [2022-01-07 13:01:45] local.DEBUG: Return redirect with cookies to "https://fidi.mydomain"
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:43 +0100] "GET /callback?code=-snip-&state=-snip- HTTP/1.1" 302 5479 "-" "browser identifier"
fidi | [2022-01-07 13:01:45] local.DEBUG: Now in App\Http\Controllers\IndexController::index
fidi | [2022-01-07 13:01:45] local.DEBUG: App\Services\Shared\Authentication\SecretManager::hasValidSecrets
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET / HTTP/1.1" 200 3174 "-" "browser identifier"
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /fa/css/all.min.css HTTP/1.1" 200 12972 "https://fidi.mydomain/" "browser identifier"
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /css/bootstrap-dark.min.css?version=0.5.0 HTTP/1.1" 200 31409 "https://fidi.mydomain/" "browser identifier"
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /favicon.ico HTTP/1.1" 200 224 "https://fidi.mydomain/" "browser identifier"
fidi | [2022-01-07 13:01:45] local.DEBUG: isReadyForBasicStep("service-validation")
fidi | [2022-01-07 13:01:45] local.DEBUG: Now at App\Http\Controllers\TokenController::doValidate
fidi | [2022-01-07 13:01:45] local.DEBUG: Connecting to Firefly III at https://firefly.mydomain
fidi | [2022-01-07 13:01:45] local.DEBUG: Now at App\Services\Nordigen\AuthenticationValidator::validate
fidi | [2022-01-07 13:01:45] local.DEBUG: No Nordigen ID in hasId(), will return config variable.
fidi | [2022-01-07 13:01:45] local.DEBUG: No Nordigen key in hasKey(), will return config variable.
fidi | [2022-01-07 13:01:45] local.DEBUG: isReadyForBasicStep("service-validation")
fidi | [2022-01-07 13:01:45] local.DEBUG: Now at App\Services\Spectre\AuthenticationValidator::validate
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /js/popper.min.js.map HTTP/1.1" 404 6803 "-" "browser identifier"
fidi | [2022-01-07 13:01:45] local.DEBUG: No Spectre App ID in hasAppId(), will return config variable.
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /validate/nordigen HTTP/1.1" 200 1081 "https://fidi.mydomain/" "browser identifier"
fidi | [2022-01-07 13:01:45] local.DEBUG: No Spectre secret in hasSecret(), will return config variable.
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /validate/spectre HTTP/1.1" 200 1081 "https://fidi.mydomain/" "browser identifier"
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /js/bootstrap.min.js.map HTTP/1.1" 200 55136 "-" "browser identifier"
fidi | [2022-01-07 13:01:45] local.ERROR: Could not connect to Firefly III: cURL error 7: Failed to connect to firefly.mydomain port 443: Connection refused (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://firefly.mydomain/api/v1/about
fidi | 172.18.0.2 - - [07/Jan/2022:13:01:45 +0100] "GET /token/validate HTTP/1.1" 200 1293 "https://fidi.mydomain/" "browser identifier"

Steps to reproduce

No response

Additional info

Vanity URL is saved at https://github.com/firefly-iii/data-importer/blob/develop/app/Http/Controllers/TokenController.php#L317

The constant is defined as base_url at https://github.com/firefly-iii/data-importer/blob/develop/app/Services/Shared/Authentication/SecretManager.php#L40

So the result is that a base_url cookie is set, causing the app to use the wrong base url to authenticate against internally.

[JC5]

Nice find, sloppy bug. Thanks, it'll be fixed in the new release!