Editing or Adding Taxons giving API Key Error

[doke]

In the backend when editing a taxon or trying to add a taxon to a product (or adding an option type) I am getting a 401 error from the server.

Solidus Version:
Solidus 2.8.2

To Reproduce
Products > Taxonomies > Edit

or

Product > Edit > Trying to Set Taxon or Option Type

Log from Server
I, [2019-03-12T17:59:56.695563 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Started GET "/api/taxonomies/1?set=nested" for 103.100.28.169 at 2019-03-12 17:59:56 +0800
I, [2019-03-12T17:59:56.697712 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Processing by Spree::Api::TaxonomiesController#show as JSON
I, [2019-03-12T17:59:56.698099 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Parameters: {"set"=>"nested", "id"=>"1"}
D, [2019-03-12T17:59:56.701539 #2380] DEBUG -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Spree::User Load (0.8ms) SELECT spree_users.* FROM spree_users WHERE spree_users.deleted_at IS NULL AND spree_users.spree_api_key = '' LIMIT 1
I, [2019-03-12T17:59:56.766073 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Rendering /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/errors/must_specify_api_key.json.jbuilder
I, [2019-03-12T17:59:56.770379 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Rendered /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/errors/must_specify_api_key.json.jbuilder (3.8ms)
I, [2019-03-12T17:59:56.771001 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Filter chain halted as :authenticate_user rendered or redirected
I, [2019-03-12T17:59:56.771340 #2380] INFO -- : [65539e80-f5d1-46c9-b467-cc9b7cec2393] Completed 401 Unauthorized in 73ms (Views: 68.4ms | ActiveRecord: 0.8ms)

I, [2019-03-12T18:57:34.726433 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Started GET "/api/option_types?q%5Bname_cont%5D=&=1552388147009" for 103.100.28.169 at 2019-03-12 18:57:34 +0800
I, [2019-03-12T18:57:34.728179 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Processing by Spree::Api::OptionTypesController#index as JSON
I, [2019-03-12T18:57:34.728435 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Parameters: {"q"=>{"name_cont"=>""}, ""=>"1552388147009"}
D, [2019-03-12T18:57:34.731072 #2572] DEBUG -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Spree::User Load (0.6ms) SELECT spree_users.* FROM spree_users WHERE spree_users.deleted_at IS NULL AND spree_users.spree_api_key = '' LIMIT 1
I, [2019-03-12T18:57:34.732287 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Rendering /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/errors/must_specify_api_key.json.jbuilder
I, [2019-03-12T18:57:34.732764 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Rendered /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/errors/must_specify_api_key.json.jbuilder (0.3ms)
I, [2019-03-12T18:57:34.733184 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Filter chain halted as :authenticate_user rendered or redirected
I, [2019-03-12T18:57:34.733474 #2572] INFO -- : [3884a9b8-cb4e-4b26-80f9-50dfcd0d574e] Completed 401 Unauthorized in 5ms (Views: 1.5ms | ActiveRecord: 0.6ms)

Additional context
Gemfile and Gemfile.lock here:
https://gist.github.com/doke/5313dce7bf013926e870696bcc16028c
https://gist.github.com/doke/4c9c80214c12fbd605c1f8bece20db76

[aitbw]

I'll be having a look at this one.

UPDATE: Hey @doke, just took a look at this issue and I can't reproduce it. Can you provide us with more details? Thanks!

[doke]

Sure, this is running in a staging environment in production. Looking at the server logs I can see that it is sometimes calling GET "/api/taxons?" with a token parameter and sometimes it is not. Please see the below gist for a more detailed log dump. This is the result of editing a product and then clicking in the taxon field, then clicking in the option types triggering the GET requests to populate the dropdowns.

https://gist.github.com/doke/5d57b9cc9e266e543c37ac10aca28b65

Request with an API key, returns ok:

Started GET "/api/taxons?per_page=50&page=1&without_children=true&q%5Bname_cont%5D=&token=e24aa8cacfaa8dd3308807568b21a4b582dfca834d424938&=1552433529651" for 49.255.167.97 at 2019-03-13 07:32:11 +0800
Processing by Spree::Api::TaxonsController#index as JSON
Parameters: {"per_page"=>"50", "page"=>"1", "without_children"=>"true", "q"=>{"name_cont"=>""}, "token"=>"e24aa8cacfaa8dd3308807568b21a4b582dfca834d424938", ""=>"1552433529651"}
Spree::User Load (0.7ms) SELECT spree_users.* FROM spree_users WHERE spree_users.deleted_at IS NULL AND spree_users.spree_api_key = 'e24aa8cacfaa8dd3308807568b21a4b582dfca834d424938' LIMIT 1
(3.8ms) SELECT spree_roles.name FROM spree_roles INNER JOIN spree_roles_users ON spree_roles.id = spree_roles_users.role_id WHERE spree_roles_users.user_id = 1
Spree::Role Load (0.5ms) SELECT spree_roles.* FROM spree_roles INNER JOIN spree_roles_users ON spree_roles.id = spree_roles_users.role_id WHERE spree_roles_users.user_id = 1
Spree::Taxon Load (2.4ms) SELECT spree_taxons.* FROM spree_taxons ORDER BY spree_taxons.taxonomy_id ASC, spree_taxons.lft ASC LIMIT 50 OFFSET 0
Spree::Taxon Load (5.2ms) SELECT spree_taxons.* FROM spree_taxons WHERE ((((((((((((spree_taxons.lft <= 1 AND spree_taxons.rgt >= 18 AND (spree_taxons.id != 1) OR spree_taxons.lft <= 2 AND spree_taxons.rgt >= 3 AND (spree_taxons.id != 3)) OR spree_taxons.lft <= 4 AND spree_taxons.rgt >= 5 AND (spree_taxons.id != 4)) OR spree_taxons.lft <= 6 AND spree_taxons.rgt >= 7 AND (spree_taxons.id != 5)) OR spree_taxons.lft <= 8 AND spree_taxons.rgt >= 9 AND (spree_taxons.id != 6)) OR spree_taxons.lft <= 10 AND spree_taxons.rgt >= 11 AND (spree_taxons.id != 8)) OR spree_taxons.lft <= 12 AND spree_taxons.rgt >= 13 AND (spree_taxons.id != 9)) OR spree_taxons.lft <= 14 AND spree_taxons.rgt >= 15 AND (spree_taxons.id != 14)) OR spree_taxons.lft <= 16 AND spree_taxons.rgt >= 17 AND (spree_taxons.id != 15)) OR spree_taxons.lft <= 19 AND spree_taxons.rgt >= 24 AND (spree_taxons.id != 11)) OR spree_taxons.lft <= 20 AND spree_taxons.rgt >= 21 AND (spree_taxons.id != 12)) OR spree_taxons.lft <= 22 AND spree_taxons.rgt >= 23 AND (spree_taxons.id != 13)) OR spree_taxons.lft <= 25 AND spree_taxons.rgt >= 26 AND (spree_taxons.id != 18)) ORDER BY spree_taxons.lft ASC
Rendering /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/taxons/index.json.jbuilder
(1.0ms) SELECT COUNT(*) FROM (SELECT 1 AS one FROM spree_taxons ORDER BY spree_taxons.taxonomy_id ASC, spree_taxons.lft ASC LIMIT 50 OFFSET 0) subquery_for_count
Rendered /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/shared/_pagination.json.jbuilder (1.8ms)
Rendered /var/www/staging/application-name/shared/bundle/ruby/2.3.0/gems/solidus_api-2.8.2/app/views/spree/api/taxons/index.json.jbuilder (4.4ms)
Completed 200 OK in 64ms (Views: 4.5ms | ActiveRecord: 13.7ms)

Request without API key, 401:

Started GET "/api/option_types?ids=1" for 49.255.167.97 at 2019-03-13 07:32:09 +0800
Processing by Spree::Api::OptionTypesController#index as JSON
Parameters: {"ids"=>"1"}
Spree::User Load (0.6ms) SELECT spree_users.* FROM spree_users WHERE spree_users.deleted_at IS NULL AND spree_users.spree_api_key = '' LIMIT 1
Rendering Rendered Filter chain halted as :authenticate_user rendered or redirected
Completed 401 Unauthorized in 8ms (Views: 1.4ms | ActiveRecord: 0.6ms)

This next gist shows editing a Taxon:

https://gist.github.com/doke/0d8d45be4af313f50a0b8b13dc9c5f39

You can see some GET requests to /api/taxons include the token (which return fine) and some do not (which 401). I can also see that it is attempting to do a user lookup without an api key:

Spree::User Load (1.2ms) SELECT spree_users.* FROM spree_users WHERE spree_users.deleted_at IS NULL AND spree_users.spree_api_key = '' LIMIT 1

I can reproduce this with Curl:

curl -X GET -H "Content-type: application/json" -H "Accept: application/json" "https://server/api/option_types?ids=1"
{"error":"You must specify an API key."}

With token:

curl -X GET -H "Content-type: application/json" -H "Accept: application/json" "https://server/api/option_types?ids=1&token=e24aa8cacfaa8dd3308807568b21a4b582dfca834d424938"
[{"id":1,"name":"Size","presentation":"Size","position":1,"option_values":[{"id":1,"name":"OS","presentation":"OS","option_type_name":"Size","option_type_id":1,"option_type_presentation":"Size"},{"id":2,"name":"XL","presentation":"XL","option_type_name":"Size","option_type_id":1,"option_type_presentation":"Size"},{"id":4,"name":"L","presentation":"L","option_type_name":"Size","option_type_id":1,"option_type_presentation":"Size"},{"id":5,"name":"M","presentation":"M","option_type_name":"Size","option_type_id":1,"option_type_presentation":"Size"},{"id":6,"name":"S","presentation":"S","option_type_name":"Size","option_type_id":1,"option_type_presentation":"Size"},{"id":3,"name":"XS","presentation":"XS","option_type_name":"Size","option_type_id":1,"option_type_presentation":"Size"}]}]

[doke]

I have resolved something here. I had the staging site behind an http basic auth (configured through nginx). Turning this off for the /api/ endpoint was not enough. Disabling it for the entire site combined with a browser history clear worked.

Issue still there though that some calls to /api/ are sending the api key and some are not.

Sorry if this started a goose chase! Perhaps the docs should reflect a warning about running behind basic auth. Thanks.

[jacobherrington]

@doke we should definitely look for opportunities to do that! If you want to open a PR that would be great, otherwise I'm sure we will get to it 😄

[MFRWDesign]

@kennyadsl Can we reopen this one? We just bumped into it today. The taxon selector on the Admin Product Edit page was showing up blank, with 2 400-errors in the console complaining about a bad request to GET /taxons?xyzabc etc.

We fixed it for now by turning off basic auth, but that's not ideal as robots.txt noindex directives are no longer allowed by google: https://searchengineland.com/google-to-stop-supporting-noindex-directive-in-robots-txt-319003

Maybe there's another suggestion? Our main reason for using basic auth is to prevent indexing.

[MFRWDesign]

@kennyadsl @jarednorman We're still experiencing this one. Can we reopen the issue? We're seeing it only when the solidus admin is behind basic auth. Here's a screenshot from today: https://monosnap.com/file/FjswyJSSKkYGCOGqAFhevUa9dnzsWx

Any ideas?

[tvdeyen]

Hey @MFRWDesign

We also ran into this today. And the issue is that an existing Authorization Header (ie. from Basic Auth) gets overwritten by Solidus with the move from a custom header in #3029

This one is tricky. We are trying to fix this by changing our NGINX conf to allow calls to /api/, if a Authorization: Bearer header is present.

[tvdeyen]

I am not sure that this is something Solidus should or even can fix.