Key Rotation Support #1165

danielsdeleo · 2015-11-21T17:55:38Z

Chef Server now supports multiple keys for a user with optional expiration. However I am only able to authenticate to supermarket using the default key. When I try to use any other key, I get this error when using knife-supermarket:

ERROR: Authentication failed due to an invalid public/private key pair. If you have changed your keys recently try logging out and logging back in to Supermarket.

My use case is that I want to publish cookbooks using delivery to a supermarket that is attached to a different Chef Server and I want to use the same key pair and user name so I have one less credential to manage. I am able to do this by setting the default key. However, enterprise users with key rotation requirements will not be able to seamlessly rotate keys for use with supermarket the same way they can with chef-client.

This is likely an issue with oc-id, feel free to move the issue there if appropriate.

The text was updated successfully, but these errors were encountered:

danielsdeleo · 2015-11-30T18:13:49Z

FYI, when authenticating API requests using signed header auth, Supermarket fetches the key via oc-id, but then it verifies the request on its own, here:

supermarket/app/controllers/api/v1/cookbook_uploads_controller.rb

Lines 158 to 202 in 0f0ee70

    
           # 
        
           # Finds a user specified in the request header or renders an error if 
        
           # the user doesn't exist. Then attempts to authorize the signed request 
        
           # against the users public key or renders an error if it fails. 
        
           # 
        
           def authenticate_user! 
        
             username = request.headers['X-Ops-Userid'] 
        
             user = Account.for('chef_oauth2').where(username: username).first.try(:user) 
        
             unless user 
        
               return error( 
        
                 { 
        
                   error_code: t('api.error_codes.authentication_failed'), 
        
                   error_messages: [t('api.error_messages.invalid_username', username: username)] 
        
                 }, 
        
                 401 
        
               ) 
        
             end 
        
             if user.public_key.nil? 
        
               return error( 
        
                 { 
        
                   error_code: t('api.error_codes.authentication_failed'), 
        
                   error_messages: [t('api.error_messages.missing_public_key_error', current_host: request.base_url)] 
        
                 }, 
        
                 401 
        
               ) 
        
             end 
        
             auth = Mixlib::Authentication::SignatureVerification.new.authenticate_user_request( 
        
               request, 
        
               OpenSSL::PKey::RSA.new(user.public_key) 
        
             ) 
        
             if auth 
        
               @current_user = user 
        
             else 
        
               error( 
        
                 { 
        
                   error_code: t('api.error_codes.authentication_failed'), 
        
                   error_messages: [t('api.error_messages.authentication_key_error')] 
        
                 }, 
        
                 401 
        
               ) 
        
             end

I'm not sure exactly what process erchef follows when there are multiple keys, but it's likely that the code I referenced would need to be updated for multi-key.

dfduarte · 2016-01-05T12:58:59Z

Good morning,

I'm facing a similar issue here, when using a self-hosted chef-enterprise and Supermarket in the same scenario.

Here is the output from knife command:

HTEG3QC:cookbooks dduarte$ knife supermarket share test2 -VV
INFO: Using configuration from /Users/dduarte/Downloads/chef-repo 2/.chef/knife.rb
INFO: Validating ruby files
DEBUG: Testing /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/metadata.rb for syntax errors...
DEBUG: Testing /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/recipes/default.rb for syntax errors...
DEBUG: Testing /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/spec/spec_helper.rb for syntax errors...
DEBUG: Testing /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/spec/unit/recipes/default_spec.rb for syntax errors...
DEBUG: Testing /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/test/integration/default/serverspec/default_spec.rb for syntax errors...
DEBUG: Testing /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/test/integration/helpers/serverspec/spec_helper.rb for syntax errors...
INFO: Validating templates
INFO: Syntax OK
DEBUG: Staging at /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/recipes/default.rb to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/recipes/default.rb
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/.kitchen.yml to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/.kitchen.yml
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/Berksfile to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/Berksfile
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/Berksfile.lock to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/Berksfile.lock
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/chefignore to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/chefignore
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/Gemfile to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/Gemfile
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/metadata.rb to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/metadata.rb
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/README.md to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/README.md
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/Thorfile to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/Thorfile
DEBUG: Staging /Users/dduarte/Downloads/chef-repo 2/cookbooks/test2/Vagrantfile to /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/Vagrantfile
DEBUG: Generating metadata
Generating metadata for test2 from /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/metadata.rb
DEBUG: Generated /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h/test2/metadata.json
DEBUG: Temp cookbook directory is "/var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/chef-test2-build20160105-23012-1tgnj1h"
Making tarball test2.tgz
DEBUG: Signing: method: post, url: https://supermarket.xxxxxxxxxxx.com/api/v1/cookbooks, file: #<File:0x007ffe4cfa88e8>, User-id: dduarte, Timestamp: 2016-01-05T12:52:23Z
ERROR: Authentication failed due to an invalid public/private key pair. If you have changed your keys recently try logging out and logging back in to Supermarket.
HTEG3QC:cookbooks dduarte$

nellshamrell · 2016-01-20T23:28:06Z

Hi @dfduarte, are you still seeing that error? Is that occurring only when you use a certain key, or all the time?

danielsdeleo · 2016-01-21T02:32:55Z

The code I referenced didn't change, so I'm pretty sure the behavior will be the same (unless of course I'm wrong about that section of code being the important thing). What I found was that the 'default' key works like you would expect, and any other key will always fail. To replicate this, you can add a second key to your user with knife user key create (use a name other than default). You should be able to use this key with any other knife command that hits the Chef Server, but it won't work for supermarket.

As far as I can tell, the Chef Server fetches all the users keys and then loops through them to authenticate a request. I think to emulate this for supermarket, you'd have to teach oc-id to give you all the public keys for a user, then update the API authentication part of supermarket to loop over them all.

danielsdeleo · 2016-01-21T17:59:00Z

BTW, here's the erchef code for validating requests. When there are multiple keys, it takes the path that goes to validate_sigs (plural).

https://github.com/chef/chef_authn/blob/defae88f9ea9f9875b1148947bbdf38c867c8ec7/src/chef_authn.erl#L561-L662

dfduarte · 2016-01-21T18:04:25Z

@nellshamrell Hello Nell,

Yep, I'm still having this problem. But for some reason, at least here, I've figured out that a logoff-logon (using the /id OAUTH URL) in Supermarket panel does the job and solves the problem. It happened all the time.

Notice: I've updated for the latest 2.3 version of Supermarket, and I'm using the latest chef-server version (both stable, of course). I've not tested with all of chef-server users yet, but I'm pretty sure that this will solve for the rest.

@danielsdeleo Good point. I'll try if I face problem again...

nellshamrell · 2016-01-21T18:30:28Z

@dfduarte Yep, in order to upload to Supermarket using your key, you need to log in to Supermarket first. This is what creates a connection between a Supermarket account and the Chef Server, and that connection is needed to use your Chef Server key to upload to Supermarket.

jasonwbarnett · 2016-01-31T14:29:38Z

I'm having issues myself (not sure if it's related to this or not). I went to https://id.chef.io/id/profile and clicked "Get a New Key" and then cleared all cookies, re-logged in and tried the following command and it keeps failing.

[2016/01/31 9:22:15] user@host ~/cookbooks/cacookbook (master)$ knife cookbook site share cacookbook Other --user jasonwbarnett --key ~/.chef/jasonwbarnett.pem
Generating metadata for cacookbook from /var/folders/tc/20xb0t9x4xb5k8jqr_7gj_d40000gn/T/chef-ca-certificates-build20160131-70195-vv1b9/cacookbook/metadata.rb
Making tarball cacookbook.tgz
ERROR: Authentication failed due to an invalid public/private key pair. If you have changed your keys recently try logging out and logging back in to Supermarket.

dfduarte · 2016-02-01T13:59:09Z

@jasonwbarnett Is your OAUTH correctly configured? The id.chef.io is the OAUTH for the chef-server from Chef, and not for the private.

Yep, I did this same mistake when I configured Supermarket some months ago. Just configure your supermarket.rb properly, and everything will be ok =).

jasonwbarnett · 2016-02-01T15:35:39Z

@dfduarte I am stupid... :(

I literally followed the supermarket instructions to the T and it still isn't working. Can you break down what you said, I didn't fully grasp it.

The credentials to authenticate with Supermarket are the same ones used to authenticate with the Chef server. E.g. if you wish to upload cookbooks to supermarket.chef.io you need to use the same username and client key that you use to authenticate with manage.chef.io.

If you do not already have your keys, or don't know if you do, you can reset your key here:

Login or create an account at: www.chef.io/account/login
Click on the "Password and Key" tab (www.chef.io/account/password)
Click on Get a New Key. This will reset your client key.
Use this key to configure knife to authenticate with the public Chef server as you normally would.
You should now be configured to authenticate with Supermarket.

dfduarte · 2016-02-01T15:43:58Z

@jasonwbarnett Chill man, no problem =),

Well, I think supermarket.rb is not well documented (at least, isn`t intuitive), and some novice users can have some problems when setting a new private instalation. In short, you need:

A Chef-Server, configured using that oauth directives (setting the URL, and generating a key for chef-server.rb).
A Supermarket Server configured using supermarket.rb.

For the second one, you have 2 options: Using the (buggy) omnibus Chef cookbook to configure Supermarket, or grab the latest deb/rpm directly from repository: https://packagecloud.io/chef/stable/. On this option, you will need to set up supermarket.rb manually.

Head to /etc/opscode/supermarket.rb (Im not sure whether this path is correct now. Please check it later), and uncomment and configure the rbs parameters. The file is self-explanatory.

Remember that: When reaching the OAUTH section (with user id, token and OAUTH URL), put the configuration from YOUR private Chef server setup, and overwrite all default configurations there.

So, just run a reconfigure and a restart and here you go.

dfduarte · 2016-02-01T15:47:16Z

Btw: The credentials mentioned above, is for using your chef-server account with your (or public) Supermarket, I -guess-...

You don't need generating a new user given that Supermarket uses the user database from chef-server throught oauth.

jasonwbarnett · 2016-02-01T15:47:59Z

I'm sorry... but this makes little sense to me. When I want to share a cookbook from my local dev machine to the public supermarket, why do I have to configure my private (or public) chef server?

dfduarte · 2016-02-01T15:50:18Z

@jasonwbarnett Oh, I guess I got your problem now.

You have to set at least a account on public chef-server, and login with it on Supermarket.

Try this: https://www.chef.io/blog/2015/03/16/using-chef-supermarket-a-guided-tour/ I think this will anwser your questions. Read from "upload to supermarket" section.

Remeber: When setting a new Starter kit, you need to logoff and logon again from your Supermarket account, because supermarket need to generate a new session key based on your new credentals.

jasonwbarnett · 2016-02-01T15:59:07Z

@dfduarte I followed that a couple of days ago and it did not work. I also tried stove without success either. TBC, I've shared a cookbook in the past to the supermarket with success, but it took some finagling last time too.

The documentation for sharing cookbooks is below the bar in my opinion.

robbkidd · 2016-02-01T16:12:16Z

@jasonwbarnett Which documentation are you using? The blog post @dfduarte mentioned? I would like to raise this documentation to at last at the bar if not above it.

jasonwbarnett · 2016-02-01T23:53:30Z

@robbkidd I leveraged multiple places for documentation.

The actual supermarket site. I don't think it's actually documented anywhere on the supermarket site which seems odd. It's really difficult (in my opinion, open to being wrong) to simply share/push a cookbook to the supermarket.
The blog post. I found the article via google.
the general knife.rb and supermarket.rb documentation.

nellshamrell · 2016-02-02T17:06:59Z

@jasonwbarnett: what does your knife.rb file look like?

jasonwbarnett · 2016-02-04T00:49:59Z

@nellshamrell

current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "jasonwbarnett"
client_key               "#{current_dir}/jasonwbarnett.pem"
validation_client_name   "jasonwbarnett-validator"
validation_key           "#{current_dir}/jasonwbarnett-validator.pem"
chef_server_url          "https://api.chef.io/organizations/jasonwbarnett"
cookbook_path            ["#{current_dir}/../cookbooks"]

jasonwbarnett · 2016-02-04T18:30:50Z

@nellshamrell it's working now... sigh

nellshamrell · 2016-02-04T18:40:19Z

Did anything change?

nellshamrell · 2016-02-04T18:40:41Z

I mean, did you make some sort of change and it started working, or did it seem to start working on its own?

jasonwbarnett · 2016-02-04T18:55:49Z

Signed up for managed chef.
Downloaded starter kit and put .chef on my local at ~/.chef
signed out of supermarket.
signed back into supermarket.
knife cookbook site share ca-certificates Other

dfduarte · 2016-02-04T18:57:28Z

Hey @jasonwbarnett Take a look on this: https://github.com/chef/knife-supermarket

This gem turns the things easier when working with knife and Supermarket. I forgot about this plugin before, sorry XD.

Anyway, nice to hear from you that you already has solved the problem.

tonyvilliams · 2016-05-31T13:46:08Z

Hi.. setting up a private chef-supermarket. ran into same issue. Exhausted on all steps. pl advise.

ERROR: Authentication failed due to an invalid public/private key pair. If you have changed your keys recently try logging out and logging back in to Supermarket.

tonyvilliams · 2016-06-01T11:47:12Z

fixed the issue. issue was due to time drift between the chef server and supermarket. configured ntp.

biox · 2016-12-27T23:28:32Z

@tonyvilliams Can't thank you enough for posting your solution here, saved me potential hours of pointless troubleshooting. Fracking ntp.

jasonwbarnett · 2018-01-25T11:35:18Z

I just want to note here that 2 years later I ran into the whole fiasco again and thank God I documented the solution here. I can't imagine I'm the only person who struggles with getting authentication to Supermarket up and running (thinking of one-off, not full time chefs). Is it really a requirement to have a chef manage account to work with Supermarket?

robbkidd · 2018-01-25T13:23:59Z

Since its beginning and for the foreseeable future, public Supermarket authenticates users with Hosted Chef (the Chef-as-a-Service, what I think you mean by "chef manage account"). This requirement is unlikely to change given the engineering and community users' effort that would be required to alter public Supermarket's user identity provider.

I'll take this as a task to review the current public Supermarket documentation and try to make the Hosted Chef requirement and knife configuration more clear.

jasonwbarnett · 2018-01-25T13:28:38Z

@robbkidd That makes sense. I hadn't even made the connection that kitchen was leveraging Hosted Chef for auth. I thought it was just some magical wiring in the backend :). Thank you for your commitment to Chef as a whole. Really grateful for the excellent engagement.

robbkidd · 2018-01-25T13:52:10Z

@jasonwbarnett ETOOMUCHMAGIC maybe. Thanks for coming back and keeping me honest about the documentation improvements I mentioned two years ago. 😊

nellshamrell self-assigned this Nov 23, 2015

nellshamrell added ready_for_analysis and removed ready_for_analysis labels Dec 14, 2015

jasonwbarnett mentioned this issue Jan 31, 2016

Upload to supermarket? jasonwbarnett/ca-certificates-cookbook#1

Closed

robbkidd mentioned this issue Dec 28, 2016

add Supermarket requirement for syncing server system clocks chef/chef-web-docs#363

Merged

nellshamrell removed their assignment May 5, 2020

tas50 added this to the Stuff we could do milestone Sep 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Key Rotation Support #1165

Key Rotation Support #1165

danielsdeleo commented Nov 21, 2015

danielsdeleo commented Nov 30, 2015

dfduarte commented Jan 5, 2016

nellshamrell commented Jan 20, 2016

danielsdeleo commented Jan 21, 2016

danielsdeleo commented Jan 21, 2016

dfduarte commented Jan 21, 2016

nellshamrell commented Jan 21, 2016

jasonwbarnett commented Jan 31, 2016

dfduarte commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

dfduarte commented Feb 1, 2016

dfduarte commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

dfduarte commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

robbkidd commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

nellshamrell commented Feb 2, 2016

jasonwbarnett commented Feb 4, 2016

jasonwbarnett commented Feb 4, 2016

nellshamrell commented Feb 4, 2016

nellshamrell commented Feb 4, 2016

jasonwbarnett commented Feb 4, 2016

dfduarte commented Feb 4, 2016

tonyvilliams commented May 31, 2016

tonyvilliams commented Jun 1, 2016

biox commented Dec 27, 2016 •

edited

jasonwbarnett commented Jan 25, 2018

robbkidd commented Jan 25, 2018

jasonwbarnett commented Jan 25, 2018

robbkidd commented Jan 25, 2018

Key Rotation Support #1165

Key Rotation Support #1165

Comments

danielsdeleo commented Nov 21, 2015

danielsdeleo commented Nov 30, 2015

dfduarte commented Jan 5, 2016

nellshamrell commented Jan 20, 2016

danielsdeleo commented Jan 21, 2016

danielsdeleo commented Jan 21, 2016

dfduarte commented Jan 21, 2016

nellshamrell commented Jan 21, 2016

jasonwbarnett commented Jan 31, 2016

dfduarte commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

dfduarte commented Feb 1, 2016

dfduarte commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

dfduarte commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

robbkidd commented Feb 1, 2016

jasonwbarnett commented Feb 1, 2016

nellshamrell commented Feb 2, 2016

jasonwbarnett commented Feb 4, 2016

jasonwbarnett commented Feb 4, 2016

nellshamrell commented Feb 4, 2016

nellshamrell commented Feb 4, 2016

jasonwbarnett commented Feb 4, 2016

dfduarte commented Feb 4, 2016

tonyvilliams commented May 31, 2016

tonyvilliams commented Jun 1, 2016

biox commented Dec 27, 2016 • edited

jasonwbarnett commented Jan 25, 2018

robbkidd commented Jan 25, 2018

jasonwbarnett commented Jan 25, 2018

robbkidd commented Jan 25, 2018

biox commented Dec 27, 2016 •

edited