New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the getCredentials and claimCredentials endpoint #54

Closed
wants to merge 4 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@walac

walac commented Jan 25, 2018

claimCrendentials is used by the worker to get its initial credentials. The worker is validated using its Instance Identity Document, alongside its PKCS7 signature.

The scopes given are the as those provided by aws-provisioner.

@djmitche Could you please validate those scopes, please? Do we need to add or remove any scope?

@walac walac requested a review from jhford Jan 25, 2018

if (!await validateIdentityDocument(sIdentDoc, signature)) {
return E();
}

This comment has been minimized.

@djmitche

djmitche Jan 25, 2018

Contributor

This is so cool :)

@djmitche

djmitche Jan 25, 2018

Contributor

This is so cool :)

This comment has been minimized.

@jhford

jhford Jan 26, 2018

Collaborator

agreed!

@jhford

jhford Jan 26, 2018

Collaborator

agreed!

Show outdated Hide outdated lib/api.js
scopes: [
`assume:worker-type:${provisionerId}/${instance.workerType}`,
'assume:worker-id:*',
],

This comment has been minimized.

@djmitche

djmitche Jan 25, 2018

Contributor

Yes, these scopes look good.

@djmitche

djmitche Jan 25, 2018

Contributor

Yes, these scopes look good.

This comment has been minimized.

@jhford

jhford Jan 26, 2018

Collaborator

I think at this point, we should also strongly consider issuing
assume:worker-id:$region:$instanceId in place of the star scopes here. Since this change is going to require a change to how workers work, this might be a good time to go through with that testing. Both because we can, and because we finally have understanding that instance ids will not be reused.

@jhford

jhford Jan 26, 2018

Collaborator

I think at this point, we should also strongly consider issuing
assume:worker-id:$region:$instanceId in place of the star scopes here. Since this change is going to require a change to how workers work, this might be a good time to go through with that testing. Both because we can, and because we finally have understanding that instance ids will not be reused.

This comment has been minimized.

@djmitche

djmitche Jan 26, 2018

Contributor

That would be so awesome!!

@djmitche

djmitche Jan 26, 2018

Contributor

That would be so awesome!!

Show outdated Hide outdated lib/identity-document.js
@jhford

Great work. The API is coming along really nicely. Let me know if I can help out with the forge stuff!

Show outdated Hide outdated lib/api.js
Show outdated Hide outdated lib/api.js
scopes: [
`assume:worker-type:${provisionerId}/${instance.workerType}`,
'assume:worker-id:*',
],

This comment has been minimized.

@jhford

jhford Jan 26, 2018

Collaborator

I think at this point, we should also strongly consider issuing
assume:worker-id:$region:$instanceId in place of the star scopes here. Since this change is going to require a change to how workers work, this might be a good time to go through with that testing. Both because we can, and because we finally have understanding that instance ids will not be reused.

@jhford

jhford Jan 26, 2018

Collaborator

I think at this point, we should also strongly consider issuing
assume:worker-id:$region:$instanceId in place of the star scopes here. Since this change is going to require a change to how workers work, this might be a good time to go through with that testing. Both because we can, and because we finally have understanding that instance ids will not be reused.

@@ -515,6 +521,62 @@ api.declare({
res.reply(prices);
});
api.declare({
method: 'delete',

This comment has been minimized.

@jhford

jhford Jan 26, 2018

Collaborator

In order to operate well with retries, etc, we should have a separate GET and DELETE endpoint here. The GET being used to obtain the credential and the DELETE is used to register receipt of the credentials. This is the same flow we have for provisioner secrets as currently implemented. Basically, the response handler running to completion without error is not a reliable signal for whether the worker type has received and understood the credential. We don't want a machine that fails to get its credential be unable to retry and have to shut down.

@jhford

jhford Jan 26, 2018

Collaborator

In order to operate well with retries, etc, we should have a separate GET and DELETE endpoint here. The GET being used to obtain the credential and the DELETE is used to register receipt of the credentials. This is the same flow we have for provisioner secrets as currently implemented. Basically, the response handler running to completion without error is not a reliable signal for whether the worker type has received and understood the credential. We don't want a machine that fails to get its credential be unable to retry and have to shut down.

Show outdated Hide outdated lib/identity-document.js
Show outdated Hide outdated package.json
Show outdated Hide outdated lib/identity-document.js
if (!await validateIdentityDocument(sIdentDoc, signature)) {
return E();
}

This comment has been minimized.

@jhford

jhford Jan 26, 2018

Collaborator

agreed!

@jhford

jhford Jan 26, 2018

Collaborator

agreed!

Show outdated Hide outdated lib/identity-document.js
Show outdated Hide outdated lib/identity-document.js

@jhford jhford changed the title from Add the claimCredentials endpoint to Add the getCredentials and removeCredentials endpoint Feb 9, 2018

@jhford

This comment has been minimized.

Show comment
Hide comment
@jhford

jhford Feb 15, 2018

Collaborator

@walac how are things going?

Collaborator

jhford commented Feb 15, 2018

@walac how are things going?

@walac

This comment has been minimized.

Show comment
Hide comment
@walac

walac Feb 15, 2018

@walac how are things going?

I am still busy with docker-worker. But before that, the only issue that remained was PKCS verification. I researched node crypto and forge and none of those implement PKCS verification, afaict.

walac commented Feb 15, 2018

@walac how are things going?

I am still busy with docker-worker. But before that, the only issue that remained was PKCS verification. I researched node crypto and forge and none of those implement PKCS verification, afaict.

@walac walac changed the title from Add the getCredentials and removeCredentials endpoint to Add the getCredentials and claimCredentials endpoint Feb 24, 2018

@walac

This comment has been minimized.

Show comment
Hide comment
@walac

walac Feb 24, 2018

@jhford I believe I fixed all the issues, could you please give another look at the patch?

walac commented Feb 24, 2018

@jhford I believe I fixed all the issues, could you please give another look at the patch?

@jhford

Great progress, looking really good. Thanks for looking into using Forge, it looks good so far.

Show outdated Hide outdated lib/identity-document.js
Show outdated Hide outdated lib/identity-document.js
Show outdated Hide outdated lib/api.js
Show outdated Hide outdated lib/api.js
Show outdated Hide outdated lib/identity-document.js
Show outdated Hide outdated test/api_test.js
Show outdated Hide outdated lib/api.js
Show outdated Hide outdated lib/api.js
Show outdated Hide outdated test/api_test.js
Show outdated Hide outdated test/api_test.js
@walac

This comment has been minimized.

Show comment
Hide comment
@walac

walac Feb 28, 2018

@jhford I think I fixed most (all?) of the comments. Regarding SHA256, I looked at it but it has no official documentation, but I know it is not PKCS7. Can we look at it in a follow-up PR?

One more thing: we currently depend on the node-forge master branch to make this work due to this and this.

walac commented Feb 28, 2018

@jhford I think I fixed most (all?) of the comments. Regarding SHA256, I looked at it but it has no official documentation, but I know it is not PKCS7. Can we look at it in a follow-up PR?

One more thing: we currently depend on the node-forge master branch to make this work due to this and this.

@jhford

This comment has been minimized.

Show comment
Hide comment
@jhford

jhford Feb 28, 2018

Collaborator

@jhford I think I fixed most (all?) of the comments. Regarding SHA256, I looked at it but it has no official documentation, but I know it is not PKCS7. Can we look at it in a follow-up PR?

There's an rsa2048 file in the instance metadata that is an RSA-SHA256 signature that we could use. My understanding is that PKCS7 is a file format for storing information. It's not inherently RSA or DSA, SHA1 or SHA256. I think they just used the /pkcs7 endpoint for whatever reason.

One more thing: we currently depend on the node-forge master branch to make this work due to this and this.

Good to know. Do you know when (if) they're going to ship those in a release?

Collaborator

jhford commented Feb 28, 2018

@jhford I think I fixed most (all?) of the comments. Regarding SHA256, I looked at it but it has no official documentation, but I know it is not PKCS7. Can we look at it in a follow-up PR?

There's an rsa2048 file in the instance metadata that is an RSA-SHA256 signature that we could use. My understanding is that PKCS7 is a file format for storing information. It's not inherently RSA or DSA, SHA1 or SHA256. I think they just used the /pkcs7 endpoint for whatever reason.

One more thing: we currently depend on the node-forge master branch to make this work due to this and this.

Good to know. Do you know when (if) they're going to ship those in a release?

@walac

This comment has been minimized.

Show comment
Hide comment
@walac

walac Mar 1, 2018

There's a rsa2048 file in the instance metadata that is an RSA-SHA256 signature that we could use

Just to make sure we are on the same page, are you talking about http://169.254.169.254/latest/dynamic/instance-identity/signature ?

walac commented Mar 1, 2018

There's a rsa2048 file in the instance metadata that is an RSA-SHA256 signature that we could use

Just to make sure we are on the same page, are you talking about http://169.254.169.254/latest/dynamic/instance-identity/signature ?

@jhford

This comment has been minimized.

Show comment
Hide comment
@jhford

jhford Jul 3, 2018

Collaborator

@walac just a heads up that the library for doing verification is up for review at taskcluster/iid-verify#1 when you have a chance :)

Collaborator

jhford commented Jul 3, 2018

@walac just a heads up that the library for doing verification is up for review at taskcluster/iid-verify#1 when you have a chance :)

@walac walac requested a review from jhford Aug 29, 2018

@jhford

It's looking like some good progress has been made. There's a couple issues that I think we should resolve before merging, but I think we're definitely in the right direction.

Show outdated Hide outdated test/api_test.js
Show outdated Hide outdated test/api_test.js
Show outdated Hide outdated test/api_test.js
await testErrorReturn(client.getCredentials, pkcs7TestData.invalid);
});
it('claimed successfully', async() => {

This comment has been minimized.

@jhford

jhford Aug 29, 2018

Collaborator

This test fails when I run it locally:

  1) Api credentials claimed successfully:
     Error: Failure to verify authorization
----
method:     getCredentials
errorCode:  AuthorizationFailed
statusCode: 403
time:       2018-08-29T17:26:45.582Z
      at node_modules/taskcluster-client/lib/client.js:327:21
      at node_modules/taskcluster-client/node_modules/promise/lib/core.js:33:15
      at flush (node_modules/taskcluster-client/node_modules/asap/asap.js:27:13)
      at process._tickCallback (internal/process/next_tick.js:61:11)
@jhford

jhford Aug 29, 2018

Collaborator

This test fails when I run it locally:

  1) Api credentials claimed successfully:
     Error: Failure to verify authorization
----
method:     getCredentials
errorCode:  AuthorizationFailed
statusCode: 403
time:       2018-08-29T17:26:45.582Z
      at node_modules/taskcluster-client/lib/client.js:327:21
      at node_modules/taskcluster-client/node_modules/promise/lib/core.js:33:15
      at flush (node_modules/taskcluster-client/node_modules/asap/asap.js:27:13)
      at process._tickCallback (internal/process/next_tick.js:61:11)
Show outdated Hide outdated lib/api.js
Show outdated Hide outdated package.json
Show resolved Hide resolved lib/state.js
@@ -406,6 +416,44 @@ class State {
return result.rowCount === 1;
}
async claimCredentials({region, id}, client) {

This comment has been minimized.

@jhford

jhford Aug 29, 2018

Collaborator

This function looks great!

@jhford

jhford Aug 29, 2018

Collaborator

This function looks great!

assert.equal(result.rowCount, 1, 'updating instance state had incorrect rowCount');
}
async canClaimCredentials({region, id}, client) {

This comment has been minimized.

@jhford

jhford Aug 29, 2018

Collaborator

Also, looking great!

@jhford

jhford Aug 29, 2018

Collaborator

Also, looking great!

Show outdated Hide outdated package.json

@walac walac requested a review from jhford Aug 29, 2018

@walac

This comment has been minimized.

Show comment
Hide comment
@walac

walac Aug 29, 2018

@jhford I believe I fixed all the problems, but Travis still fails and it feels like it is unrelated. Could you please confirm that?

walac commented Aug 29, 2018

@jhford I believe I fixed all the problems, but Travis still fails and it feels like it is unrelated. Could you please confirm that?

@jhford

This comment has been minimized.

Show comment
Hide comment
@jhford

jhford Aug 29, 2018

Collaborator

@walac I believe that the problem is there's something funky going on with the Postgres installation in travis. I haven't ever been able to reproduce that failure locally or in heroku, but it happens on occasion in travis. I will be adding extra logging to see if I can figure out what's going on.

Collaborator

jhford commented Aug 29, 2018

@walac I believe that the problem is there's something funky going on with the Postgres installation in travis. I haven't ever been able to reproduce that failure locally or in heroku, but it happens on occasion in travis. I will be adding extra logging to see if I can figure out what's going on.

Show outdated Hide outdated test/state_test.js
@jhford

This comment has been minimized.

Show comment
Hide comment
@jhford

jhford Aug 29, 2018

Collaborator

@walac I've taken another look and a lot of the concerns aren't addressed. Maybe we should hop on vidyo and chat about it? I can meet tomorrow at 16:30 CEST (Berlin) if you'd like.

Collaborator

jhford commented Aug 29, 2018

@walac I've taken another look and a lot of the concerns aren't addressed. Maybe we should hop on vidyo and chat about it? I can meet tomorrow at 16:30 CEST (Berlin) if you'd like.

@walac

This comment has been minimized.

Show comment
Hide comment
@walac

walac Aug 30, 2018

@jhford hah, I see a lot of your comments don't show up for me in the conversation tab.

walac commented Aug 30, 2018

@jhford hah, I see a lot of your comments don't show up for me in the conversation tab.

@walac walac requested a review from jhford Aug 30, 2018

@jhford

Thanks, most of the feedback is addressed. I think the only things left to do here are figure out the unreliable test and add the facilities for picking the correct public key, then we're ready to merge I think.

Show outdated Hide outdated lib/identity-document.js
Show outdated Hide outdated lib/state.js
Show outdated Hide outdated test/api_test.js

@walac walac requested a review from jhford Sep 3, 2018

walac added some commits Jan 22, 2018

Add the claimCredentials endpoint
claimCredentials receives a RSA2048 signature of the identity document
and returns the credentials for the worker.

The document is validated according using the iid-verify [1] package.

Informations about the instance, like instance-id and region
are extracted from the identity document.

The given instance must be in a running state, otherwise the endpoint
will fail.

As iid-verify requires node 10 or newer, we upgrade the required node
version, as well as packages that fail to build with this node version.

[1] https://www.npmjs.com/package/iid-verify

@walac walac closed this Oct 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment