NotImplemented - A header you provided implies functionality that is not implemented #15

Closed
tristanls opened this Issue Dec 23, 2012 · 16 comments

Comments

Projects
None yet
2 participants
Contributor

tristanls commented Dec 23, 2012

I am consistently receiving a NotImplemented error when trying to write a simple text file to S3 using the sdk.

var AWS = require( 'aws-sdk' );

AWS.config.update( { 
  accessKeyId : 'ACCESS_KEY_ID', 
  secretAccessKey : 'SECRET_ACCESS_KEY' 
});

var s3 = new AWS.S3({ endpoint : 'https://s3.amazonaws.com' });

s3.client.putObject({
  ACL : 'private',
  Body : 'some string',
  Bucket : 'my-bucket',
  Key : 'some/path/here/stuff.txt',
  ServerSideEncryption : 'AES256'
}, function ( error, response ) {
 // error listed below
});

results in the following error:

{
  "code":"NotImplemented",
  "message":"A header you provided implies functionality that is not implemented",
  "statusCode":501,
  "retryable":true
}

which, in itself seems weird considering it is NotImplemented but is retryable.

Contributor

lsegal commented Dec 23, 2012

I can't reproduce this error on my side. I am getting a successful result when I put an object with that set of values.

The note about NotImplemented is valid, though. We don't (yet) have special rules for handling this error code-- the current logic is to simply retry 500 errors. This can be fixed.

Can you provide any more data about how you are getting this error? Probably unhelpful would be your node version and aws-sdk version (log AWS.VERSION)-- perhaps a little more helpful could be output from npm list.

Contributor

tristanls commented Dec 23, 2012

node: 0.8.16
AWS.VERSION: 0.9.1-pre.2
npm list:

├─┬ aws-sdk@0.9.1-pre.2
│ ├─┬ xml2js@0.2.2
│ │ └── sax@0.4.2
│ └── xmlbuilder@0.4.2

Another condition necessary for failure is multiple processes ( say 4 ) running on the same machine all making the putObject call within a short period of time:

2012-12-23T20:15:05.010Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-23T20:15:05.013Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-23T20:15:06.284Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-23T20:15:06.431Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}

I can confirm that a manually triggered individual call succeeds.

It's a somewhat arcane failure case. Those putObject() calls are created using setInterval() (30 seconds in this particular failure case).

I will continue digging and try to provide more information.

Contributor

lsegal commented Dec 24, 2012

Seems like one of the headers is invalid, try printing the results of response.httpRequest.headers in your callback.

Contributor

tristanls commented Dec 24, 2012

response.httpRequest.headers does not exist / is empty (I'm displaying it via DEBUG below after transport response)

2012-12-24T21:19:34.312Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.314Z DEBUG transport response 
2012-12-24T21:19:34.314Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.588Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.588Z DEBUG transport response 
2012-12-24T21:19:34.588Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.981Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:19:34.981Z DEBUG transport response 

I'll keep looking.

Contributor

tristanls commented Dec 24, 2012

Ok, I have a contrast between successful and failed operations. Well, there's really no contrast, looks the same to me.

The _endpoint property is for my internal use.

successful operation

2012-12-24T21:28:48.811Z DEBUG AWS.config {"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false}
2012-12-24T21:28:48.811Z DEBUG s3 {"client":{"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}},"_endpoint":"https://s3.amazonaws.com"}
2012-12-24T21:28:48.812Z DEBUG s3.client {"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}}
2012-12-24T21:28:48.954Z DEBUG whole response {"ServerSideEncryption":"AES256","ETag":"\"ETAG_VALUE_HERE\"","RequestId":"56E04C735CB16D95"}
2012-12-24T21:28:48.955Z ERROR transport error 
2012-12-24T21:28:48.955Z DEBUG transport response.httpRequest.headers

failed operation

2012-12-24T21:29:19.725Z DEBUG AWS.config {"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false}
2012-12-24T21:29:19.725Z DEBUG s3 {"client":{"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}},"_endpoint":"https://s3.amazonaws.com"}
2012-12-24T21:29:19.725Z DEBUG s3.client {"serviceName":"s3","config":{"credentials":{"accessKeyId":"ACCESS_KEY_ID","secretAccessKey":"SECRET_ACCESS_KEY"},"maxRetries":3,"sslEnabled":true,"s3ForcePathStyle":false,"keys":{"sslEnabled":true,"s3ForcePathStyle":false},"endpoint":"https://s3.amazonaws.com"},"endpoint":{"protocol":"https:","slashes":true,"host":"s3.amazonaws.com","hostname":"s3.amazonaws.com","href":"https://s3.amazonaws.com/","pathname":"/","path":"/","port":443}}
2012-12-24T21:29:20.891Z DEBUG whole response 
2012-12-24T21:29:20.891Z ERROR transport error {"code":"NotImplemented","message":"A header you provided implies functionality that is not implemented","statusCode":501,"retryable":true}
2012-12-24T21:29:20.891Z DEBUG transport response.httpRequest.headers
Contributor

tristanls commented Dec 24, 2012

I have a hint. If Body is an empty string, that would probably result in Content-Length header not being created, which is probably the culprit for NotImplemented error.

Will verify shortly.

Contributor

lsegal commented Dec 24, 2012

That would be an easy fix if true-- but this might be a symptom of a larger problem. Is there a reason why your requests would have intermittent empty bodies? Or were some of the files being uploaded empty?

Contributor

tristanls commented Dec 24, 2012

Confirmed, Body = '' // empty string will result in the errors I've reported.


Some files were being uploaded empty.

Allowing uploading empty files is useful from the perspective of having a "log heartbeat" if you will. Say, I want to dump activity/log data into s3 at a set time interval. If I have no data, the presence of an empty file still lets me know that the reporting agent is still alive (i.e. heartbeat).

Contributor

lsegal commented Dec 24, 2012

Supporting empty files doesn't need justifying, I was just confirming that there was no other issue at play here. Thanks for looking into this and confirming, this is an easy fix, see: https://github.com/aws/aws-sdk-js/blob/master/lib/rest_xml_client.js#L69-L72

Contributor

lsegal commented Dec 24, 2012

Thanks for getting to this, I was just about to commit my own-- there are some extra things to adjust in addition to that patch though, which I am dealing with now :)

Contributor

tristanls commented Dec 24, 2012

Ah, sorry for stepping on you :D . You're more familiar with the code base, so I'm sure I missed a bunch. Thanks!

Contributor

lsegal commented Dec 24, 2012

You only missed a minor change, actually.

lsegal added a commit that referenced this issue Dec 24, 2012

Contributor

lsegal commented Dec 24, 2012

(pulling a valid body back out in the same scenario)

Contributor

tristanls commented Dec 24, 2012

Interesting, I'll have to take a look at what's going on there later on. Going offline. Thanks again, cheers!

Contributor

lsegal commented Dec 24, 2012

The failure scenario with that change is expressed in a test added in the commit prior to that one. The problem is raised if you make assumptions about the data['Body'] value being a String object (like, calling .length on it). This matters, since that body should actually be a 0-length file, so it should be a string, not null.

Contributor

lsegal commented Dec 27, 2012

This can be closed, as it was fixed by #16.

@lsegal lsegal closed this Dec 27, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment