S3: set_metadata does not set metadata remotely in the simplest case. #1007

Closed
acrefoot opened this Issue Sep 20, 2012 · 5 comments

Projects

None yet

4 participants

@acrefoot

S3: set_metadata doesn't actually update metadata unless you call one of the set_contents_from_* command.

It will keep it in the key that you have in memory, but if you restart python and get that key again, it won't have that metadata.

@garnaat
the boto project member

Metadata can only be set at the time the object is written to S3 or when it is copied. There really isn't anything boto can do about this.

@garnaat garnaat closed this Nov 14, 2012
@mfschwartz
the boto project member

Hi Mitch,

I don't think that's correct. You can do copy-in-the-cloud from an object to itself, specifying x-amz-copy-source to point to the origin object and x-amz-metadata-directive:REPLACE, and then setting headers (which has the effect of just updating the metadata/headers). For example, running the following command:

gsutil -d setmeta -h Content-Type:text/html s3://mfsbucket/abc

here's the HTTP that's sent:

send: 'PUT /mfsbucket/abc HTTP/1.1\r\nHost: s3.amazonaws.com\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nx-amz-storage-class: STANDARD\r\nUser-Agent: Boto/2.6.0-dev (darwin) gsutil/3.19_PRE (darwin)\r\nx-amz-copy-source: mfsbucket/abc\r\nDate: Wed, 14 Nov 2012 22:05:27 GMT\r\ncontent-type: text/html\r\nAuthorization: ....=\r\nx-amz-metadata-directive: REPLACE\r\n\r\n'

And after running the command, I verified that the Content-Type was indeed updated.

(I implemented this command for Google Cloud Storage, but it works for S3 as well.)

Mike

@garnaat
the boto project member

I agree that you can do that but I don't think you should do that every time someone changes a value in the local object, do you?

@mfschwartz
the boto project member
@rngadam rngadam referenced this issue in GoogleCloudPlatform/gsutil Apr 17, 2013
Closed

gsutil -d setmeta returns S3ResponseError #140

@xiongchiamiov

I agree that you can do that but I don't think you should do that every time someone changes a value in the local object, do you?

It would be nice, however, to have a save() method that does this on-demand, as with most ORMs. Right now, the lack of something like this or a disclaimer in the docs makes it seem like calling set_metadata() will, well, set the metadata on s3. This is particularly non-obvious because of the perceived symmetry between the various set_* methods.

I guess some additional documentation would be an easier solution, although save() would be rather convenient as a user. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment