put block list #31

Closed
gabpaquin opened this Issue Mar 27, 2013 · 9 comments

Comments

Projects
None yet
3 participants
@gabpaquin
Contributor

gabpaquin commented Mar 27, 2013

Hi Johnny,

It took while to find this one (I was not looking in the good direction) but I just realized that the headers for put blob and put block list are slightly different regarding the content type.

Put block list :
http://msdn.microsoft.com/en-us/library/windowsazure/dd179467.aspx, it is mention that to set the content type we need to set the x-ms-blob-content-type in the header.

This is different from put blob (http://msdn.microsoft.com/en-us/library/windowsazure/dd179451.aspx) where we need to set the Content type through the Content-Type properties in the header.

So I had to set the default_headers to the following in order to work in the put_block_list:

         default_headers = {"Content-Type" => content_type, "x-ms-blob-content-type" =>        content_type, :x_ms_version => "2011-08-18"}

However I still don't understand why if I go through the put_blob method i am getting a 403 rest:client error (even though my content is less than 4 mb). Any hint on this one?

Thanks a lot your gem is quite useful!

Gabriel

@johnnyhalife

This comment has been minimized.

Show comment Hide comment
@johnnyhalife

johnnyhalife Mar 27, 2013

Owner

I think @smarx is the right person to answer here as he is the one that put that together. I don't remember exactly how it worked.

Owner

johnnyhalife commented Mar 27, 2013

I think @smarx is the right person to answer here as he is the one that put that together. I don't remember exactly how it worked.

@smarx

This comment has been minimized.

Show comment Hide comment
@smarx

smarx Mar 27, 2013

Contributor

FYI, it's safe to just always use x-ms-blob-content-type, since that header works for Put Blob too.

As to why you're getting a 403 when calling put_blob, I have no idea. I haven't touched put_blob. Are you saying that you get that error using an unmodified version of this librar? What is the body of the 403 response?

Contributor

smarx commented Mar 27, 2013

FYI, it's safe to just always use x-ms-blob-content-type, since that header works for Put Blob too.

As to why you're getting a 403 when calling put_blob, I have no idea. I haven't touched put_blob. Are you saying that you get that error using an unmodified version of this librar? What is the body of the 403 response?

@gabpaquin

This comment has been minimized.

Show comment Hide comment
@gabpaquin

gabpaquin Mar 28, 2013

Contributor

Ok thanks a lot for your input I really appreciate.

so regarding x-ms-blob-content-type what is the best approach? Should we put it into the default_headers like suggested into my first comment?

Then the 403 with put_blob, it is with an unmodified version of this library. If it is the payload is a simple string I don't have any problem. However if the payload is more complex like File.open(/path/to/file) then I am getting an error. Any ideas, I am thinking that is probably related to the Authentification signature, but I can't tell how it is affected. Any thoughts?

Contributor

gabpaquin commented Mar 28, 2013

Ok thanks a lot for your input I really appreciate.

so regarding x-ms-blob-content-type what is the best approach? Should we put it into the default_headers like suggested into my first comment?

Then the 403 with put_blob, it is with an unmodified version of this library. If it is the payload is a simple string I don't have any problem. However if the payload is more complex like File.open(/path/to/file) then I am getting an error. Any ideas, I am thinking that is probably related to the Authentification signature, but I can't tell how it is affected. Any thoughts?

@smarx

This comment has been minimized.

Show comment Hide comment
@smarx

smarx Mar 28, 2013

Contributor

I would say that x-ms-blob-content-type should go in the default_headers list, and Content-Type should not.

As to your put_blob error, perhaps you can share the exact code you're using? I wouldn't expect passing File.open('/path/to/file') to work, but File.open('/path/to/file').read should. (Really, we're now talking about two different issues. Perhaps this put_blob issue should be logged separately with sample code.)

Contributor

smarx commented Mar 28, 2013

I would say that x-ms-blob-content-type should go in the default_headers list, and Content-Type should not.

As to your put_blob error, perhaps you can share the exact code you're using? I wouldn't expect passing File.open('/path/to/file') to work, but File.open('/path/to/file').read should. (Really, we're now talking about two different issues. Perhaps this put_blob issue should be logged separately with sample code.)

@gabpaquin

This comment has been minimized.

Show comment Hide comment
@gabpaquin

gabpaquin Mar 28, 2013

Contributor

Ok thanks for the quick feedback!

During my test, the only way I can get it to work was if there was both x-ms-blob-content-type and Content-Type where in the default_headers. Otherwise I was getting a 403 error. Here is the exact code:

test=File.open("/Users/gabriel/code/npathway/r8_image/2005_May_second/p13a.png")
my_container = WAZ::Blobs::Container::find("images")
my_container.upload("2005_may_p13a",test,"image/png")

Sorry about the confusion regarding the put_blob method, I will open a different issues with the exact code.

thanks.

Contributor

gabpaquin commented Mar 28, 2013

Ok thanks for the quick feedback!

During my test, the only way I can get it to work was if there was both x-ms-blob-content-type and Content-Type where in the default_headers. Otherwise I was getting a 403 error. Here is the exact code:

test=File.open("/Users/gabriel/code/npathway/r8_image/2005_May_second/p13a.png")
my_container = WAZ::Blobs::Container::find("images")
my_container.upload("2005_may_p13a",test,"image/png")

Sorry about the confusion regarding the put_blob method, I will open a different issues with the exact code.

thanks.

@smarx

This comment has been minimized.

Show comment Hide comment
@smarx

smarx Mar 28, 2013

Contributor

Perhaps a Content-Type header with the value application/xml would be the most accurate, since the actual body of the Put Block call is XML. I notice that the documentation uses a value of text/plain; charset=UTF-8 in the example request, but application/xml seems more correct to me.

My guess is that leaving out Content-Type entirely is causing a signing error, and the simplest thing to do is have one there. But what wouldn't make sense to me is to have something like Content-Type: image/jpeg on a call to Put Block, just because the blob itself is going to be a jpeg.

So I would suggest Content-Type: application/xml and x-ms-blob-content-type: <desired type of the blob>. Note that I haven't tested this.

Contributor

smarx commented Mar 28, 2013

Perhaps a Content-Type header with the value application/xml would be the most accurate, since the actual body of the Put Block call is XML. I notice that the documentation uses a value of text/plain; charset=UTF-8 in the example request, but application/xml seems more correct to me.

My guess is that leaving out Content-Type entirely is causing a signing error, and the simplest thing to do is have one there. But what wouldn't make sense to me is to have something like Content-Type: image/jpeg on a call to Put Block, just because the blob itself is going to be a jpeg.

So I would suggest Content-Type: application/xml and x-ms-blob-content-type: <desired type of the blob>. Note that I haven't tested this.

@gabpaquin

This comment has been minimized.

Show comment Hide comment
@gabpaquin

gabpaquin Mar 28, 2013

Contributor

I think also having Content-Type to application/xml would make more sense. I ran couple of tests (not extensive though) but If I have the following:

      def put_block_list(path, blockids, content_type = "application/octet-stream", metadata = {})        
        default_headers = {"Content-Type" => "application/xml", "x-ms-blob-content-type" => content_type, :x_ms_version => "2011-08-18"}
        execute :put, path, { :comp => 'blocklist' }, metadata.merge(default_headers), '<?xml version="1.0" encoding="utf-8"?><BlockList>' + blockids.map {|id| "<Latest>#{id.rstrip}</Latest>"}.join + '</BlockList>'
      end

It seems to be working really well. Could we use that approach?

Contributor

gabpaquin commented Mar 28, 2013

I think also having Content-Type to application/xml would make more sense. I ran couple of tests (not extensive though) but If I have the following:

      def put_block_list(path, blockids, content_type = "application/octet-stream", metadata = {})        
        default_headers = {"Content-Type" => "application/xml", "x-ms-blob-content-type" => content_type, :x_ms_version => "2011-08-18"}
        execute :put, path, { :comp => 'blocklist' }, metadata.merge(default_headers), '<?xml version="1.0" encoding="utf-8"?><BlockList>' + blockids.map {|id| "<Latest>#{id.rstrip}</Latest>"}.join + '</BlockList>'
      end

It seems to be working really well. Could we use that approach?

@smarx

This comment has been minimized.

Show comment Hide comment
@smarx

smarx Mar 28, 2013

Contributor

That looks great to me. Would you be willing to submit a pull request and let @johnnyhalife merge it in?

Contributor

smarx commented Mar 28, 2013

That looks great to me. Would you be willing to submit a pull request and let @johnnyhalife merge it in?

@johnnyhalife

This comment has been minimized.

Show comment Hide comment
@johnnyhalife

johnnyhalife Mar 28, 2013

Owner

Hey guys, I've just published v1.3.5 on rubygems (and merged on master). I'm assuming that this closes this issue, however if this also closes #32 please close it.

Owner

johnnyhalife commented Mar 28, 2013

Hey guys, I've just published v1.3.5 on rubygems (and merged on master). I'm assuming that this closes this issue, however if this also closes #32 please close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment