binary http uploads are corrupted by p:http-request #242

Closed
Conal-Tuohy opened this Issue Oct 7, 2016 · 6 comments

Comments

Projects
None yet
3 participants
@Conal-Tuohy
Contributor

Conal-Tuohy commented Oct 7, 2016

#241 has the details

@ndw

This comment has been minimized.

Show comment
Hide comment
@ndw

ndw Dec 15, 2016

Owner

Sorry. I feel like I've lost the focus on this change and I did rework http-request to support Apache HTTP Client 2.5.x (because it was needed for https: to sites secured with Let's Encrypt certificates).

Can you provide a test that demonstrates the corruption?

Owner

ndw commented Dec 15, 2016

Sorry. I feel like I've lost the focus on this change and I did rework http-request to support Apache HTTP Client 2.5.x (because it was needed for https: to sites secured with Let's Encrypt certificates).

Can you provide a test that demonstrates the corruption?

@fgeorges

This comment has been minimized.

Show comment
Hide comment
@fgeorges

fgeorges Dec 15, 2016

Quick note, I guess you mean 4.5.* (not 2.5.*)...

Quick note, I guess you mean 4.5.* (not 2.5.*)...

@ndw

This comment has been minimized.

Show comment
Hide comment
@ndw

ndw Dec 15, 2016

Owner

Oops. Yes.

Owner

ndw commented Dec 15, 2016

Oops. Yes.

@Conal-Tuohy

This comment has been minimized.

Show comment
Hide comment
@Conal-Tuohy

Conal-Tuohy Dec 16, 2016

Contributor

OK I found an "echo" server that will do the job. This test uploads a three-byte sequence (181, 181, 181) to an echo server and reports the bytes which the server says it received.

<p:declare-step version="1.0" 
   xmlns:p="http://www.w3.org/ns/xproc" 
   xmlns:c="http://www.w3.org/ns/xproc-step" 
   xmlns:j="http://marklogic.com/json" 
   name="http-binary-upload-test">
   <p:output port="result"/>
   <p:documentation>
      This test posts three bytes to an echo server on the internet.
      The bytes sent have identical values (181 = 0xB5).
      The byte sequence 0xB5 0xB5 0xB5 encoded in base64 = "tbW1".
   </p:documentation>
   <p:http-request name="post-three-bytes">
      <p:input port="source">
         <p:inline>
            <c:request href="http://echo.getpostman.com/post" method="POST">
               <c:body content-type="application/octet-stream" encoding="base64">
                  tbW1
               </c:body>
            </c:request>
         </p:inline>
      </p:input>
   </p:http-request>
   <p:unescape-markup name="parse-json-response">
      <p:with-option name="content-type" select="/c:body/@content-type"/>
      <p:with-option name="encoding" select="/c:body/@encoding"/>
   </p:unescape-markup>
   <p:template name="list-content-bytes-echoed-from-server">
      <p:with-param name="bytes" 
         select="string-join(/c:body/j:json/j:data/j:data/j:item, ', ')"/>
      <p:input port="template">
         <p:inline exclude-inline-prefixes="c j">
            <bytes>{$bytes}</bytes>
         </p:inline>
      </p:input>
   </p:template>
</p:declare-step>

My fork returns the expected result:

<bytes>181, 181, 181</bytes>

Calabash 1.1.14-97 returns:

<bytes>239, 191, 189, 239, 191, 189, 239, 191, 189</bytes>
Contributor

Conal-Tuohy commented Dec 16, 2016

OK I found an "echo" server that will do the job. This test uploads a three-byte sequence (181, 181, 181) to an echo server and reports the bytes which the server says it received.

<p:declare-step version="1.0" 
   xmlns:p="http://www.w3.org/ns/xproc" 
   xmlns:c="http://www.w3.org/ns/xproc-step" 
   xmlns:j="http://marklogic.com/json" 
   name="http-binary-upload-test">
   <p:output port="result"/>
   <p:documentation>
      This test posts three bytes to an echo server on the internet.
      The bytes sent have identical values (181 = 0xB5).
      The byte sequence 0xB5 0xB5 0xB5 encoded in base64 = "tbW1".
   </p:documentation>
   <p:http-request name="post-three-bytes">
      <p:input port="source">
         <p:inline>
            <c:request href="http://echo.getpostman.com/post" method="POST">
               <c:body content-type="application/octet-stream" encoding="base64">
                  tbW1
               </c:body>
            </c:request>
         </p:inline>
      </p:input>
   </p:http-request>
   <p:unescape-markup name="parse-json-response">
      <p:with-option name="content-type" select="/c:body/@content-type"/>
      <p:with-option name="encoding" select="/c:body/@encoding"/>
   </p:unescape-markup>
   <p:template name="list-content-bytes-echoed-from-server">
      <p:with-param name="bytes" 
         select="string-join(/c:body/j:json/j:data/j:data/j:item, ', ')"/>
      <p:input port="template">
         <p:inline exclude-inline-prefixes="c j">
            <bytes>{$bytes}</bytes>
         </p:inline>
      </p:input>
   </p:template>
</p:declare-step>

My fork returns the expected result:

<bytes>181, 181, 181</bytes>

Calabash 1.1.14-97 returns:

<bytes>239, 191, 189, 239, 191, 189, 239, 191, 189</bytes>
@Conal-Tuohy

This comment has been minimized.

Show comment
Hide comment
@Conal-Tuohy

Conal-Tuohy Dec 16, 2016

Contributor

In case you're wondering "why 181?", it was simply the first corrupted byte I discovered when I uploaded a PDF file and sniffed the network traffic.

Contributor

Conal-Tuohy commented Dec 16, 2016

In case you're wondering "why 181?", it was simply the first corrupted byte I discovered when I uploaded a PDF file and sniffed the network traffic.

@Conal-Tuohy

This comment has been minimized.

Show comment
Hide comment
@Conal-Tuohy

Conal-Tuohy Dec 16, 2016

Contributor

A build of my fork is available here, by the way: https://github.com/Conal-Tuohy/xmlcalabash1/releases/tag/1.1.11-ctuohy

Contributor

Conal-Tuohy commented Dec 16, 2016

A build of my fork is available here, by the way: https://github.com/Conal-Tuohy/xmlcalabash1/releases/tag/1.1.11-ctuohy

ndw added a commit that referenced this issue Dec 16, 2016

@ndw ndw closed this in f091455 Dec 16, 2016

ndw added a commit that referenced this issue Dec 16, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment