New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unzip with p:store (cx:decode) #41

Closed
xquery opened this Issue Apr 10, 2012 · 4 comments

Comments

Projects
None yet
2 participants
@xquery
Contributor

xquery commented Apr 10, 2012

I want to download a zip and extract all contents ... I thought this would be made easier/possible using cx:decode="true" on store step. This is especially a problem in terms of handling zips containing multiple file types (containing images, html, xml, etc)... its not entirely clear.

Consider the following example pipeline, which downloads manifest from a test.zip and attempts to store each item in its native form ... this results in a bunch of base64 encoded files.

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step 
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:p="http://www.w3.org/ns/xproc"
    xmlns:cx="http://xmlcalabash.com/ns/extensions"
    xmlns:depx="https://github.com/xquery/depx"
    name="generate-package"
    version="1.0"
    exclude-inline-prefixes="cx c p">

  <p:import href="extension-library.xml"/>

    <cx:unzip>
       <p:with-option name="href" select="'http://localhost/test.zip'"/>
    </cx:unzip>

    <p:for-each>
        <p:iteration-source select="//c:file"/>
        <p:variable name="file" select="c:file/@name"/>

        <cx:unzip>
          <p:with-option name="content-type" select="'*'"/>
          <p:with-option name="href" select="'http://localhost/test.zip'"/>
          <p:with-option name="file" select="$file"/>
        </cx:unzip>

        <p:store cx:decode="true">
          <p:with-option name="href" select="concat('/Users/jfuller/Source/Webcomposite/xproc-tests/',$file)"/>
        </p:store>

    </p:for-each>

  <!-- run pipeline manually in emacs with m-x compile //-->
  <p:documentation>
    (:
    -- Local Variables:
    -- compile-command: "/usr/local/bin/calabash test-zip.xpl"
    -- End:
    :)
  </p:documentation>


</p:declare-step>
@xquery

This comment has been minimized.

Show comment
Hide comment
@xquery

xquery Apr 11, 2012

Contributor

some further info;

in p:store impl

line 116-120 ... you branch logic based on c:data element having an encoding attribute which equals base64 ... I think cx:unzip does not emit this so we never branch to storeBinary()

Contributor

xquery commented Apr 11, 2012

some further info;

in p:store impl

line 116-120 ... you branch logic based on c:data element having an encoding attribute which equals base64 ... I think cx:unzip does not emit this so we never branch to storeBinary()

@xquery

This comment has been minimized.

Show comment
Hide comment
@xquery

xquery Apr 11, 2012

Contributor

I think adding a tree.addAttribute() near line 157 (Unzip.java) and adding an encoding attribute with value 'base64' should do the trick.

Contributor

xquery commented Apr 11, 2012

I think adding a tree.addAttribute() near line 157 (Unzip.java) and adding an encoding attribute with value 'base64' should do the trick.

@xquery

This comment has been minimized.

Show comment
Hide comment
@xquery

xquery Apr 11, 2012

Contributor

for completeness this works when I add p:add-attribute setp

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step 
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:p="http://www.w3.org/ns/xproc"
    xmlns:cx="http://xmlcalabash.com/ns/extensions"
    xmlns:depx="https://github.com/xquery/depx"
    name="generate-package"
    version="1.0"
    exclude-inline-prefixes="cx c p">

  <!--p:output port="source"/-->

  <p:import href="extension-library.xml"/>

    <cx:unzip>
       <p:with-option name="href" select="'http://localhost/test.zip'"/>
    </cx:unzip>

    <p:for-each>
        <p:iteration-source select="//c:file"/>
        <p:variable name="file" select="c:file/@name"/>

        <cx:unzip content-type="*">
          <p:with-option name="href" select="'http://localhost/test.zip'"/>
          <p:with-option name="file" select="$file"/>
        </cx:unzip>

<p:add-attribute match="/c:data" attribute-name="encoding" attribute-value="base64"/>

        <p:store cx:decode="true">
          <p:with-option name="href" select="concat('/Users/jfuller/Source/Webcomposite/xproc-tests/',$file)"/>
        </p:store>

    </p:for-each>

  <!-- run pipeline manually in emacs with m-x compile //-->
  <p:documentation>
    (:
    -- Local Variables:
    -- compile-command: "/usr/local/bin/calabash test-zip.xpl"
    -- End:
    :)
  </p:documentation>


</p:declare-step>
Contributor

xquery commented Apr 11, 2012

for completeness this works when I add p:add-attribute setp

<?xml version="1.0" encoding="UTF-8"?>
<p:declare-step 
    xmlns:c="http://www.w3.org/ns/xproc-step"
    xmlns:p="http://www.w3.org/ns/xproc"
    xmlns:cx="http://xmlcalabash.com/ns/extensions"
    xmlns:depx="https://github.com/xquery/depx"
    name="generate-package"
    version="1.0"
    exclude-inline-prefixes="cx c p">

  <!--p:output port="source"/-->

  <p:import href="extension-library.xml"/>

    <cx:unzip>
       <p:with-option name="href" select="'http://localhost/test.zip'"/>
    </cx:unzip>

    <p:for-each>
        <p:iteration-source select="//c:file"/>
        <p:variable name="file" select="c:file/@name"/>

        <cx:unzip content-type="*">
          <p:with-option name="href" select="'http://localhost/test.zip'"/>
          <p:with-option name="file" select="$file"/>
        </cx:unzip>

<p:add-attribute match="/c:data" attribute-name="encoding" attribute-value="base64"/>

        <p:store cx:decode="true">
          <p:with-option name="href" select="concat('/Users/jfuller/Source/Webcomposite/xproc-tests/',$file)"/>
        </p:store>

    </p:for-each>

  <!-- run pipeline manually in emacs with m-x compile //-->
  <p:documentation>
    (:
    -- Local Variables:
    -- compile-command: "/usr/local/bin/calabash test-zip.xpl"
    -- End:
    :)
  </p:documentation>


</p:declare-step>

ndw added a commit that referenced this issue May 19, 2012

@ndw

This comment has been minimized.

Show comment
Hide comment
@ndw

ndw May 19, 2012

Owner

Good catch, Jim. Fixed.

Owner

ndw commented May 19, 2012

Good catch, Jim. Fixed.

@ndw ndw closed this May 19, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment