Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary reference files build system #2014

Closed
wants to merge 3 commits into from

Conversation

lierdakil
Copy link
Contributor

As in https://groups.google.com/forum/#!topic/pandoc-discuss/i9lconsh0yw

What it is:
A simple cabal post-build hook that zip-archives unpacked data for data/reference.docx and data/reference.odt. No additional dependencies, should work on all target platforms.

Why:
I firmly believe this will greatly simplify working with reference files for binary formats, since most of the work on these is done by-hand anyway (to my experience, that is). As an added benefit, it offers proper revision-tracking, instead of "binary file differs".

All relevant code is in lierdakil/pandoc@4d1267a.

@lierdakil
Copy link
Contributor Author

NB: No idea why travis failed, but this has little to do with changes proposed here...

@ShabbyX
Copy link
Contributor

ShabbyX commented Mar 19, 2015

@lierdakil Travis fails because of #2013 which depends on jgm/highlighting-kate#64

It would be fixed until either a new highlighting-kate version is released or the Travis build would build and install highlighting-kate (and pandoc-type while we're at it) from source.

@lierdakil
Copy link
Contributor Author

@ShabbyX, apparently, there is a new version of highlighting-kate (0.5.12, released 4 hours ago). I was a little too quick with this PR, it seems 😆

@ShabbyX
Copy link
Contributor

ShabbyX commented Mar 19, 2015

@lierdakil yes, John was a step ahead of us

@lierdakil lierdakil closed this Mar 20, 2015
@lierdakil lierdakil reopened this Mar 20, 2015
@lierdakil
Copy link
Contributor Author

Triggered a rebuild by reopening PR. Let's see if it works now.

@lierdakil
Copy link
Contributor Author

@jgm sorry to nag, but I'm sort of waiting on this one to move forward with #458, since it requires a toc heading style in reference.docx, and I'd prefer to work with text files if at all possible.

@jgm
Copy link
Owner

jgm commented Mar 28, 2015

This looks good. But when I tried to compile your branch, it didn't quite work. When I tried to open the generated files data/reference.docx and data/reference.odt, they were corrupted. (In the case of the docx, I got a message that the file was corrupted, but Word was able to repair it.)

I switched back to master, and found that data/reference.docx in master would not open either. (Message; one or more of the footnotes in the document is missing or corrupt.)

So, I think a recent commit involving this must have broken it. I'm not sure about the odt, because the reference.odt in master opens just fine, so maybe there is some additional problem with your build script.

@jgm
Copy link
Owner

jgm commented Mar 28, 2015

More on this: the error I have in master when I try to open reference.docx is different from the one I have in your branch. (In master, the error refers specifically to footnotes; in your branch it's more generic.) The error I get when I try to open reference.odt in your branch with LibreOffice is "General input/output error."

@jgm
Copy link
Owner

jgm commented Mar 28, 2015

Clue:

% unzip -d new-ref-odt reference.odt 
Archive:  reference.odt
   creating: new-ref-odt/Configurations2/
   creating: new-ref-odt/Configurations2/accelerator/
 extracting: new-ref-odt/Configurations2/accelerator/current.xml  
  inflating: new-ref-odt/content.xml  
  inflating: new-ref-odt/manifest.rdf  
   creating: new-ref-odt/META-INF/
  inflating: new-ref-odt/META-INF/manifest.xml  
  inflating: new-ref-odt/meta.xml    
 extracting: new-ref-odt/mimetype    
  inflating: new-ref-odt/settings.xml  
  inflating: new-ref-odt/styles.xml  
   creating: new-ref-odt/Thumbnails/
  inflating: new-ref-odt/Thumbnails/thumbnail.png  
warning:  skipped "../" path component(s) in ../
warning:  skipped "../" path component(s) in ../docx/
   creating: new-ref-odt/docx/
warning:  skipped "../" path component(s) in ../docx/[Content_Types].xml
  inflating: new-ref-odt/docx/[Content_Types].xml  
warning:  skipped "../" path component(s) in ../docx/_rels/
   creating: new-ref-odt/docx/_rels/
warning:  skipped "../" path component(s) in ../docx/_rels/.rels
  inflating: new-ref-odt/docx/_rels/.rels  
warning:  skipped "../" path component(s) in ../docx/docProps/
   creating: new-ref-odt/docx/docProps/
warning:  skipped "../" path component(s) in ../docx/docProps/app.xml
  inflating: new-ref-odt/docx/docProps/app.xml  
warning:  skipped "../" path component(s) in ../docx/docProps/core.xml
  inflating: new-ref-odt/docx/docProps/core.xml  
warning:  skipped "../" path component(s) in ../docx/word/
   creating: new-ref-odt/docx/word/
warning:  skipped "../" path component(s) in ../docx/word/_rels/
   creating: new-ref-odt/docx/word/_rels/
warning:  skipped "../" path component(s) in ../docx/word/_rels/document.xml.rels
  inflating: new-ref-odt/docx/word/_rels/document.xml.rels  
warning:  skipped "../" path component(s) in ../docx/word/_rels/footnotes.xml.rels
  inflating: new-ref-odt/docx/word/_rels/footnotes.xml.rels  
warning:  skipped "../" path component(s) in ../docx/word/document.xml
  inflating: new-ref-odt/docx/word/document.xml  
warning:  skipped "../" path component(s) in ../docx/word/fontTable.xml
  inflating: new-ref-odt/docx/word/fontTable.xml  
warning:  skipped "../" path component(s) in ../docx/word/footnotes.xml
  inflating: new-ref-odt/docx/word/footnotes.xml  
warning:  skipped "../" path component(s) in ../docx/word/numbering.xml
  inflating: new-ref-odt/docx/word/numbering.xml  
warning:  skipped "../" path component(s) in ../docx/word/settings.xml
  inflating: new-ref-odt/docx/word/settings.xml  
warning:  skipped "../" path component(s) in ../docx/word/styles.xml
  inflating: new-ref-odt/docx/word/styles.xml  
warning:  skipped "../" path component(s) in ../docx/word/theme/
   creating: new-ref-odt/docx/word/theme/
warning:  skipped "../" path component(s) in ../docx/word/theme/theme1.xml
  inflating: new-ref-odt/docx/word/theme/theme1.xml  
warning:  skipped "../" path component(s) in ../docx/word/webSettings.xml
  inflating: new-ref-odt/docx/word/webSettings.xml  
warning:  skipped "../" path component(s) in ../dzslides/
   creating: new-ref-odt/dzslides/
warning:  skipped "../" path component(s) in ../dzslides/template.html
  inflating: new-ref-odt/dzslides/template.html  
warning:  skipped "../" path component(s) in ../epub.css
  inflating: new-ref-odt/epub.css    
warning:  skipped "../" path component(s) in ../LaTeXMathML.js
  inflating: new-ref-odt/LaTeXMathML.js  
warning:  skipped "../" path component(s) in ../make-reference-files.hs
  inflating: new-ref-odt/make-reference-files.hs  
warning:  skipped "../" path component(s) in ../MathMLinHTML.js
  inflating: new-ref-odt/MathMLinHTML.js  
warning:  skipped "../" path component(s) in ../odt/
   creating: new-ref-odt/odt/
warning:  skipped "../" path component(s) in ../odt/Configurations2/
   creating: new-ref-odt/odt/Configurations2/
warning:  skipped "../" path component(s) in ../odt/Configurations2/accelerator/
   creating: new-ref-odt/odt/Configurations2/accelerator/
warning:  skipped "../" path component(s) in ../odt/Configurations2/accelerator/current.xml
 extracting: new-ref-odt/odt/Configurations2/accelerator/current.xml  
warning:  skipped "../" path component(s) in ../odt/content.xml
  inflating: new-ref-odt/odt/content.xml  
warning:  skipped "../" path component(s) in ../odt/manifest.rdf
  inflating: new-ref-odt/odt/manifest.rdf  
warning:  skipped "../" path component(s) in ../odt/META-INF/
   creating: new-ref-odt/odt/META-INF/
warning:  skipped "../" path component(s) in ../odt/META-INF/manifest.xml
  inflating: new-ref-odt/odt/META-INF/manifest.xml  
warning:  skipped "../" path component(s) in ../odt/meta.xml
  inflating: new-ref-odt/odt/meta.xml  
warning:  skipped "../" path component(s) in ../odt/mimetype
 extracting: new-ref-odt/odt/mimetype  
warning:  skipped "../" path component(s) in ../odt/settings.xml
  inflating: new-ref-odt/odt/settings.xml  
warning:  skipped "../" path component(s) in ../odt/styles.xml
  inflating: new-ref-odt/odt/styles.xml  
warning:  skipped "../" path component(s) in ../odt/Thumbnails/
   creating: new-ref-odt/odt/Thumbnails/
warning:  skipped "../" path component(s) in ../odt/Thumbnails/thumbnail.png
  inflating: new-ref-odt/odt/Thumbnails/thumbnail.png  
warning:  skipped "../" path component(s) in ../reference.docx
  inflating: new-ref-odt/reference.docx  
warning:  skipped "../" path component(s) in ../sample.lua
  inflating: new-ref-odt/sample.lua  
warning:  skipped "../" path component(s) in ../templates/
   creating: new-ref-odt/templates/
warning:  skipped "../" path component(s) in ../templates/.git
  inflating: new-ref-odt/templates/.git  
warning:  skipped "../" path component(s) in ../templates/default.asciidoc
  inflating: new-ref-odt/templates/default.asciidoc  
warning:  skipped "../" path component(s) in ../templates/default.beamer
  inflating: new-ref-odt/templates/default.beamer  
warning:  skipped "../" path component(s) in ../templates/default.context
  inflating: new-ref-odt/templates/default.context  
warning:  skipped "../" path component(s) in ../templates/default.docbook
  inflating: new-ref-odt/templates/default.docbook  
warning:  skipped "../" path component(s) in ../templates/default.dokuwiki
  inflating: new-ref-odt/templates/default.dokuwiki  
warning:  skipped "../" path component(s) in ../templates/default.dzslides
  inflating: new-ref-odt/templates/default.dzslides  
warning:  skipped "../" path component(s) in ../templates/default.epub
  inflating: new-ref-odt/templates/default.epub  
warning:  skipped "../" path component(s) in ../templates/default.epub3
  inflating: new-ref-odt/templates/default.epub3  
warning:  skipped "../" path component(s) in ../templates/default.haddock
 extracting: new-ref-odt/templates/default.haddock  
warning:  skipped "../" path component(s) in ../templates/default.html
  inflating: new-ref-odt/templates/default.html  
warning:  skipped "../" path component(s) in ../templates/default.html5
  inflating: new-ref-odt/templates/default.html5  
warning:  skipped "../" path component(s) in ../templates/default.icml
  inflating: new-ref-odt/templates/default.icml  
warning:  skipped "../" path component(s) in ../templates/default.latex
  inflating: new-ref-odt/templates/default.latex  
warning:  skipped "../" path component(s) in ../templates/default.man
  inflating: new-ref-odt/templates/default.man  
warning:  skipped "../" path component(s) in ../templates/default.markdown
  inflating: new-ref-odt/templates/default.markdown  
warning:  skipped "../" path component(s) in ../templates/default.mediawiki
  inflating: new-ref-odt/templates/default.mediawiki  
warning:  skipped "../" path component(s) in ../templates/default.opendocument
  inflating: new-ref-odt/templates/default.opendocument  
warning:  skipped "../" path component(s) in ../templates/default.opml
  inflating: new-ref-odt/templates/default.opml  
warning:  skipped "../" path component(s) in ../templates/default.org
  inflating: new-ref-odt/templates/default.org  
warning:  skipped "../" path component(s) in ../templates/default.plain
  inflating: new-ref-odt/templates/default.plain  
warning:  skipped "../" path component(s) in ../templates/default.revealjs
  inflating: new-ref-odt/templates/default.revealjs  
warning:  skipped "../" path component(s) in ../templates/default.rst
  inflating: new-ref-odt/templates/default.rst  
warning:  skipped "../" path component(s) in ../templates/default.rtf
  inflating: new-ref-odt/templates/default.rtf  
warning:  skipped "../" path component(s) in ../templates/default.s5
  inflating: new-ref-odt/templates/default.s5  
warning:  skipped "../" path component(s) in ../templates/default.slideous
  inflating: new-ref-odt/templates/default.slideous  
warning:  skipped "../" path component(s) in ../templates/default.slidy
  inflating: new-ref-odt/templates/default.slidy  
warning:  skipped "../" path component(s) in ../templates/default.texinfo
  inflating: new-ref-odt/templates/default.texinfo  
warning:  skipped "../" path component(s) in ../templates/default.textile
  inflating: new-ref-odt/templates/default.textile  
warning:  skipped "../" path component(s) in ../templates/README.markdown
  inflating: new-ref-odt/templates/README.markdown 

I see the problem, and I've got the fix. getDirectoryContents returns . and .., and you normally need to filter these out. However, in this case there's an even simpler fix:

-  archive <-
-    getDirectoryContents "."
-    >>= addFilesToArchive [OptRecursive] emptyArchive
+  archive <- addFilesToArchive [OptRecursive] emptyArchive ["."]

I'll go ahead and merge this with this change.

@jgm
Copy link
Owner

jgm commented Mar 28, 2015

I have pushed these changes. Closing. Will open a new issue about the footnote issue in reference.docx.

@jgm jgm closed this Mar 28, 2015
@lierdakil
Copy link
Contributor Author

Thank you. And sorry about the mess.

@lierdakil lierdakil deleted the reference-build branch March 29, 2015 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants