Pdftk on Darwin

[berdario]

Issue description

This is just to note down a couple of things that I found:

We are currently shipping GCJ built on GCC 4.9. Updating it to GCC 6.4 makes it build on Darwin.
The only dependency that truly needs GCJ is Pdftk (Rhino can just run on the OpenJDK).

The latest version of Pdftk is 2.0.2, its source has been released in July 2013

Reverse deps of Pdftk are Impressive (presentation tool, latest release in 2015), diffoscope (optional dependecy), mwlib (python software to convert from mediawiki to formats like pdf, latest release in 2014)

GCC dropped support for GCJ in GCC7, and thus (even on Linux) we might eventually drop pdftk in a few years, luckily the only actively developed dependency (diffoscope) seems to have it only as an optional dependency.

I attempted to build Pdftk on Darwin, but it's failing due to it being unable to find ecj1 and gcjh.

The former might be the eclipse compiler (but it's not available when building gcj4.9 on Linux, either) and it's not clear if it's truly included with GCC (I'd be surprised). The latter is a binary that would generate header files. To check if it's missing from GCJ6, we should rebuild it on Linux.

Here's a page with discussion on difficulties (and workarounds and alternatives) with the installation of pdtk on Macos

To get it working on Homebrew others are relying on the prebuilt binary downloaded directly from upstream.

Steps to reproduce

Once bumped version of gcj

nix-build -A pdftk

Technical details

• System: Macos 10.12.6
• Nix version: 1.11.14
• Nixpkgs version: f7d7c7b

[vcunat]

IIRC pdftk is quite useful by itself, too.

[7c6f434c]

Yes, pdftk, even in a stagnated state, is a huge useful tool; I know no opensource cross-platform alternatives for command-line PDF manipulation like adding background or stamping something on top of every page…

[sjau]

Also getting build errors but not sure if that's proper to report here:

shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/liblto_plugin.so.0.0.0    
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/jvgenmain                 
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/collect2                  
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/cc1                       
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/cc1plus                   
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/lto1                      
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/lto-wrapper               
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/plugin/gengtype           
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/ecj1                      
shrinking /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec/gcc/x86_64-unknown-linux-gnu/6.4.0/jc1                       
stripping (with flags -S) in /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/lib  /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/lib64  /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/libexec  /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/bin 
patching script interpreter paths in /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0                                           
/nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0/bin/rebuild-gcj-db: interpreter directive changed from "/bin/bash" to "/nix/store/flb9ar1xdd13c606aa4my9miy3iv4vyk-bash-4.4-p12/bin/bash"
checking for references to /tmp/nix-build-gcj-6.4.0.drv-0 in /nix/store/qxjpxjb5dkb5d0lhknd9rbla6lkxnr06-gcj-6.4.0...                
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
wrong ELF type                   
shrinking RPATHs of ELF executables and libraries in /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib                       
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libcc1.so.0.0.0                                              
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libcilkrts.so.5.0.0                                          
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libquadmath.so.0.0.0                                         
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgcj_bc.so.1.0.0                                           
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgij.so.17.0.0                                             
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libubsan.so.0.0.0                                            
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgcj.so.17.0.0                                             
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libitm.so.1.0.0                                              
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libmpx.so.2.0.0                                              
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgcc_s.so.1                                                
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libstdc++.so.6.0.22                                          
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgomp.so.1.0.0                                             
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libasan.so.3.0.0                                             
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgcj-tools.so.17.0.0                                       
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libgcj_bc.so                                                 
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libatomic.so.1.2.0                                           
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libssp.so.0.0.0                                              
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/liblsan.so.0.0.0                                             
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libmpxwrappers.so.2.0.0                                      
shrinking /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib/libtsan.so.0.0.0                                             
stripping (with flags -S) in /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib  /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib/lib64 
patching script interpreter paths in /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib                                       
checking for references to /tmp/nix-build-gcj-6.4.0.drv-0 in /nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib...            
shrinking RPATHs of ELF executables and libraries in /nix/store/1w9fwww0f89x6csc6p7c3ag3jxj2hbfd-gcj-6.4.0-man                       
gzipping man pages under /nix/store/1w9fwww0f89x6csc6p7c3ag3jxj2hbfd-gcj-6.4.0-man/share/man/                                        
patching script interpreter paths in /nix/store/1w9fwww0f89x6csc6p7c3ag3jxj2hbfd-gcj-6.4.0-man                                       
checking for references to /tmp/nix-build-gcj-6.4.0.drv-0 in /nix/store/1w9fwww0f89x6csc6p7c3ag3jxj2hbfd-gcj-6.4.0-man...            
shrinking RPATHs of ELF executables and libraries in /nix/store/aa50fs68h96576x5yfp1q4hnlna2z34z-gcj-6.4.0-info                      
patching script interpreter paths in /nix/store/aa50fs68h96576x5yfp1q4hnlna2z34z-gcj-6.4.0-info                                      
checking for references to /tmp/nix-build-gcj-6.4.0.drv-0 in /nix/store/aa50fs68h96576x5yfp1q4hnlna2z34z-gcj-6.4.0-info...           
cycle detected in the references of ‘/nix/store/cb5fkr0yf7pgd8jzq4gsw9xyqsvwa8ab-gcj-6.4.0-lib’                                      
cannot build derivation ‘/nix/store/yjf5p0bdvmh0rqp04radd5y07kpap37d-pdftk-2.02.drv’: 1 dependencies couldn't be built               
cannot build derivation ‘/nix/store/rcdfs6ljval0wjgfjnpfvir6m304iclg-system-path.drv’: 1 dependencies couldn't be built              
cannot build derivation ‘/nix/store/3zhxn1vvfgxaqdyy068468g1lx66pf95-nixos-system-subi-18.03pre116572.9824ca6975.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/3zhxn1vvfgxaqdyy068468g1lx66pf95-nixos-system-subi-18.03pre116572.9824ca6975.drv’ failed                 

And pdftk is a very, very useful tool. I work a lot with PDFs and I did create some scripts for Dolpin integration:

https://github.com/sjau/pdfForts/blob/master/servicemenu.png

I even have a .nix file to install it: https://github.com/sjau/pdfForts

[vcunat]

Yes, gcj is broken on Linux since #29758. I suppose we'll go for a workaround analogical to 909a46f (on all platforms, to be more consistent).

[vcunat]

Done by 80c57fd. I hope that change didn't break gcj/pdftk on Darwin (so far it didn't succeed on Hydra due to exceeding log length anyway).

[berdario]

@vcunat
I think you need to add patchelf among gcc's inputs:

patching script interpreter paths in /nix/store/aa8wbvr9b5sx4m2x3dzqymyw8i5x2v1q-gcj-6.4.0-info
/nix/store/l1pdvl3yylngm46qzxwnywhdc9gsszzx-stdenv-darwin/setup: line 99: patchelf: command not found
/nix/store/l1pdvl3yylngm46qzxwnywhdc9gsszzx-stdenv-darwin/setup: line 99: patchelf: command not found
builder for ‘/nix/store/lvxhrczzzb97qldkw26zgr62zya19yp7-gcj-6.4.0.drv’ failed with exit code 127

[vcunat]

Oops, I didn't realize this. Patchelf is only for ELFs, so not for Darwin. I'll just drop the phase on Darwin – I don't expect problems due to that, as it worked without it before. @berdario: can you/someone test it on Darwin before I push it? https://github.com/NixOS/nixpkgs/commit/2bad0d83bee.patch

[berdario]

Oh, yeah... forgot that Darwin has Mach executables, not ELF... we don't have a patchMach, do we?

Anyhow @vcunat your patch works: gcj builds fine on Darwin

pdftk still fails with the same issue that I mentioned above:

I attempted to build Pdftk on Darwin, but it's failing due to it being unable to find ecj1 and gcjh.

gcj -fsource=1.3 -O2 --encoding=UTF-8 --classpath="/usr/share/java/libgcj.jar:/private/tmp/nix-build-pdftk-2.02.drv-0/pdftk-2.02-dist/java:." -C pdftk/org/bouncycastle/crypto/DataLengthException.java
gcj: error trying to exec 'ecj1': execvp: No such file or directory
make[1]: [Makefile:44: pdftk/org/bouncycastle/crypto/DataLengthException.class] Error 1 (ignored)
gcjh -force --classpath="/usr/share/java/libgcj.jar:/private/tmp/nix-build-pdftk-2.02.drv-0/pdftk-2.02-dist/java:." pdftk/org/bouncycastle/crypto/DataLengthException
/nix/store/5j8r4n35rzbwg95ayrfwn5ir07c1v6f6-bash-4.4-p12/bin/bash: gcjh: command not found
make[1]: [Makefile:47: pdftk/org/bouncycastle/crypto/DataLengthException.h] Error 127 (ignored)
gjar -cf java_lib.jar pdftk/com/lowagie/*/*/*/*.class pdftk/com/lowagie/*/*/*.class pdftk/com/lowagie/*/*.class pdftk/org/bouncycastle/*/*.class pdftk/org/bouncycastle/*/*/*.class pdftk/com/lowagie/text/pdf/fonts/Times-BoldItalic.afm pdftk/com/lowagie/text/pdf/fonts/ZapfDingbats.afm pdftk/com/lowagie/text/pdf/fonts/Times-Italic.afm pdftk/com/lowagie/text/pdf/fonts/Helvetica-Bold.afm pdftk/com/lowagie/text/pdf/fonts/Helvetica.afm pdftk/com/lowagie/text/pdf/fonts/Courier-BoldOblique.afm pdftk/com/lowagie/text/pdf/fonts/Symbol.afm pdftk/com/lowagie/text/pdf/fonts/Courier-Oblique.afm pdftk/com/lowagie/text/pdf/fonts/Times-Bold.afm pdftk/com/lowagie/text/pdf/fonts/Times-Roman.afm pdftk/com/lowagie/text/pdf/fonts/Courier-Bold.afm pdftk/com/lowagie/text/pdf/fonts/Helvetica-BoldOblique.afm pdftk/com/lowagie/text/pdf/fonts/Courier.afm pdftk/com/lowagie/text/pdf/fonts/Helvetica-Oblique.afm
/nix/store/5j8r4n35rzbwg95ayrfwn5ir07c1v6f6-bash-4.4-p12/bin/bash: gjar: command not found
make[1]: [Makefile:51: java_lib.o] Error 127 (ignored)
gcj -fsource=1.3 -O2 --encoding=UTF-8 --classpath="/usr/share/java/libgcj.jar:/private/tmp/nix-build-pdftk-2.02.drv-0/pdftk-2.02-dist/java:." -c java_lib.jar
gcj: error: java_lib.jar: No such file or directory
gcj: error: libgcj.spec: No such file or directory

I see that ecj should be handled in the gcc derivation, though when I tried to check the built derivation with the -K option last weekend, I couldn't find it... apparently we're also missing gjar

[vcunat]

On Linux the tools are there, even though not directly visible (i.e. only in the unwrapped compiler)

$ ls /nix/store/xp461jc29wmfixkbvv8gnxbfhb8hvsk1-gcj-6.4.0/bin/
aot-compile    gcc         gcjh       gjarsigner     grmic         jv-convert                          x86_64-unknown-linux-gnu-gcc-ar
c++            gcc-ar      gcov       gjavah         grmid         rebuild-gcj-db                      x86_64-unknown-linux-gnu-gcc-nm
cpp            gcc-nm      gcov-dump  gjdoc          grmiregistry  x86_64-unknown-linux-gnu-c++        x86_64-unknown-linux-gnu-gcc-ranlib
g++            gcc-ranlib  gcov-tool  gkeytool       gserialver    x86_64-unknown-linux-gnu-g++        x86_64-unknown-linux-gnu-gcj
gappletviewer  gcj         gij        gnative2ascii  gtnameserv    x86_64-unknown-linux-gnu-gcc
gc-analyze     gcj-dbtool  gjar       gorbd          jcf-dump      x86_64-unknown-linux-gnu-gcc-6.4.0

[alyssais]

we don't have a patchMach, do we?

install_name_tool from darwin.cctools.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[7c6f434c]

Is new/ported pdftk currently in Nixpkgs usable on macOS?

[veprbl]

Is new/ported pdftk currently in Nixpkgs usable on macOS?

Yes, it is.

[7c6f434c]

OK, I guess I close it for now to reopen if there is something broken…