LibreOffice path on FreeBSD is not autodetected

[surli]

Hi @sbraconnier

I recently discovered that I wasn't able to make jodconverter running on a FreeBSD. I tested by just installing a FreeBSD and libreoffice using the standard way (i.e. pkg install libreoffice). Then when trying to run a toy example with jodconverter I have the following issues:

1. libreoffice path on FreeBSD is not autodetected
2. when specifying its path it considers it as invalid.

See:

Exception in thread "main" java.lang.IllegalStateException: Invalid officeHome: it doesn't contain soffice.bin: /usr/local/lib/libreoffice/program

and

$ ls -l /usr/local/lib/libreoffice/program/soffice.bin
-rwxr-xr-x  1 root  wheel  5320 Oct 25 19:13 /usr/local/lib/libreoffice/program/soffice.bin

I don't know much about FreeBSD, mostly testing because of an XWiki user issue. But I have a virtual machine ready to make some tests if you need more info or if you might have an idea of a fix to test.

[EDIT] I made my tests with jodconverter 4.2.2

[sbraconnier]

Thanks to report this! I was about to release a new version. I'll wait for this one to be fixed.

Could you please take a look here and report back which path(s) are missing there for FreeBSD ?

You can also look here for the default executable name. I wonder which it is not found on FreeBSD. Could you please tell me if the executable is there?

Thanks!

[surli]

So the missing path for autodetect seems to be: /usr/local/lib/libreoffice. Now regarding the LO home it seems that I made a mistake: I should have used a path without program/.
So I just restarted my test and I now get the following error:

Exception in thread "main" org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at org.jodconverter.office.OfficeProcess.start(OfficeProcess.java:412)
	at org.jodconverter.office.OfficeProcessManager.doStartProcessAndConnect(OfficeProcessManager.java:100)

Here's what I'm running:

LocalOfficeManager.Builder configuration = LocalOfficeManager.builder();
        LocalOfficeManager officeManager = configuration.officeHome("/usr/local/lib/libreoffice").build();
        officeManager.start();

[sbraconnier]

Could you please set the log level to TRACE ?

You will then be able to see which OS commands are used and their output. Maybe it will help...

[surli]

Interestingly the command to find the PID seems to use /bin/grep which seems wrong in FreeBSD:

2019-03-14 14:36:53.657 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Command Error: /bin/grep: not found

It's just a wrong path apparently:

$ /bin/grep
/bin/grep: not found
$ /usr/bin/grep
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.

Here's the complete trace I get:

/usr/local/openjdk8/bin/java -javaagent:/usr/local/intellij/lib/idea_rt.jar=35044:/usr/local/intellij/bin -Dfile.encoding=UTF-8 -classpath /usr/local/openjdk8/jre/lib/charsets.jar:/usr/local/openjdk8/jre/lib/compilefontconfig.jar:/usr/local/openjdk8/jre/lib/ext/cldrdata.jar:/usr/local/openjdk8/jre/lib/ext/dnsns.jar:/usr/local/openjdk8/jre/lib/ext/jaccess.jar:/usr/local/openjdk8/jre/lib/ext/localedata.jar:/usr/local/openjdk8/jre/lib/ext/nashorn.jar:/usr/local/openjdk8/jre/lib/ext/sunec.jar:/usr/local/openjdk8/jre/lib/ext/sunjce_provider.jar:/usr/local/openjdk8/jre/lib/ext/sunpkcs11.jar:/usr/local/openjdk8/jre/lib/ext/zipfs.jar:/usr/local/openjdk8/jre/lib/javazic.jar:/usr/local/openjdk8/jre/lib/jce.jar:/usr/local/openjdk8/jre/lib/jsse.jar:/usr/local/openjdk8/jre/lib/management-agent.jar:/usr/local/openjdk8/jre/lib/resources.jar:/usr/local/openjdk8/jre/lib/rt.jar:/home/urli/testjodconverter/target/classes:/usr/home/urli/.m2/repository/org/jodconverter/jodconverter-local/4.2.2/jodconverter-local-4.2.2.jar:/usr/home/urli/.m2/repository/org/jodconverter/jodconverter-core/4.2.2/jodconverter-core-4.2.2.jar:/usr/home/urli/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar:/usr/home/urli/.m2/repository/org/apache/commons/commons-lang3/3.7/commons-lang3-3.7.jar:/usr/home/urli/.m2/repository/com/google/code/gson/gson/2.8.5/gson-2.8.5.jar:/usr/home/urli/.m2/repository/org/openoffice/juh/4.1.2/juh-4.1.2.jar:/usr/home/urli/.m2/repository/org/openoffice/jurt/4.1.2/jurt-4.1.2.jar:/usr/home/urli/.m2/repository/org/openoffice/ridl/4.1.2/ridl-4.1.2.jar:/usr/home/urli/.m2/repository/org/openoffice/unoil/4.1.2/unoil-4.1.2.jar:/usr/home/urli/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/usr/home/urli/.m2/repository/ch/qos/logback/logback-classic/1.2.2/logback-classic-1.2.2.jar:/usr/home/urli/.m2/repository/ch/qos/logback/logback-core/1.2.2/logback-core-1.2.2.jar ConvertPresentation
2019-03-14 14:36:53.566 [main] INFO  o.j.office.OfficeProcessManager - Submitting task 'Start' and waiting...
2019-03-14 14:36:53.581 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Finding PID using
Command to get current running processes: [/bin/sh, -c, /bin/ps -e -o pid,args | /bin/grep soffice | /bin/grep -v grep]
Regex used to match current running process lines: ^\s*(?<Pid>\d+)\s+(?<CommanLine>.*)$
Regex used to match running office process we are looking for: \Qsoffice\E.*\Qsocket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager\E
2019-03-14 14:36:53.657 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Command Error: /bin/grep: not found
/bin/grep: not found
2019-03-14 14:36:53.658 [OfficeProcessThread-0] WARN  o.jodconverter.office.OfficeProcess - Profile dir '/tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1' already exists; deleting
2019-03-14 14:36:53.658 [OfficeProcessThread-0] DEBUG o.jodconverter.office.OfficeProcess - Deleting instance profile directory '/tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1'
2019-03-14 14:36:53.901 [OfficeProcessThread-0] INFO  o.j.office.OfficeDescriptor - soffice info (from exec path): Product: LibreOffice - Version: ??? - useLongOptionNameGnuStyle: true
2019-03-14 14:36:54.319 [OfficeProcessThread-0] DEBUG o.j.office.OfficeDescriptor - Building org.jodconverter.office.OfficeDescriptor from help output lines
2019-03-14 14:36:54.324 [OfficeProcessThread-0] INFO  o.j.office.OfficeDescriptor - soffice info (from help output): Product: LibreOffice - Version: 6.0.5.2 - useLongOptionNameGnuStyle: true
2019-03-14 14:36:54.325 [OfficeProcessThread-0] INFO  o.jodconverter.office.OfficeProcess - Starting process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' and profileDir '/tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1'
2019-03-14 14:36:54.576 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Finding PID using
Command to get current running processes: [/bin/sh, -c, /bin/ps -e -o pid,args | /bin/grep soffice | /bin/grep -v grep]
Regex used to match current running process lines: ^\s*(?<Pid>\d+)\s+(?<CommanLine>.*)$
Regex used to match running office process we are looking for: \Qsoffice\E.*\Qsocket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager\E
Exception in thread "main" org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at org.jodconverter.office.OfficeProcess.start(OfficeProcess.java:412)
	at org.jodconverter.office.OfficeProcessManager.doStartProcessAndConnect(OfficeProcessManager.java:100)
	at org.jodconverter.office.OfficeProcessManager.access$100(OfficeProcessManager.java:40)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:261)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:256)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2019-03-14 14:36:57.052 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Command Error: /bin/grep: not found
/bin/grep: not found
2019-03-14 14:36:57.052 [OfficeProcessThread-0] INFO  o.jodconverter.office.OfficeProcess - Started process; pid = -2
2019-03-14 14:36:57.053 [main] DEBUG o.j.office.OfficeProcessManager - ExecutionException catched in submitAndWait
java.util.concurrent.ExecutionException: org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.jodconverter.office.OfficeProcessManager.submitAndWait(OfficeProcessManager.java:299)
	at org.jodconverter.office.OfficeProcessManager.startAndWait(OfficeProcessManager.java:254)
	at org.jodconverter.office.OfficeProcessManagerPoolEntry.doStart(OfficeProcessManagerPoolEntry.java:168)
	at org.jodconverter.office.AbstractOfficeManagerPoolEntry.start(AbstractOfficeManagerPoolEntry.java:144)
	at org.jodconverter.office.AbstractOfficeManagerPool.doStart(AbstractOfficeManagerPool.java:188)
	at org.jodconverter.office.AbstractOfficeManagerPool.start(AbstractOfficeManagerPool.java:113)
	at org.jodconverter.office.LocalOfficeManager.start(LocalOfficeManager.java:38)
	at ConvertPresentation.main(ConvertPresentation.java:35)
Caused by: org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at org.jodconverter.office.OfficeProcess.start(OfficeProcess.java:412)
	at org.jodconverter.office.OfficeProcessManager.doStartProcessAndConnect(OfficeProcessManager.java:100)
	at org.jodconverter.office.OfficeProcessManager.access$100(OfficeProcessManager.java:40)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:261)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:256)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Process finished with exit code 1

[sbraconnier]

The key line here seems to be:

2019-03-14 14:36:57.052 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Command Error: /bin/grep: not found

Could you please check where is located this command ?

[surli]

Could you please check where is located this command ?

I edited my previous comment afterwards, it's in /usr/bin/grep

[sbraconnier]

Ok... hmmm.... I'm not sure what to think about this one. All "Unix" based OS will use the UnixProcessManager and the command to look for office processes is located here. It's hard to predict what would happen on other OS if I change the command line. It is possible to create a custom ProcessManager but it's sad that all FreeBSD users would have to do that.

I'll think about it.

[surli]

According to that: https://wiki.freebsd.org/BSDgrep the grep command is included in FreeBSD by default since a recent date. Wouldn't that be safer to only do the ps by command line, and to do the filtering using Java?

[surli]

Hm actually you might obtained pages of processes, so very bad in term of perf I guess.

[sbraconnier]

@surli

Could you please test with the latest SNAPSHOT? I created a new process manager specific to FreeeBSD, but I didn't have the chance to test it properly.

https://oss.sonatype.org/content/repositories/snapshots/

[surli]

OK I'll check that and come back to you.

[surli]

So I just tested with 4.2.3-SNAPSHOT and apparently I still have some problems. From what I see the grep command to find the PID is indeed fixed, but I've the feeling that the office process never really starts.

I tried to put a breakpoint on the findPID call and to do myself a ps aux|grep office at this point and I don't get any result. Now I also tried to see what's the command which is called to start office, it appears to be:

/usr/local/lib/libreoffice/program/soffice.bin --accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager --headless --invisible --nocrashreport --nodefault --nofirststartwizard --nolockcheck --nologo --norestore -env:UserInstallation=file:///tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1

when I run it directly from command line I got libreoffice starting and opening the start window, I tried on my linux machine and I obtained the same behaviour so I guess it's normal even though there is an --headless argument.

Here's the log I obtained:

/usr/local/openjdk8/bin/java -javaagent:/usr/local/intellij/lib/idea_rt.jar=40966:/usr/local/intellij/bin -Dfile.encoding=UTF-8 -classpath /usr/local/openjdk8/jre/lib/charsets.jar:/usr/local/openjdk8/jre/lib/compilefontconfig.jar:/usr/local/openjdk8/jre/lib/ext/cldrdata.jar:/usr/local/openjdk8/jre/lib/ext/dnsns.jar:/usr/local/openjdk8/jre/lib/ext/jaccess.jar:/usr/local/openjdk8/jre/lib/ext/localedata.jar:/usr/local/openjdk8/jre/lib/ext/nashorn.jar:/usr/local/openjdk8/jre/lib/ext/sunec.jar:/usr/local/openjdk8/jre/lib/ext/sunjce_provider.jar:/usr/local/openjdk8/jre/lib/ext/sunpkcs11.jar:/usr/local/openjdk8/jre/lib/ext/zipfs.jar:/usr/local/openjdk8/jre/lib/javazic.jar:/usr/local/openjdk8/jre/lib/jce.jar:/usr/local/openjdk8/jre/lib/jsse.jar:/usr/local/openjdk8/jre/lib/management-agent.jar:/usr/local/openjdk8/jre/lib/resources.jar:/usr/local/openjdk8/jre/lib/rt.jar:/home/urli/testjodconverter/target/classes:/usr/home/urli/.m2/repository/org/jodconverter/jodconverter-local/4.2.3-SNAPSHOT/jodconverter-local-4.2.3-20190503.124236-3.jar:/usr/home/urli/.m2/repository/org/jodconverter/jodconverter-core/4.2.3-SNAPSHOT/jodconverter-core-4.2.3-20190503.124230-3.jar:/usr/home/urli/.m2/repository/commons-io/commons-io/2.6/commons-io-2.6.jar:/usr/home/urli/.m2/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar:/usr/home/urli/.m2/repository/com/google/code/gson/gson/2.8.5/gson-2.8.5.jar:/usr/home/urli/.m2/repository/org/openoffice/juh/4.1.2/juh-4.1.2.jar:/usr/home/urli/.m2/repository/org/openoffice/jurt/4.1.2/jurt-4.1.2.jar:/usr/home/urli/.m2/repository/org/openoffice/ridl/4.1.2/ridl-4.1.2.jar:/usr/home/urli/.m2/repository/org/openoffice/unoil/4.1.2/unoil-4.1.2.jar:/usr/home/urli/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/usr/home/urli/.m2/repository/ch/qos/logback/logback-classic/1.2.2/logback-classic-1.2.2.jar:/usr/home/urli/.m2/repository/ch/qos/logback/logback-core/1.2.2/logback-core-1.2.2.jar ConvertPresentation
2019-05-04 16:26:01.622 [main] INFO  o.j.office.OfficeProcessManager - Submitting task 'Start' and waiting...
2019-05-04 16:26:01.634 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Finding PID using
Command to get current running processes: [/bin/sh, -c, /bin/ps -e -o pid,args | /usr/bin/grep soffice | /usr/bin/grep -v grep]
Regex used to match current running process lines: ^\s*(?<Pid>\d+)\s+(?<CommanLine>.*)$
Regex used to match running office process we are looking for: \Qsoffice\E.*\Qsocket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager\E
2019-05-04 16:26:01.708 [OfficeProcessThread-0] WARN  o.jodconverter.office.OfficeProcess - Profile dir '/tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1' already exists; deleting
2019-05-04 16:26:01.708 [OfficeProcessThread-0] DEBUG o.jodconverter.office.OfficeProcess - Deleting instance profile directory '/tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1'
2019-05-04 16:26:01.732 [OfficeProcessThread-0] INFO  o.j.office.OfficeDescriptor - soffice info (from exec path): Product: LibreOffice - Version: ??? - useLongOptionNameGnuStyle: true
2019-05-04 16:26:02.427 [OfficeProcessThread-0] DEBUG o.j.office.OfficeDescriptor - Building org.jodconverter.office.OfficeDescriptor from help output lines
2019-05-04 16:26:02.429 [OfficeProcessThread-0] INFO  o.j.office.OfficeDescriptor - soffice info (from help output): Product: LibreOffice - Version: 6.0.5.2 - useLongOptionNameGnuStyle: true
2019-05-04 16:26:02.429 [OfficeProcessThread-0] INFO  o.jodconverter.office.OfficeProcess - Starting process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' and profileDir '/tmp/.jodconverter_socket_host-127.0.0.1_port-2002_tcpNoDelay-1'
2019-05-04 16:26:02.467 [OfficeProcessThread-0] TRACE o.j.process.AbstractProcessManager - Finding PID using
Command to get current running processes: [/bin/sh, -c, /bin/ps -e -o pid,args | /usr/bin/grep soffice | /usr/bin/grep -v grep]
Regex used to match current running process lines: ^\s*(?<Pid>\d+)\s+(?<CommanLine>.*)$
Regex used to match running office process we are looking for: \Qsoffice\E.*\Qsocket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager\E
2019-05-04 16:26:02.483 [OfficeProcessThread-0] INFO  o.jodconverter.office.OfficeProcess - Started process; pid = -2
2019-05-04 16:26:02.538 [main] DEBUG o.j.office.OfficeProcessManager - ExecutionException catched in submitAndWait
java.util.concurrent.ExecutionException: org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.jodconverter.office.OfficeProcessManager.submitAndWait(OfficeProcessManager.java:299)
	at org.jodconverter.office.OfficeProcessManager.startAndWait(OfficeProcessManager.java:254)
	at org.jodconverter.office.OfficeProcessManagerPoolEntry.doStart(OfficeProcessManagerPoolEntry.java:168)
	at org.jodconverter.office.AbstractOfficeManagerPoolEntry.start(AbstractOfficeManagerPoolEntry.java:144)
	at org.jodconverter.office.AbstractOfficeManagerPool.doStart(AbstractOfficeManagerPool.java:188)
	at org.jodconverter.office.AbstractOfficeManagerPool.start(AbstractOfficeManagerPool.java:113)
	at org.jodconverter.office.LocalOfficeManager.start(LocalOfficeManager.java:38)
	at ConvertPresentation.main(ConvertPresentation.java:35)
Caused by: org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at org.jodconverter.office.OfficeProcess.start(OfficeProcess.java:411)
	at org.jodconverter.office.OfficeProcessManager.doStartProcessAndConnect(OfficeProcessManager.java:100)
	at org.jodconverter.office.OfficeProcessManager.access$100(OfficeProcessManager.java:40)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:261)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:256)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" org.jodconverter.office.OfficeException: A process with acceptString 'socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ServiceManager' started but its pid could not be found
	at org.jodconverter.office.OfficeProcess.start(OfficeProcess.java:411)
	at org.jodconverter.office.OfficeProcessManager.doStartProcessAndConnect(OfficeProcessManager.java:100)
	at org.jodconverter.office.OfficeProcessManager.access$100(OfficeProcessManager.java:40)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:261)
	at org.jodconverter.office.OfficeProcessManager$4.call(OfficeProcessManager.java:256)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Process finished with exit code 1

[sbraconnier]

I finally got time to work on this! I was a hard one.

The right command was:
/bin/ps -awx -o pid,args

Also, I had to add a sleep of 2 seconds before trying to get the PID of an office process once started, orelse the first connection attempt to the office process would hang for about 5 minutes before throwing a connection exception. Not sure why.

4.2.3 will be released this week.

[surli]

Great, thanks!