Common clang/gcc compiler bug causes make to hang when generating htmldoc.pdf

[ghost]

I cannot compile neither htmldoc 1.9.5 nor 1.8.30.

The make process gets stuck at 'Formatting htmldoc.pdf' forever. Waited for several hours, with no result.

Tried both local and built-in libraries.

How can I debug this issue?

[ghost]

Launched this under GDB, found that the code is stuck at

ps-pdf.cxx:5499

Then I found that in Makedefs the CC and CXX are set to clang and clang++.
So I reset them to gcc and g++, and the compilation works, although prints out many memory overflow errors.

Don't know how to fix this, I don't understand what the line

*lineptr++ = ' ';

does.

[michaelrsweet]

@lockywolf Hmm, never seen this problem before. What OS?

[ghost]

Also happens on FreeBSD 13.0 with clang 8 during package builds:
http://beefy12.nyi.freebsd.org/data/head-amd64-default/p499421_s346424/logs/errors/htmldoc-1.9.3.log (IPv6 only)

Updating the package to 1.9.5 does not help.

[ghost]

I can also reproduce the problem on Void Linux with latest master and building with clang 8 via CC=clang CXX=clang++ ./configure && make.

[michaelrsweet]

Can one of you try current Github master (what will eventually become 1.9.6)?

[ghost]

Same problem as before.

[ghost]

The problem seems to be in parser_pre() and it goes away if I disable optimizations on it via

diff --git a/htmldoc/ps-pdf.cxx b/htmldoc/ps-pdf.cxx
index f7981b7..bdb5940 100644
--- a/htmldoc/ps-pdf.cxx
+++ b/htmldoc/ps-pdf.cxx
@@ -5352,6 +5352,7 @@ parse_paragraph(tree_t *t,	/* I - Tree to parse */
  * 'parse_pre()' - Parse preformatted text and produce rendering list output.
  */
 
+__attribute__ ((optnone))
 static void
 parse_pre(tree_t *t,		/* I - Tree to parse */
           float  left,		/* I - Left margin */

Specifically as @lockywolf has already pointed out it seems to be stuck in this loop [1]:

do
{
    *lineptr++ = ' ';
    col ++;
}
while (col & 7);

[1] https://github.com/michaelrsweet/htmldoc/blob/master/htmldoc/ps-pdf.cxx#L5503

[michaelrsweet]

I have now tried building on Fedora 29 with gcc 8.3.1, in addition to Ubuntu 18.04 with gcc 7.3.0 and Clang 6.0, and macOS with clang 10.0.1. NONE of these exhibit these symptoms, and there is absolutely nothing wrong with the code in question, so this is a compiler bug.

Please try building with the "--enable-debug" configure option, which will just use "-g" to disable optimizations.

[michaelrsweet]

(Alternately you can build with GCC or a newer version of clang)

[ghost]

Yes, this is probably a bug in Clang 8.

Apple has its own version of Clang, so there might be some
misunderstanding here. Apple's clang-10.0.1 is actually based on
LLVM 7.0.0. See [1]. The problem happens with LLVM 8.0.0 with its
clang compiler. It already is the newest release [2].

Switching to GCC or older Clang is only a short term solution.

Please try building with the "--enable-debug" configure option,
which will just use "-g" to disable optimizations.

With no optimizations it works, but we have already established
that.

[1] https://en.wikipedia.org/wiki/Xcode#Latest_versions
[2] http://releases.llvm.org/

[michaelrsweet]

Please file a bug against clang/llvm

[ghost]

I've filed a bug for FreeBSD first: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237515

Marking col as volatile works around this problem too, so this is what I've applied to FreeBSD's htmldoc package for now.

--- htmldoc/ps-pdf.cxx.orig	2018-04-10 11:56:03 UTC
+++ htmldoc/ps-pdf.cxx
@@ -5361,7 +5361,7 @@ parse_pre(tree_t *t,		/* I - Tree to parse */
 		line[10240],
 		*lineptr,
 		*dataptr;
-  int		col;
+  volatile int	col;
   float		width,
 		height,
 		rgb[3];

[DimitryAndric]

I submitted a clang/llvm bug with a minimized test case here: https://bugs.llvm.org/show_bug.cgi?id=41998

[DimitryAndric]

FWIW, the clang/llvm bug has now been fixed here: llvm/llvm-project@2d756c4. It didn't make it into 8.0.1, but will at least end up in 9.0.0. I have backported the fix to FreeBSD.

[michaelrsweet]

Reopening - latest Xcode from Apple includes a buggy version of Clang, and there have been a bunch of dupes now... Will investigate refactoring this code that hasn't needed changes for the last 15 years...

[michaelrsweet]

[master ee020c7] Refactor the PRE tab support code to work around stupid compiler optimization bugs (Issue #349)