lexical error at character '\n'

[asr]

@jstolarek reported an error installing Agda. A MWE is as follows

$ ghc -cpp -pgmPcpphs -optP--cpp Test.hs

Test.hs:0:2: error: lexical error at character '\n'

$ cat Test.hs
module Main where

#define FOO 1

#ifdef FOO
foo = 42
#endif

main = print foo

$ cpphs --version
cpphs 1.20.2

Blocking agda/agda#1285.

[malcolmwallace]

What platform? What version of ghc? What locale settings? I cannot reproduce on MacOS X with ghc-7.6.1:

$ ghc -cpp -pgmPcpphs -optP--cpp asr.hs
[1 of 1] Compiling Main             ( asr.hs, asr.o )
Linking asr …
$ cpphs --version
cpphs 1.20.2

[jstolarek]

Debian Wheezy x86_64 with multiarch, GHC 8.0.1. Locale:

[killy@GLaDOS : ~] locale
LANG=pl_PL.UTF-8
LANGUAGE=
LC_CTYPE="pl_PL.UTF-8"
LC_NUMERIC="pl_PL.UTF-8"
LC_TIME="pl_PL.UTF-8"
LC_COLLATE="pl_PL.UTF-8"
LC_MONETARY="pl_PL.UTF-8"
LC_MESSAGES="pl_PL.UTF-8"
LC_PAPER="pl_PL.UTF-8"
LC_NAME="pl_PL.UTF-8"
LC_ADDRESS="pl_PL.UTF-8"
LC_TELEPHONE="pl_PL.UTF-8"
LC_MEASUREMENT="pl_PL.UTF-8"
LC_IDENTIFICATION="pl_PL.UTF-8"
LC_ALL=

I used to have exactly the same problem on OpenSUSE with GHC 7.10.3.

[asr]

The issue was reported in the Agda mailing list a couple of years ago. I have not been able to reproduce the problem using various versions of GHC and cpphs in Ubuntu.

[malcolmwallace]

Is there any way you can capture the output of cpphs before it gets to ghc, so we can examine what is the mysterious character at line 0, columns 1, preceding the unexpected newline at column 2? My suspicion is that it might be a byte-order mark. Since cpphs is not intentionally introducing a BOM, it could be that the ghc + locale + platform which which cpphs is compiled, is responsible for introducing a BOM, and then the ghc which consumes the output is unprepared for it.

[asr]

Is there any way you can capture the output of cpphs before it gets to ghc

@jstolarek What about

$ cpphs Test.hs > Test.hspp
$ ghc Test.hspp

or using the -E option

$ ghc -cpp -pgmPcpphs -optP--cpp -E Test.hs
$ ghc Tests.hspp

?

[jstolarek]

@asr, using the first set of commands I get a file that looks like this:

#line 1 "Test.hs"
module Main where




foo = 42


main = print foo

This file compiles without problems. Using the -E option gives me this file (extension changed to txt because github won't allow hspp). Attempting to compile this file ends with the compilation error originally reported in this ticket.

[malcolmwallace]

OK, so the line that is giving ghc trouble is the one starting

    #line 1 "/dane/sandboxes/ghc/8.0.1

continuing with many spaces, and for which there is no terminating double-quote. When I run ghc -cpp -pgmPcpphs -optP--cpp -E, I do not see the same output as yours, and in particular this line mentioning sandboxes, with the missing quote terminator, is absent.

[jstolarek]

On my machine I have several GHC installation installed in the directory /dane/sandboxes/ghc, with each GHC version having its separate directory. In a sandbox directory (say, for 8.0.1) there are directories bin, lib and share (this is a directory into which I make installed GHC) as well as .cabal directory for all the stuff required by cabal. In /dane/sandboxes/ghc there is a symlink called active, which points to the currently active sandbox. In my home directory I have several symlinks: .ghc-sandbox points to /dane/sandboxes/ghc/active, .ghc points to .ghc-sandbox/.ghc/ and .cabal points to .ghc-sandbox/.cabal/. Finally, .ghc-sandbox/bin and .cabal/bin are on my PATH so that I can locate all the binaries installed within a sandbox.

My guess would be that the problem might be caused by handling of symlinks.

[asr]

FWIF, I have several versions installed of GHC and I use stow for activating a particular version. As consequence, the GHC executables are also handling via symlinks.

OK, so the line that is giving ghc trouble is the one starting continuing with many spaces, and for which there is no terminating double-quote

In my machine, that line is correctly generated

#line 1 "/usr/local/stow/ghc-8.0.1/lib/ghc-8.0.1/include/ghcversion.h"

@jstolarek could you remove the symlinks from the equation and test again?

[jstolarek]

@jstolarek could you remove the symlinks from the equation and test again?

I just tried doing two things:

1. Like I said, three of my directories are symlinks: .ghc-sandbox (this is GHC installation directory), .ghc and .cabal. I replaced them with proper directories and copied the contents of directories that the symlinks were pointing to. The result was the same as previously, ie. despite all GHC binaries being in .ghc-sandbox/bin the generated hspp file still pointed to location within /dane/sandboxes/ghc and the closing " was missing.
2. I deleted copied installation of GHC from .ghc-sandbox and installed fresh GHC installation into this directory. This time the result is correct:

#line 1 "/home/killy/.ghc-sandbox/lib/ghc-8.0.1/include/ghcversion.h"

Now, one thing that might be important. I install GHC by running:

./configure --prefix=$HOME/.ghc-sandbox
make install prefix=/dane/sandboxes/ghc/X.Y.Z

So I'm pointing the .configure script to the install symlink location, but when it comes to installing the files I'm using the actual location that the symlink will be pointing to. At the moment of running make install, the .ghc-sandbox symlink is pointing to another sandbox and I cannot change it to point to the new sandbox because then make install will fail (it requires ghc to be available).

[asr]

Thanks for testing! It's very import to know that the problem isn't related to your locale.

[asr]

I couldn't reproduce the problem creating $HOME/.ghc-sandbox and /dane/sandboxes/ghc/X.Y.Z following your above description.

[asr]

@jstolarek are you installing a source o binary GHC distribution?

[jstolarek]

I am using binary build for Debian 7.

[asr]

$ cpphs Test.hs > Test.hspp
$ ghc Test.hspp

$ ghc -cpp -pgmPcpphs -optP--cpp -E Test.hs
$ ghc Tests.hspp

Since the above tests success and fail respectively, I propose to file an issue on GHC. Are you agree? If so, which are your users on the GHC bug-tracker?

[malcolmwallace]

It does seem fairly clear that it is not cpphs's fault, and that ghc seems to be responsible for adding a #include with a filename that is not syntactically valid.

[jstolarek]

I am on GHC bugtracker as jstolarek

[asr]

@jstolarek, could you reproduce the issue with GHC 8.0.2?

[jstolarek]

I'll let you know once I upgrade to GHC 8.0.2. But was this reported as a GHC bug? If so, was this bug fixed by the 8.0.2 release?

[asr]

But was this reported as a GHC bug?

No. I didn't report it.

[asr]

Closing due to agda/agda#3223.