New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using CC to build C sources complicates cross-compilation #349

Closed
bgamari opened this Issue Jul 7, 2017 · 19 comments

Comments

Projects
5 participants
@bgamari
Collaborator

bgamari commented Jul 7, 2017

While testing cross compilation (#177) I encountered a failure wherein a stage0 C source file would fail to build,

/--------------------------------------------------------\
| Run Cc CompileC Stage0 (stage = Stage0, package = ghc) |
|      input: compiler/cbits/genSym.c                    |
|  => output: _build/stage0/compiler/c/cbits/genSym.o    |
\--------------------------------------------------------/
cc1: all warnings being treated as errors
shakeArgsWith   0.000s    0%                           
Function shake  0.317s    8%  ==                       
Database read   0.077s    1%                           
With database   0.005s    0%                           
Running rules   3.513s   89%  =========================
Total           3.912s  100%                           
Error when running Shake build system:
* inplace/bin/check-api-annotations
* _build/stage0/compiler/libHSghc-8.3.a
* _build/stage0/compiler/c/parser/cutils.o
user error (Development.Shake.cmd, system command failed
Command: /usr/bin/gcc -fno-stack-protector -Iincludes -I_build/generated -I_build/stage0/compiler -Icompiler/. -Icompiler/parser -Icompiler/utils -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/process-1.4.3.0/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/directory-1.3.0.0/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/unix-2.7.2.1/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/time-1.6.0.1/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/bytestring-0.10.8.1/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/base-4.9.1.0/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/integer-gmp-1.0.0.1/include -I/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/include -Werror -c compiler/parser/cutils.c -o _build/stage0/compiler/c/parser/cutils.o
Exit code: 1
Stderr:
In file included from includes/Rts.h:181:0,
                 from compiler/parser/cutils.c:6:
includes/rts/storage/Block.h: In function ‘Bdescr’:
includes/rts/storage/Block.h:175:8: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
     ((((W_)p &  MBLOCK_MASK & ~BLOCK_MASK) >> (BLOCK_SHIFT-BDESCR_SHIFT))
        ^

The problem here is that we are using the host's gcc with a hand-crafted set of include-paths to compile this file, whereas the old make build system would use the stage0 ghc,

"/mnt/work/ghc/roots/8.0.2/bin/ghc" -optc-fno-stack-protector -optc-Wall -optc-Icompiler/stage1/build/./autogen -optc-Icompiler/. -optc-Icompiler/parser -optc-Icompiler/utils -optc-Icompiler/stage1 -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/process-1.4.3.0/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/directory-1.3.0.0/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/unix-2.7.2.1/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/time-1.6.0.1/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/bytestring-0.10.8.1/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/base-4.9.1.0/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/integer-gmp-1.0.0.1/include' -optc-I'/mnt/work/ghc/roots/8.0.2/lib/ghc-8.0.2/include' -optc-Werror=unused-but-set-variable -optc-Wno-error=inline -static  -H32m -O -Wall  -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -package-db libraries/bootstrapping.conf  -this-unit-id ghc-8.3 -hide-all-packages -i -icompiler/backpack -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main -icompiler/nativeGen -icompiler/parser -icompiler/prelude -icompiler/profiling -icompiler/rename -icompiler/simplCore -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils -icompiler/vectorise -icompiler/stage1/build -Icompiler/stage1/build -icompiler/stage1/build/./autogen -Icompiler/stage1/build/./autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/stage1    -optP-include -optPcompiler/stage1/build/./autogen/cabal_macros.h -package-id base-4.9.1.0 -package-id deepseq-1.4.2.0 -package-id directory-1.3.0.0 -package-id process-1.4.3.0 -package-id bytestring-0.10.8.1 -package-id binary-0.8.4.1 -package-id time-1.6.0.1 -package-id containers-0.5.7.1 -package-id array-0.5.1.1 -package-id filepath-1.4.1.1 -package-id template-haskell-2.12.0.0 -package-id hpc-0.6.0.3 -package-id transformers-0.5.2.0 -package-id ghc-boot-8.3 -package-id ghc-boot-th-8.3 -package-id ghci-8.3 -package-id unix-2.7.2.1 -package-id terminfo-0.4.1.0 -Wall -fno-warn-name-shadowing -this-unit-id ghc -XHaskell2010 -DSTAGE=1 -Rghc-timing   -no-user-package-db -rtsopts        -c compiler/parser/cutils.c -o compiler/stage1/build/parser/cutils.o 

I believe what is happening here is that Hadrian gets the include path wrong, meaning that we end up picking up header files (e.g. ghcautoconf.h) from the tree being built (which is targeting ARM, in this case), whereas we want the headers provided by the stage0 compiler (which targets the host).

I don't think there is a good reason to deviate from the make build system here; it seems like the easiest way to solve this is to simply use the stage0 GHC to build C objects. To do otherwise would essentially require that we be able to anticipate what flags the stage0 GHC would pass to the C compiler, which sounds fragile at best.

@bgamari bgamari changed the title from Using cc to build C sources complicates cross-compilation to Using CC to build C sources complicates cross-compilation Jul 7, 2017

@bgamari

This comment has been minimized.

Show comment
Hide comment
@bgamari

bgamari Jul 7, 2017

Collaborator

Indeed changing Rules.Compile to use Ghc CompileHs instead of Cc CompileC appears to resolve the issue (except for some errors during dependency generation, although this makes sense as I didn't fix the dependency generation rules),

diff --git a/src/Rules/Compile.hs b/src/Rules/Compile.hs
index 2d3eb4a..c5155de 100644
--- a/src/Rules/Compile.hs
+++ b/src/Rules/Compile.hs
@@ -28,7 +28,7 @@ compilePackage rs context@Context {..} = do
             buildWithResources rs $ Target context (Ghc CompileHs stage) [src] [obj]
 
     priority 2.0 $ do
-        nonHs "c"   %> compile (Cc  CompileC ) (obj2src "c"   isGeneratedCFile  )
+        nonHs "c"   %> compile (Ghc CompileHs) (obj2src "c"   isGeneratedCFile  )
         nonHs "cmm" %> compile (Ghc CompileHs) (obj2src "cmm" isGeneratedCmmFile)
         nonHs "s"   %> compile (Ghc CompileHs) (obj2src "S"   $ const False     )
Collaborator

bgamari commented Jul 7, 2017

Indeed changing Rules.Compile to use Ghc CompileHs instead of Cc CompileC appears to resolve the issue (except for some errors during dependency generation, although this makes sense as I didn't fix the dependency generation rules),

diff --git a/src/Rules/Compile.hs b/src/Rules/Compile.hs
index 2d3eb4a..c5155de 100644
--- a/src/Rules/Compile.hs
+++ b/src/Rules/Compile.hs
@@ -28,7 +28,7 @@ compilePackage rs context@Context {..} = do
             buildWithResources rs $ Target context (Ghc CompileHs stage) [src] [obj]
 
     priority 2.0 $ do
-        nonHs "c"   %> compile (Cc  CompileC ) (obj2src "c"   isGeneratedCFile  )
+        nonHs "c"   %> compile (Ghc CompileHs) (obj2src "c"   isGeneratedCFile  )
         nonHs "cmm" %> compile (Ghc CompileHs) (obj2src "cmm" isGeneratedCmmFile)
         nonHs "s"   %> compile (Ghc CompileHs) (obj2src "S"   $ const False     )
@snowleopard

This comment has been minimized.

Show comment
Hide comment
@snowleopard

snowleopard Jul 8, 2017

Owner

My hope was to get rid of the magic built into the GHC, hence I was trying to build C files directly.

However, I agree that it may be best to stick to how the Make build system works.

@bgamari Could you take care of this issue? I don't know much about subtleties of cross compilation.

Owner

snowleopard commented Jul 8, 2017

My hope was to get rid of the magic built into the GHC, hence I was trying to build C files directly.

However, I agree that it may be best to stick to how the Make build system works.

@bgamari Could you take care of this issue? I don't know much about subtleties of cross compilation.

@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Jul 9, 2017

Collaborator

This is an interesting finding. I am looking into cross-compilation as well, so I'll keep track of this issue at the same time :)

Collaborator

izgzhen commented Jul 9, 2017

This is an interesting finding. I am looking into cross-compilation as well, so I'll keep track of this issue at the same time :)

@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Jul 10, 2017

Collaborator

The changes below can pass the compilation:

diff --git a/src/Rules/Compile.hs b/src/Rules/Compile.hs
index 2d3eb4a..c5155de 100644
--- a/src/Rules/Compile.hs
+++ b/src/Rules/Compile.hs
@@ -28,7 +28,7 @@ compilePackage rs context@Context {..} = do
             buildWithResources rs $ Target context (Ghc CompileHs stage) [src] [obj]
 
     priority 2.0 $ do
-        nonHs "c"   %> compile (Cc  CompileC ) (obj2src "c"   isGeneratedCFile  )
+        nonHs "c"   %> compile (Ghc CompileHs) (obj2src "c"   isGeneratedCFile  )
         nonHs "cmm" %> compile (Ghc CompileHs) (obj2src "cmm" isGeneratedCmmFile)
         nonHs "s"   %> compile (Ghc CompileHs) (obj2src "S"   $ const False     )
 
diff --git a/src/Settings/Packages/Rts.hs b/src/Settings/Packages/Rts.hs
index e278204..6e22226 100644
--- a/src/Settings/Packages/Rts.hs
+++ b/src/Settings/Packages/Rts.hs
@@ -49,7 +49,51 @@ rtsPackageArgs = package rts ? do
     ffiIncludeDir  <- getSetting FfiIncludeDir
     ffiLibraryDir  <- getSetting FfiLibDir
     mconcat
-        [ builder Cc ? mconcat
+        [ builder Ghc ? mconcat
+          [ arg "-optc-Irts"
+          , arg $ "-optc-I" ++ path
+          , arg $ "-optc-DRtsWay=\"rts_" ++ show way ++ "\""
+          -- rts *must* be compiled with optimisations. The INLINE_HEADER macro
+          -- requires that functions are inlined to work as expected. Inlining
+          -- only happens for optimised builds. Otherwise we can assume that
+          -- there is a non-inlined variant to use instead. But rts does not
+          -- provide non-inlined alternatives and hence needs the function to
+          -- be inlined. See also #90.
+          , arg "-optc-O2"
+
+          , Debug     `wayUnit` way          ? arg "-optc-DDEBUG"
+          , way `elem` [debug, debugDynamic] ? arg "-optc-DTICKY_TICKY"
+          , Profiling `wayUnit` way          ? arg "-optc-DPROFILING"
+          , Threaded  `wayUnit` way          ? arg "-optc-DTHREADED_RTS"
+
+          , inputs ["//RtsMessages.c", "//Trace.c"] ?
+            arg ("-optc-DProjectVersion=" ++ show projectVersion)
+
+          , input "//RtsUtils.c" ? append
+            [ "-optc-DProjectVersion="            ++ show projectVersion
+            , "-optc-DHostPlatform="              ++ show hostPlatform
+            , "-optc-DHostArch="                  ++ show hostArch
+            , "-optc-DHostOS="                    ++ show hostOs
+            , "-optc-DHostVendor="                ++ show hostVendor
+            , "-optc-DBuildPlatform="             ++ show buildPlatform
+            , "-optc-DBuildArch="                 ++ show buildArch
+            , "-optc-DBuildOS="                   ++ show buildOs
+            , "-optc-DBuildVendor="               ++ show buildVendor
+            , "-optc-DTargetPlatform="            ++ show targetPlatform
+            , "-optc-DTargetArch="                ++ show targetArch
+            , "-optc-DTargetOS="                  ++ show targetOs
+            , "-optc-DTargetVendor="              ++ show targetVendor
+            , "-optc-DGhcUnregisterised="         ++ show ghcUnreg
+            , "-optc-DGhcEnableTablesNextToCode=" ++ show ghcEnableTNC ]
+
+            , inputs ["//Evac.c", "//Evac_thr.c"] ? arg "-optc-funroll-loops"
+
+            , inputs ["//Evac_thr.c", "//Scav_thr.c"] ?
+              append [ "-optc-DPARALLEL_GC", "-optc-Irts/sm" ]
+
+            , input "//StgCRun.c" ? windowsHost ? arg "-optc-Wno-return-local-addr"
+          ]
+        , builder Cc ? mconcat
           [ arg "-Irts"
           , arg $ "-I" ++ path
           , arg $ "-DRtsWay=\"rts_" ++ show way ++ "\""

I didn't delete the Cc's duplicated arguments because Cc FindCDependencies also need them.

Collaborator

izgzhen commented Jul 10, 2017

The changes below can pass the compilation:

diff --git a/src/Rules/Compile.hs b/src/Rules/Compile.hs
index 2d3eb4a..c5155de 100644
--- a/src/Rules/Compile.hs
+++ b/src/Rules/Compile.hs
@@ -28,7 +28,7 @@ compilePackage rs context@Context {..} = do
             buildWithResources rs $ Target context (Ghc CompileHs stage) [src] [obj]
 
     priority 2.0 $ do
-        nonHs "c"   %> compile (Cc  CompileC ) (obj2src "c"   isGeneratedCFile  )
+        nonHs "c"   %> compile (Ghc CompileHs) (obj2src "c"   isGeneratedCFile  )
         nonHs "cmm" %> compile (Ghc CompileHs) (obj2src "cmm" isGeneratedCmmFile)
         nonHs "s"   %> compile (Ghc CompileHs) (obj2src "S"   $ const False     )
 
diff --git a/src/Settings/Packages/Rts.hs b/src/Settings/Packages/Rts.hs
index e278204..6e22226 100644
--- a/src/Settings/Packages/Rts.hs
+++ b/src/Settings/Packages/Rts.hs
@@ -49,7 +49,51 @@ rtsPackageArgs = package rts ? do
     ffiIncludeDir  <- getSetting FfiIncludeDir
     ffiLibraryDir  <- getSetting FfiLibDir
     mconcat
-        [ builder Cc ? mconcat
+        [ builder Ghc ? mconcat
+          [ arg "-optc-Irts"
+          , arg $ "-optc-I" ++ path
+          , arg $ "-optc-DRtsWay=\"rts_" ++ show way ++ "\""
+          -- rts *must* be compiled with optimisations. The INLINE_HEADER macro
+          -- requires that functions are inlined to work as expected. Inlining
+          -- only happens for optimised builds. Otherwise we can assume that
+          -- there is a non-inlined variant to use instead. But rts does not
+          -- provide non-inlined alternatives and hence needs the function to
+          -- be inlined. See also #90.
+          , arg "-optc-O2"
+
+          , Debug     `wayUnit` way          ? arg "-optc-DDEBUG"
+          , way `elem` [debug, debugDynamic] ? arg "-optc-DTICKY_TICKY"
+          , Profiling `wayUnit` way          ? arg "-optc-DPROFILING"
+          , Threaded  `wayUnit` way          ? arg "-optc-DTHREADED_RTS"
+
+          , inputs ["//RtsMessages.c", "//Trace.c"] ?
+            arg ("-optc-DProjectVersion=" ++ show projectVersion)
+
+          , input "//RtsUtils.c" ? append
+            [ "-optc-DProjectVersion="            ++ show projectVersion
+            , "-optc-DHostPlatform="              ++ show hostPlatform
+            , "-optc-DHostArch="                  ++ show hostArch
+            , "-optc-DHostOS="                    ++ show hostOs
+            , "-optc-DHostVendor="                ++ show hostVendor
+            , "-optc-DBuildPlatform="             ++ show buildPlatform
+            , "-optc-DBuildArch="                 ++ show buildArch
+            , "-optc-DBuildOS="                   ++ show buildOs
+            , "-optc-DBuildVendor="               ++ show buildVendor
+            , "-optc-DTargetPlatform="            ++ show targetPlatform
+            , "-optc-DTargetArch="                ++ show targetArch
+            , "-optc-DTargetOS="                  ++ show targetOs
+            , "-optc-DTargetVendor="              ++ show targetVendor
+            , "-optc-DGhcUnregisterised="         ++ show ghcUnreg
+            , "-optc-DGhcEnableTablesNextToCode=" ++ show ghcEnableTNC ]
+
+            , inputs ["//Evac.c", "//Evac_thr.c"] ? arg "-optc-funroll-loops"
+
+            , inputs ["//Evac_thr.c", "//Scav_thr.c"] ?
+              append [ "-optc-DPARALLEL_GC", "-optc-Irts/sm" ]
+
+            , input "//StgCRun.c" ? windowsHost ? arg "-optc-Wno-return-local-addr"
+          ]
+        , builder Cc ? mconcat
           [ arg "-Irts"
           , arg $ "-I" ++ path
           , arg $ "-DRtsWay=\"rts_" ++ show way ++ "\""

I didn't delete the Cc's duplicated arguments because Cc FindCDependencies also need them.

@snowleopard

This comment has been minimized.

Show comment
Hide comment
@snowleopard

snowleopard Jul 10, 2017

Owner

@izgzhen In the above code, you probably want to say builder (Ghc CompileC) instead of builder Ghc and builder (Ghc CompileHs)? That is, we probably need to add a new mode CompileC into GhcMode.

Owner

snowleopard commented Jul 10, 2017

@izgzhen In the above code, you probably want to say builder (Ghc CompileC) instead of builder Ghc and builder (Ghc CompileHs)? That is, we probably need to add a new mode CompileC into GhcMode.

@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Jul 12, 2017

Collaborator

That is, we probably need to add a new mode CompileC into GhcMode.

Sure, will do later

Collaborator

izgzhen commented Jul 12, 2017

That is, we probably need to add a new mode CompileC into GhcMode.

Sure, will do later

@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Jul 12, 2017

Collaborator

The current problem is, after stage0 completed with above patch, we need to build some stage1 packages, and Hadrian errors here:

inplace/bin/ghc-stage1 -Wall -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-this-unit-id rts' -i -i_build/stage1/rts -i_build/stage1/rts/build/autogen -Iincludes -I_build/generated -I_build/stage1/rts -I_build/generated -optc-I_build/generated -optc-marm -optc-fno-stack-protector -odir _build/stage1/rts -hidir _build/stage1/rts -stubdir _build/stage1/rts -c rts/linker/elf_reloc_aarch64.c -o _build/stage1/rts/c/linker/elf_reloc_aarch64.o -O0 -H32m -optc-Irts -optc-I_build/stage1/rts '-optc-DRtsWay="rts_v"' -optc-O2 -Irts -Wno-deprecated-flags

In file included from includes/Rts.h:181:0: error:
    0,
                     from rts/LinkerInternals.h:11,
                     from rts/linker/elf_reloc_aarch64.h:3,
                     from rts/linker/elf_reloc_aarch64.c:4:
includes/rts/storage/Block.h: In function ‘Bdescr’:

includes/rts/storage/Block.h:175:8: error:
     warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
         ((((W_)p &  MBLOCK_MASK & ~BLOCK_MASK) >> (BLOCK_SHIFT-BDESCR_SHIFT))
            ^
    |
175 |     ((((W_)p &  MBLOCK_MASK & ~BLOCK_MASK) >> (BLOCK_SHIFT-BDESCR_SHIFT))
    |        ^
Collaborator

izgzhen commented Jul 12, 2017

The current problem is, after stage0 completed with above patch, we need to build some stage1 packages, and Hadrian errors here:

inplace/bin/ghc-stage1 -Wall -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-this-unit-id rts' -i -i_build/stage1/rts -i_build/stage1/rts/build/autogen -Iincludes -I_build/generated -I_build/stage1/rts -I_build/generated -optc-I_build/generated -optc-marm -optc-fno-stack-protector -odir _build/stage1/rts -hidir _build/stage1/rts -stubdir _build/stage1/rts -c rts/linker/elf_reloc_aarch64.c -o _build/stage1/rts/c/linker/elf_reloc_aarch64.o -O0 -H32m -optc-Irts -optc-I_build/stage1/rts '-optc-DRtsWay="rts_v"' -optc-O2 -Irts -Wno-deprecated-flags

In file included from includes/Rts.h:181:0: error:
    0,
                     from rts/LinkerInternals.h:11,
                     from rts/linker/elf_reloc_aarch64.h:3,
                     from rts/linker/elf_reloc_aarch64.c:4:
includes/rts/storage/Block.h: In function ‘Bdescr’:

includes/rts/storage/Block.h:175:8: error:
     warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
         ((((W_)p &  MBLOCK_MASK & ~BLOCK_MASK) >> (BLOCK_SHIFT-BDESCR_SHIFT))
            ^
    |
175 |     ((((W_)p &  MBLOCK_MASK & ~BLOCK_MASK) >> (BLOCK_SHIFT-BDESCR_SHIFT))
    |        ^
@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Jul 12, 2017

Collaborator

I checked the old system's way of building, in this case, Hash.o, it is:

"inplace/bin/ghc-stage1" -optc-marm -optc-fno-stack-protector -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-DUSE_LIBFFI_FOR_ADJUSTORS -optc-fno-strict-aliasing -optc-fno-common -optc-Irts/dist/build/./autogen -optc-Werror=unused-but-set-variable -optc-Wno-error=inline -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-DRtsWay=\"rts_v\" -static  -O0 -H64m -Wall   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -this-unit-id rts -dcmm-lint      -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen           -O2    -Wnoncanonical-monad-instances  -c rts/Hash.c -o rts/dist/build/Hash.o

However, if I run the same command in Hadrian's tree, it will output the same error.

Collaborator

izgzhen commented Jul 12, 2017

I checked the old system's way of building, in this case, Hash.o, it is:

"inplace/bin/ghc-stage1" -optc-marm -optc-fno-stack-protector -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-DUSE_LIBFFI_FOR_ADJUSTORS -optc-fno-strict-aliasing -optc-fno-common -optc-Irts/dist/build/./autogen -optc-Werror=unused-but-set-variable -optc-Wno-error=inline -optc-O2 -optc-fomit-frame-pointer -optc-g -optc-DRtsWay=\"rts_v\" -static  -O0 -H64m -Wall   -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -this-unit-id rts -dcmm-lint      -i -irts -irts/dist/build -Irts/dist/build -irts/dist/build/./autogen -Irts/dist/build/./autogen           -O2    -Wnoncanonical-monad-instances  -c rts/Hash.c -o rts/dist/build/Hash.o

However, if I run the same command in Hadrian's tree, it will output the same error.

@snowleopard

This comment has been minimized.

Show comment
Hide comment
@snowleopard

snowleopard Jul 12, 2017

Owner

This means the difference is in the environment -- perhaps, we have a bug in one of the generated C headers? There is a lot of CPP and incorrect settings in headers may break CPP conditionals.

Owner

snowleopard commented Jul 12, 2017

This means the difference is in the environment -- perhaps, we have a bug in one of the generated C headers? There is a lot of CPP and incorrect settings in headers may break CPP conditionals.

@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Jul 14, 2017

Collaborator

perhaps, we have a bug in one of the generated C headers?

Probably. Although I failed to extract compile-time information about the exact size of StgWord and W_, I manually modified includes/stg/Types.h and includes/Cmm.h. It looks like the warnings about size incompatibility disappeared, but now there will be seemly random segfault core dump:

Error when running Shake build system:
* _build/stage1/libraries/Cabal/Cabal/HSCabal-2.0.0.0.o
* _build/stage1/libraries/Cabal/Cabal/Distribution/Simple/Program/Ar.o
* _build/stage1/libraries/Cabal/Cabal/Distribution/Simple/Program/Ar.o _build/stage1/libraries/Cabal/Cabal/Distribution/Simple/Program/Ar.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Char8.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Char8.o _build/stage1/libraries/bytestring/Data/ByteString/Char8.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Internal.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Internal.o _build/stage1/libraries/bytestring/Data/ByteString/Internal.hi
* _build/stage1/libraries/ghc-prim/GHC/CString.hi
* _build/stage1/libraries/ghc-prim/GHC/CString.o _build/stage1/libraries/ghc-prim/GHC/CString.hi
* _build/stage1/libraries/ghc-prim/GHC/Types.hi
* _build/stage1/libraries/ghc-prim/GHC/Types.o _build/stage1/libraries/ghc-prim/GHC/Types.hi
* _build/stage1/rts/libHSrts.a
* _build/stage1/rts/c/ClosureFlags.o
user error (Development.Shake.cmd, system command failed
Command: inplace/bin/ghc-stage1 -Wall -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-this-unit-id rts' -i -i_build/stage1/rts -i_build/stage1/rts/build/autogen -Iincludes -I_build/generated -I_build/stage1/rts -I_build/generated -optc-I_build/generated -optc-marm -optc-fno-stack-protector -odir _build/stage1/rts -hidir _build/stage1/rts -stubdir _build/stage1/rts -c rts/ClosureFlags.c -o _build/stage1/rts/c/ClosureFlags.o -O0 -H32m -optc-Irts -optc-I_build/stage1/rts '-optc-DRtsWay="rts_v"' -optc-O2 -Irts -Wno-deprecated-flags
Exit code: -11
Stderr:
)

By random, I mean the _build/stage1/rts/c/ClosureFlags.o here can be another *.o in RTS.

Collaborator

izgzhen commented Jul 14, 2017

perhaps, we have a bug in one of the generated C headers?

Probably. Although I failed to extract compile-time information about the exact size of StgWord and W_, I manually modified includes/stg/Types.h and includes/Cmm.h. It looks like the warnings about size incompatibility disappeared, but now there will be seemly random segfault core dump:

Error when running Shake build system:
* _build/stage1/libraries/Cabal/Cabal/HSCabal-2.0.0.0.o
* _build/stage1/libraries/Cabal/Cabal/Distribution/Simple/Program/Ar.o
* _build/stage1/libraries/Cabal/Cabal/Distribution/Simple/Program/Ar.o _build/stage1/libraries/Cabal/Cabal/Distribution/Simple/Program/Ar.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Char8.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Char8.o _build/stage1/libraries/bytestring/Data/ByteString/Char8.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Internal.hi
* _build/stage1/libraries/bytestring/Data/ByteString/Internal.o _build/stage1/libraries/bytestring/Data/ByteString/Internal.hi
* _build/stage1/libraries/ghc-prim/GHC/CString.hi
* _build/stage1/libraries/ghc-prim/GHC/CString.o _build/stage1/libraries/ghc-prim/GHC/CString.hi
* _build/stage1/libraries/ghc-prim/GHC/Types.hi
* _build/stage1/libraries/ghc-prim/GHC/Types.o _build/stage1/libraries/ghc-prim/GHC/Types.hi
* _build/stage1/rts/libHSrts.a
* _build/stage1/rts/c/ClosureFlags.o
user error (Development.Shake.cmd, system command failed
Command: inplace/bin/ghc-stage1 -Wall -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-this-unit-id rts' -i -i_build/stage1/rts -i_build/stage1/rts/build/autogen -Iincludes -I_build/generated -I_build/stage1/rts -I_build/generated -optc-I_build/generated -optc-marm -optc-fno-stack-protector -odir _build/stage1/rts -hidir _build/stage1/rts -stubdir _build/stage1/rts -c rts/ClosureFlags.c -o _build/stage1/rts/c/ClosureFlags.o -O0 -H32m -optc-Irts -optc-I_build/stage1/rts '-optc-DRtsWay="rts_v"' -optc-O2 -Irts -Wno-deprecated-flags
Exit code: -11
Stderr:
)

By random, I mean the _build/stage1/rts/c/ClosureFlags.o here can be another *.o in RTS.

@izgzhen

This comment has been minimized.

Show comment
Hide comment
@izgzhen

izgzhen Sep 1, 2017

Collaborator

I think this is already resolved.

Collaborator

izgzhen commented Sep 1, 2017

I think this is already resolved.

@izgzhen izgzhen closed this Sep 1, 2017

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Oct 5, 2017

The current configure script gets its CC env vars a bit confused. I'd like to fix that at some point, in which case it would probably be easy to go back to not using GHC here, which I think would be good. I'd hope the same change wouldn't be too hard in the old build system too, so it could be done soon.

Ericson2314 commented Oct 5, 2017

The current configure script gets its CC env vars a bit confused. I'd like to fix that at some point, in which case it would probably be easy to go back to not using GHC here, which I think would be good. I'd hope the same change wouldn't be too hard in the old build system too, so it could be done soon.

@snowleopard

This comment has been minimized.

Show comment
Hide comment
@snowleopard

snowleopard Oct 5, 2017

Owner

it would probably be easy to go back to not using GHC here, which I think would be good

@Ericson2314 That would be nice! At the moment the use of GHC slows down the build, because we have to wait for building Stage1 GHC to compile C files.

Owner

snowleopard commented Oct 5, 2017

it would probably be easy to go back to not using GHC here, which I think would be good

@Ericson2314 That would be nice! At the moment the use of GHC slows down the build, because we have to wait for building Stage1 GHC to compile C files.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Oct 5, 2017

Neat, so cross and native alike would benefit. In general, my first priority is upstreaming some nixpkgs changes for cross-building ghc I've already done, which has taken a while and could take a while longer. But I'd be happy to illustrate the changes needed / just do the configure script part, more immediately.

Ericson2314 commented Oct 5, 2017

Neat, so cross and native alike would benefit. In general, my first priority is upstreaming some nixpkgs changes for cross-building ghc I've already done, which has taken a while and could take a while longer. But I'd be happy to illustrate the changes needed / just do the configure script part, more immediately.

@ggreif

This comment has been minimized.

Show comment
Hide comment
@ggreif

ggreif Oct 5, 2017

Contributor

Why not teach ghc to output command ... args for non-HS compilation of files. Then hadrian simply would execute those.

Of course this feature should be part of the bootstrapghc sometime.

Contributor

ggreif commented Oct 5, 2017

Why not teach ghc to output command ... args for non-HS compilation of files. Then hadrian simply would execute those.

Of course this feature should be part of the bootstrapghc sometime.

@snowleopard

This comment has been minimized.

Show comment
Hide comment
@snowleopard

snowleopard Oct 5, 2017

Owner

@ggreif Interesting idea. Indeed, GHC does need to know how to compile non-HS files, because it has to do it when compiling multi-language projects, so it might as well share this information with the build system. (Or maybe there should be a different source of this information, used both by GHC and Hadrian.)

Owner

snowleopard commented Oct 5, 2017

@ggreif Interesting idea. Indeed, GHC does need to know how to compile non-HS files, because it has to do it when compiling multi-language projects, so it might as well share this information with the build system. (Or maybe there should be a different source of this information, used both by GHC and Hadrian.)

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Oct 5, 2017

Yeah, when cross compiling, I would not trust the stage0 GHC to know about how to properly invoke the stage1 CC. Perhaps I don't understand this "multi-language project" case very well, but I'd think that would be a thing for Cabal, too.

Ericson2314 commented Oct 5, 2017

Yeah, when cross compiling, I would not trust the stage0 GHC to know about how to properly invoke the stage1 CC. Perhaps I don't understand this "multi-language project" case very well, but I'd think that would be a thing for Cabal, too.

@snowleopard

This comment has been minimized.

Show comment
Hide comment
@snowleopard

snowleopard Oct 5, 2017

Owner

I don't understand this "multi-language project" case very well

I simply meant you can do things like ghc --make Main.hs lib.c and GHC will compile both Haskell and C files. For anything more advanced one would certainly use Cabal.

Owner

snowleopard commented Oct 5, 2017

I don't understand this "multi-language project" case very well

I simply meant you can do things like ghc --make Main.hs lib.c and GHC will compile both Haskell and C files. For anything more advanced one would certainly use Cabal.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Oct 5, 2017

Ah, right. I'd love to deprecate that :).

Ericson2314 commented Oct 5, 2017

Ah, right. I'd love to deprecate that :).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment