Skip to content

Will cwrap work with Nim? #2

Open
Open
Will cwrap work with Nim?#2
@simonhf

Description

@simonhf
simonhf
opened on Jul 23, 2022

Yes, here's an example:

  • Clone and build Nim:
$ git clone https://github.com/nim-lang/Nim.git
$ pushd ./Nim/
$ time ./build_all.sh
...
real    4m24.324s
$ popd

$ ./Nim/bin/nim --version | egrep -i nim
Nim Compiler Version 1.7.1 [Linux: amd64]
  • Create an example Nim source file, and build and run it without cwrap:
$ cat helloworld.nim 
proc foo(): void =
  echo "Hello World from foo()"

proc bar(): void =
  foo()

echo "Hello World 2"
bar()

$ ./Nim/bin/nim c -r --verbosity:2 helloworld.nim
...
CC: helloworld.nim: gcc -c  -w -fmax-errors=3 -pthread   -I/home/simon/work/20220722-nim/Nim/lib -I/home/simon/work/20220722-nim -o /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c
Hint: gcc   -o /home/simon/work/20220722-nim/helloworld  /home/simon/.cache/nim/helloworld_d/@mNim@slib@sstd@sprivate@sdigitsutils.nim.c.o /home/simon/.cache/nim/helloworld_d/@mNim@slib@ssystem@sdollars.nim.c.o /home/simon/.cache/nim/helloworld_d/@mNim@slib@spure@scollections@ssharedlist.nim.c.o /home/simon/.cache/nim/helloworld_d/@mNim@slib@sstd@ssyncio.nim.c.o /home/simon/.cache/nim/helloworld_d/@mNim@slib@ssystem.nim.c.o /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o  -pthread   -ldl [Link]
/home/simon/work/20220722-nim/Nim/compiler/extccomp.nim(402, 13) compiler msg initiated here [MsgOrigin]
...
Hello World 2
Hello World from foo()

$ wc -c helloworld
116680 helloworld
  • Try to tell Nim to use a different compiler:
$ ./Nim/bin/nim c --cc:xyz -r --verbosity:2 helloworld.nim
...
command line(1, 2) Error: unknown C compiler: 'xyz'. Available options are: gcc, switch_gcc, llvm_gcc, clang, bcc, vcc, tcc, env, icl, icc, clang_cl
  • To get cwrap to wrap Nim compilation, we're going to have to trick Nim into thinking we want to to use one of its supported compilers, e.g. icc:
$ which icc
$

$ ./Nim/bin/nim c --cc:icc -r --verbosity:2 helloworld.nim
...
Error: execution of an external compiler program 'icc -c  -pthread   -I/home/simon/work/20220722-nim/Nim/lib -I/home/simon/work/20220722-nim -o /home/simon/.cache/nim/helloworld_d/@mNim@slib@sstd@sprivate@sdigitsutils.nim.c.o /home/simon/.cache/nim/helloworld_d/@mNim@slib@sstd@sprivate@sdigitsutils.nim.c' failed with exit code: 127
  • Create a dummy icc "compiler" for Nim to run:
$ PATH=$PATH:`pwd`

$ cat icc 
echo "hello from icc"
exit 1

$ chmod +x icc

$ ./Nim/bin/nim c --cc:icc --forceBuild:on -r --verbosity:2 helloworld.nim
...
Error: execution of an external compiler program 'icc -c  -pthread   -I/home/simon/work/20220722-nim/Nim/lib -I/home/simon/work/20220722-nim -o /home/simon/.cache/nim/helloworld_d/@mNim@slib@sstd@sprivate@sdigitsutils.nim.c.o /home/simon/.cache/nim/helloworld_d/@mNim@slib@sstd@sprivate@sdigitsutils.nim.c' failed with exit code: 1
...
hello from icc
  • Clone cwrap and install its dependencies:
$ git clone $ https://github.com/corelight/cwrap.git

$ sudo apt-get install liblist-moreutils-perl

$ sudo apt install cpanminus

$ sudo cpanm FFI::Platypus::Lang::CPP::Demangle::XS

$ sudo apt install llvm
$ which llvm-cxxfilt
/usr/bin/llvm-cxxfilt
  • Modify icc fake compiler to point to cwrap (which uses gcc under the covers):
$ cat icc
`pwd`/cwrap/cwrap.pl "$@"
  • Hot patch a small bug in cwrap:
$ git diff
diff --git a/cwrap.pl b/cwrap.pl
-        $gcc_4_s =~ s~$source_file_argument~$s_file.2.s~;
+        if    ($gcc_4_s =~ s~$source_file_argument[ ]~$s_file.2.s ~) {} # if source file is not at the end of the command line
+        elsif ($gcc_4_s =~ s~$source_file_argument$~$s_file.2.s~) {}    # if source file is     at the end of the command line
+        else { cwrap_die(sprintf qq[%f ERROR: cwrap regex failed to substitute source files; gcc_4_s=%s!\n], Time::HiRes::time() - $ts, $gcc_4_s); }
  • Tell Nim to force re-build helloworld.nim using icc:
$ ./Nim/bin/nim c --cc:icc --forceBuild:on -r --verbosity:2 helloworld.nim
...
Hello World 2
Hello World from foo()

$ wc -c helloworld
226696 helloworld
  • helloworld executable is now bigger because 100s of functions are instrumented by cwrap.
  • Run helloworld and ask cwrap to list all the instruction function (note: many functions are grepped away for brevity):
$ CWRAP_LOG_SHOW=1 ./helloworld | egrep -v "(system|sdigitsutils|ssyncio|ssharedlist)"
C0 + cwrap_log_show() { #1
C0   - func_addr=(nil)
C0   - #1: verbosity 9 for 1 of 1 function variation(s) for cwrap_log_show() from helloworld.cwrap.c
C0   - #2: verbosity 9 for 1 of 1 function variation(s) for cwrap_log_stats() from helloworld.cwrap.c
C0   - #3: verbosity 9 for 1 of 1 function variation(s) for cwrap_log_verbosity_set() from helloworld.cwrap.c
C0   - #4: verbosity 9 for 1 of 1 function variation(s) for cwrap_log_quiet_until() from helloworld.cwrap.c
C0   - #107: verbosity 9 for 1 of 1 function variation(s) for main() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #126: verbosity 9 for 1 of 1 function variation(s) for initStackBottomWith() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #153: verbosity 9 for 1 of 1 function variation(s) for foo__helloworld_1() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #194: verbosity 9 for 1 of 1 function variation(s) for bar__helloworld_2() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #225: verbosity 9 for 1 of 1 function variation(s) for PreMainInner() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #226: verbosity 9 for 1 of 1 function variation(s) for PreMain() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #227: verbosity 9 for 1 of 1 function variation(s) for NimMainModule() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #228: verbosity 9 for 1 of 1 function variation(s) for NimMainInner() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   - #229: verbosity 9 for 1 of 1 function variation(s) for NimMain() from /home/simon/.cache/nim/helloworld_d/@mhelloworld.nim.c.o
C0   } // cwrap_log_show()
  • Run helloworld and ask cwrap to run with full verbosity except for functions in ssystem.nim.c and sdigitsutils.nim.c:
$ CWRAP_LOG_STATS=1 CWRAP_LOG_NUM=1 CWRAP_LOG_TIMESTAMP=1 CWRAP_LOG_THREAD_ID=1 CWRAP_LOG_CURT=1 CWRAP_LOG_VERBOSITY_SET=1/9=file-ssystem.nim.c/9=file-sdigitsutils.nim.c ./helloworld | less +G
cwrap_log_init() {} // CWRAP_LOG: _VERBOSITY_SET=1/9=file-ssystem.nim.c/9=file-sdigitsutils.nim.c (<verbosity>[={file|function}-<keyword>][/...]) _QUIET_UNTIL=(null) _STATS=1 _SHOW=0 _CURT=1 _FILE=0 _NUM=1 _COR_ID=1 _LIMIT=10,000 _THREAD_ID=1 _STACK_PTR=0
#1 T06822 C0 0.000011s + cwrap_log_verbosity_set(verbosity=1/9=file-ssystem.nim.c/9=file-sdigitsutils.nim.c) { // #1 [cwrap_log_verbosity_set() ignores verbosity!]
#2 T06822 C0 0.000029s   - verbosity 1 set for 240 matches in 240 functions for 1 byte clause '1' // type=FILE|FUNCTION keyword=(null)
#3 T06822 C0 0.000031s   - verbosity 9 set for 210 matches in 240 functions for 20 byte clause '9=file-ssystem.nim.c' // type=FILE keyword=ssystem.nim.c
#4 T06822 C0 0.000032s   - verbosity 9 set for 10 matches in 240 functions for 25 byte clause '9=file-sdigitsutils.nim.c' // type=FILE keyword=sdigitsutils.nim.c
#5 T06822 C0 0.000034s   } // cwrap_log_verbosity_set() 
#6 T06822 C0 0.000036s + main() { // #1 
#7 T06822 C0 0.000039s   + NimMain() { // #1 
#8 T06822 C0 0.000040s     + PreMain() { // #1 
#9 T06822 C0 0.000042s       + initStackBottomWith() {} // #1 
#10 T06822 C0 0.000102s       + init__system_6081() { // #1 
#11 T06822 C0 0.000105s         + initLock__coreZlocks_64() {} // #1 
#12 T06822 C0 0.000106s         } // init__system_6081() 
#13 T06822 C0 0.000109s       + PreMainInner() {} // #1 
#14 T06822 C0 0.000110s       } // PreMain() 
#15 T06822 C0 0.000111s     + initStackBottomWith() {} // #2 
#16 T06822 C0 0.000112s     + NimMainInner() { // #1 
#17 T06822 C0 0.000113s       + NimMainModule() { // #1 
#18 T06822 C0 0.000114s         + echoBinSafe() { // #1 
#19 T06822 C0 0.000115s           + nimToCStringConv() {} // #1 
Hello World 2
#20 T06822 C0 0.000120s           } // echoBinSafe() 
#21 T06822 C0 0.000121s         + bar__helloworld_2() { // #1 
#22 T06822 C0 0.000122s           + foo__helloworld_1() { // #1 
#23 T06822 C0 0.000122s             + echoBinSafe() { // #2 
#24 T06822 C0 0.000123s               + nimToCStringConv() {} // #2 
Hello World from foo()
#25 T06822 C0 0.000125s               } // echoBinSafe() 
#26 T06822 C0 0.000126s             } // foo__helloworld_1() 
#27 T06822 C0 0.000126s           } // bar__helloworld_2() 
#28 T06822 C0 0.000127s         } // NimMainModule() 
#29 T06822 C0 0.000128s       } // NimMainInner() 
#30 T06822 C0 0.000129s     } // NimMain() 
#31 T06822 C0 0.000129s   } // main() 
#32 T06822 C0 0.000131s + cwrap_log_stats() { // #1 [cwrap_log_stats() ignores verbosity!]
#33 T06822 C0 0.000132s   - 1 calls to 1 of 1 function variation(s) for cwrap_log_stats()
#34 T06822 C0 0.000133s   - 1 calls to 1 of 1 function variation(s) for cwrap_log_verbosity_set()
#35 T06822 C0 0.000135s   - 2 calls to 1 of 1 function variation(s) for nimToCStringConv()
#36 T06822 C0 0.000136s   - 1 calls to 1 of 1 function variation(s) for main()
#37 T06822 C0 0.000137s   - 1 calls to 1 of 1 function variation(s) for init__system_6081()
#38 T06822 C0 0.000138s   - 2 calls to 1 of 1 function variation(s) for initStackBottomWith()
#39 T06822 C0 0.000139s   - 1 calls to 1 of 1 function variation(s) for initLock__coreZlocks_64()
#40 T06822 C0 0.000140s   - 1 calls to 1 of 1 function variation(s) for foo__helloworld_1()
#41 T06822 C0 0.000140s   - 2 calls to 1 of 1 function variation(s) for echoBinSafe()
#42 T06822 C0 0.000142s   - 1 calls to 1 of 1 function variation(s) for bar__helloworld_2()
#43 T06822 C0 0.000143s   - 1 calls to 1 of 1 function variation(s) for PreMainInner()
#44 T06822 C0 0.000143s   - 1 calls to 1 of 1 function variation(s) for PreMain()
#45 T06822 C0 0.000144s   - 1 calls to 1 of 1 function variation(s) for NimMainModule()
#46 T06822 C0 0.000145s   - 1 calls to 1 of 1 function variation(s) for NimMainInner()
#47 T06822 C0 0.000146s   - 1 calls to 1 of 1 function variation(s) for NimMain()
#48 T06822 C0 0.000147s   - 18 calls to 15 of 240 functions instrumented
#49 T06822 C0 0.000148s   } // cwrap_log_stats()
  • We get to see a hint into the inner life of the helloworld Nim executable.
  • Run helloworld and ask cwrap to run with no verbosity until function NimMainInner() is called for the first time.
  • Note: Now we can see a but more detail, e.g. previously hidden functions, e.g. nimFrame():
$ CWRAP_LOG_QUIET_UNTIL=NimMainInner CWRAP_LOG_STATS=1 CWRAP_LOG_NUM=1 CWRAP_LOG_TIMESTAMP=1 CWRAP_LOG_THREAD_ID=1 CWRAP_LOG_CURT=1 CWRAP_LOG_VERBOSITY_SET=1 ./helloworld
cwrap_log_init() {} // CWRAP_LOG: _VERBOSITY_SET=1 (<verbosity>[={file|function}-<keyword>][/...]) _QUIET_UNTIL=NimMainInner _STATS=1 _SHOW=0 _CURT=1 _FILE=0 _NUM=1 _COR_ID=1 _LIMIT=10,000 _THREAD_ID=1 _STACK_PTR=0 _TIMESTAMP=1 _UNWIND=0 _ON_VALGRIND=0
#1 T06837 C0 0.000036s + cwrap_log_verbosity_set(verbosity=1) { // #1 [cwrap_log_verbosity_set() ignores verbosity!]
#2 T06837 C0 0.000055s   - verbosity 1 set for 240 matches in 240 functions for 1 byte clause '1' // type=FILE|FUNCTION keyword=(null)
#3 T06837 C0 0.000062s   } // cwrap_log_verbosity_set() 
#4 T06837 C0 0.000064s + cwrap_log_quiet_until(name=NimMainInner) {} // #1 going quiet until function NimMainInner() [cwrap_log_quiet_until() ignores verbosity!]
#5 T06837 C0 0.000127s + NimMainInner() { // #1 
#6 T06837 C0 0.000129s   + NimMainModule() { // #1 
#7 T06837 C0 0.000147s     + nimFrame() {} // #1 
#8 T06837 C0 0.000150s     + echoBinSafe() { // #1 
#9 T06837 C0 0.000151s       + nimFrame() {} // #2 
#10 T06837 C0 0.000154s       + nimToCStringConv() {} // #1 
Hello World 2
#11 T06837 C0 0.000156s       + nimAddInt() {} // #1 
#12 T06837 C0 0.000158s       + popFrame() {} // #1 
#13 T06837 C0 0.000159s       } // echoBinSafe() 
#14 T06837 C0 0.000161s     + bar__helloworld_2() { // #1 
#15 T06837 C0 0.000162s       + nimFrame() {} // #3 
#16 T06837 C0 0.000164s       + foo__helloworld_1() { // #1 
#17 T06837 C0 0.000165s         + nimFrame() {} // #4 
#18 T06837 C0 0.000167s         + echoBinSafe() { // #2 
#19 T06837 C0 0.000168s           + nimFrame() {} // #5 
#20 T06837 C0 0.000170s           + nimToCStringConv() {} // #2 
Hello World from foo()
#21 T06837 C0 0.000172s           + nimAddInt() {} // #2 
#22 T06837 C0 0.000174s           + popFrame() {} // #2 
#23 T06837 C0 0.000175s           } // echoBinSafe() 
#24 T06837 C0 0.000177s         + popFrame() {} // #3 
#25 T06837 C0 0.000178s         } // foo__helloworld_1() 
#26 T06837 C0 0.000180s       + popFrame() {} // #4 
#27 T06837 C0 0.000181s       } // bar__helloworld_2() 
#28 T06837 C0 0.000183s     + popFrame() {} // #5 
#29 T06837 C0 0.000185s     } // NimMainModule() 
#30 T06837 C0 0.000187s   } // NimMainInner() 
#31 T06837 C0 0.000189s } // NimMain() 
#32 T06837 C0 0.000206s } // main() 
#33 T06837 C0 0.000208s + colonanonymous___system_5714() {} // #1 
#34 T06837 C0 0.000221s + cwrap_log_stats() { // #1 [cwrap_log_stats() ignores verbosity!]
#35 T06837 C0 0.000223s   - 1 calls to 1 of 1 function variation(s) for cwrap_log_stats()
#36 T06837 C0 0.000226s   - 1 calls to 1 of 1 function variation(s) for cwrap_log_verbosity_set()
#37 T06837 C0 0.000228s   - 1 calls to 1 of 1 function variation(s) for cwrap_log_quiet_until()
#38 T06837 C0 0.000230s   - 5 calls to 1 of 1 function variation(s) for popFrame()
#39 T06837 C0 0.000233s   - 2 calls to 1 of 1 function variation(s) for nimToCStringConv()
#40 T06837 C0 0.000234s   - 5 calls to 1 of 1 function variation(s) for nimFrame()
#41 T06837 C0 0.000236s   - 2 calls to 1 of 1 function variation(s) for nimAddInt()
#42 T06837 C0 0.000239s   - 1 calls to 1 of 1 function variation(s) for foo__helloworld_1()
#43 T06837 C0 0.000242s   - 2 calls to 1 of 1 function variation(s) for echoBinSafe()
#44 T06837 C0 0.000244s   - 1 calls to 1 of 1 function variation(s) for colonanonymous___system_5714()
#45 T06837 C0 0.000246s   - 1 calls to 1 of 1 function variation(s) for bar__helloworld_2()
#46 T06837 C0 0.000248s   - 1 calls to 1 of 1 function variation(s) for NimMainModule()
#47 T06837 C0 0.000249s   - 1 calls to 1 of 1 function variation(s) for NimMainInner()
#48 T06837 C0 0.000251s   - 24 calls to 13 of 240 functions instrumented
#49 T06837 C0 0.000253s   } // cwrap_log_stats()
  • That's it :-)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions