Skip to content
This repository

Node.js compile fails in Amazon AMI x64 - snapshot.cc - pure functional method call #2912

Closed
brandwe opened this Issue · 45 comments

8 participants

Brandon Werner Ben Noordhuis Pablo Tejada quexer Clayton Ben Walding Shawn Jonnet Sergey Kovalyov
Brandon Werner

Version 0.7.5

LINK(target) /usr/local/src/node/out/Release/mksnapshot
LINK(target) /usr/local/src/node/out/Release/mksnapshot: Finished
ACTION v8_snapshot_run_mksnapshot /usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.cc
pure virtual method called
terminate called without an active exception
/bin/sh: line 1: 23819 Aborted "/usr/local/src/node/out/Release/mksnapshot" --log-snapshot-positions --logfile "/usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.log" "/usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.cc"
make[1]: *** [/usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.cc] Error 134
make[1]: Leaving directory `/usr/local/src/node/out'

Brandon Werner

Contents of log file:

code-creation,Stub,0x270d63206000,351,"JSEntryStub"
code-creation,Stub,0x270d63206160,351,"JSConstructEntryStub"
code-creation,Stub,0x270d632062c0,762,"CEntryStub"
code-creation,Stub,0x270d632065c0,1113,"CEntryStub"

Ben Noordhuis

Looks like a V8 issue. You can disable snapshots with ./configure --without-snapshot. If you want to debug the issue, attach a debugger to mksnapshot and inspect the call stack. make BUILDTYPE=Debug compiles a debug build.

Brandon Werner

Attaching gdb now. I can get a clean compile in v8 standalone so it'll be interesting to see what's happening in snapshot in v8.

Pablo Tejada

After ./configure --without-snapshot i run make install and here is the output

make -C out BUILDTYPE=Release
make[1]: Entering directory /root/node-v0.7.6/out'
make[1]: Nothing to be done forall'.
make[1]: Leaving directory `/root/node-v0.7.6/out'
ln -fs out/Release/node node
out/Release/node tools/installer.js ./config.gypi install
pure virtual method called
terminate called without an active exception
make: *** [install] Aborted

Below is the output of g++ -v

Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC)
Deleted user
ghost commented

The content you are editing has changed. Reload the page and try again.

I'm seeing the same results as ptejada - any thoughts here?

Sending Request…

Attach images by dragging & dropping or selecting them. Octocat-spinner-32 Uploading your images… Unfortunately, we don't support that file type. Try again with a PNG, GIF, or JPG. Yowza, that's a big file. Try again with an image file smaller than 10MB. This browser doesn't support image attachments. We recommend updating to the latest Internet Explorer, Google Chrome, or Firefox. Something went really wrong, and we can't process that image. Try again.

Ben Noordhuis

@spack8rivers: Try hooking up the faulting process (mksnapshot, node) to gdb and check where the method call comes from. Please post the backtrace here.

quexer

same issue on my box.

#uname -a

Linux foo 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

Ben Noordhuis

People, I need a little more to go on. Can you try this?

$ touch empty.js
$ gdb --args node empty.js
> run
# wait for "pure virtual method called" exception
> backtrace full

Please post the backtrace here.

Pablo Tejada

Here is my output, do note i ran it in CentOS 6 x64

Starting program: /home/ptejada/webapps/node/src/node empty.js
[Thread debugging using libthread_db enabled]
pure virtual method called
terminate called without an active exception

Program received signal SIGABRT, Aborted.
0x00007ffff6cb7885 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
(gdb) backtrace full
#0  0x00007ffff6cb7885 in raise () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007ffff6cb9065 in abort () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007ffff7789a7d in __gnu_cxx::__verbose_terminate_handler() ()
   from /usr/lib64/libstdc++.so.6
No symbol table info available.
#3  0x00007ffff7787c06 in ?? () from /usr/lib64/libstdc++.so.6
No symbol table info available.
#4  0x00007ffff7787c33 in std::terminate() () from /usr/lib64/libstdc++.so.6
No symbol table info available.
#5  0x00007ffff778855f in __cxa_pure_virtual () from /usr/lib64/libstdc++.so.6
No symbol table info available.
#6  0x000000000095aac4 in v8::internal::CodeStub::FindCodeInCache(v8::internal::Code**) ()
No symbol table info available.
#7  0x000000000095afb8 in v8::internal::CodeStub::GetCode() ()
No symbol table info available.
#8  0x00000000008cb1f1 in v8::internal::CodeStub::GenerateStubsAheadOfTime() ()

Ben Noordhuis

@ptejada: Thanks. This looks like it could be a number of things but it's probably a V8 issue.

Can you check if it happens with debug builds too? You can build one with make -C out BUILDTYPE=Debug, then run gdb --args out/Debug/node empty.js. Please post the backtrace, it should have line numbers.

Pablo Tejada

There was no stack for the backtrace, here is the console output:

(gdb) run
Starting program: /home/ptejada/webapps/node/src/out/Debug/node empty.js
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff7fe4700 (LWP 11211)]
[Thread 0x7ffff7fe4700 (LWP 11211) exited]

Program exited normally.
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
(gdb) backtrace
No stack.
(gdb) backtrace full
No stack.
(gdb)

Ben Noordhuis

Interesting. So debug builds work but release builds don't. That suggests we're looking at either a compiler bug or a build type-dependent code path in V8.

Can I persuade you to try it with the V8 shell? Here is how you build it:

$ git clone git://github.com/v8/v8.git
$ cd v8/
$ git checkout -b bleeding_edge -t origin/bleeding_edge
$ scons --jobs=<num-cores> arch=x64 mode=debug,release sample=shell
# wait while it compiles
$ touch empty.js
$ gdb --args ./shell_g empty.js
# do the run/backtrace thing here
$ gdb --args ./shell empty.js
# ditto

Testing the 32 bits build might be worthwhile too, just replace arch=x64 with arch=ia32.

Clayton

Linux hail.liaoningbeauties.com 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

-bash-4.1$ gdb --args node t.js
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-48.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /home/alibaba/joyent-node-93cefab/node...done.
(gdb) run
Starting program: /home/alibaba/joyent-node-93cefab/node t.js
[Thread debugging using libthread_db enabled]
pure virtual method called
terminate called without an active exception

Program received signal SIGABRT, Aborted.
0x0000003e6e232a45 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.25.el6.x86_64 libgcc-4.4.5-6.el6.x86_64 libstdc++-4.4.5-6.el6.x86_64
(gdb) bt
#0 0x0000003e6e232a45 in raise () from /lib64/libc.so.6
#1 0x0000003e6e234225 in abort () from /lib64/libc.so.6
#2 0x0000003e7aebea7d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib64/libstdc++.so.6
#3 0x0000003e7aebcc06 in ?? () from /usr/lib64/libstdc++.so.6
#4 0x0000003e7aebcc33 in std::terminate() () from /usr/lib64/libstdc++.so.6
#5 0x0000003e7aebd55f in __cxa_pure_virtual () from /usr/lib64/libstdc++.so.6
#6 0x0000000000971dc4 in v8::internal::CodeStub::FindCodeInCache(v8::internal::Code**) ()
#7 0x00000000009722b8 in v8::internal::CodeStub::GetCode() ()
#8 0x00000000008e2501 in v8::internal::CodeStub::GenerateStubsAheadOfTime() ()
#9 0x0000000000747623 in v8::internal::Heap::CreateFixedStubs() ()
#10 0x0000000000751594 in v8::internal::Heap::CreateInitialObjects() ()
#11 0x000000000075e284 in v8::internal::Heap::SetUp(bool) ()
#12 0x00000000007b8bc0 in v8::internal::Isolate::Init(v8::internal::Deserializer*) ()
#13 0x000000000057ef4b in node::Start(int, char**) ()
#14 0x0000003e6e21ec9d in __libc_start_main () from /lib64/libc.so.6
#15 0x0000000000575d8d in _start ()

Clayton

Right, the debug compile doesn't crash on my hello world script.

out/Debug/node t.js
hello world.

Clayton

I ran the steps provided in comment 5139451 by Ben Noordhuis, and there were no errors no backtrace.

Ben Noordhuis

@hammer2j2: Does a release build work if you disable snapshots (--without-snapshot)? Does the V8 shell work?

Clayton

I did, disable snapshot. It's the only way the release build will succeed. There's I think several ppl reporting similar issue there. thanks.

Ben Noordhuis

Can someone test the V8 shell? I need to narrow down the problem and I cannot reproduce it. You can try the bundled V8:

$ cd deps/v8
$ scons --jobs=<num-cores> arch=x64 mode=debug,release sample=shell
$ touch empty.js
$ ./shell_g empty.js
$ ./shell empty.js

If either shell or shell_g crashes, please post the backtrace.

Clayton

Regarding v8 shell: I ran the steps provided in comment 5139451. There were no errors no backtrace.

Ben Noordhuis

Thanks, that's useful info. Can you try it with the bundled V8? If that one does crash, it's been fixed upstream.

Clayton

$ cd deps/v8
$ scons --jobs= arch=x64 mode=debug,release sample=shell
$ touch empty.js
$ ./shell_g empty.js
$ ./shell empty.js

Runs w/o error.

Ben Noordhuis

Okay, that's kind of unfortunate: it means it's not a straightforward V8 bug and I'll have to do some digging. Can I get shell access on a machine that exhibits the behavior?

Clayton

bnoordhuis - I will work on getting a hosted environment setup for this by Monday.

Thanks.

Pablo Tejada

I have a clean test account with access to bash on a centOS 6 x64

How can i get the accout info to you?

Ben Noordhuis

@ptejada: Email me the account and machine details (address is in my profile) and I'll email back my public key. vim, git and gdb is all I need. No rush, I can't look at it until later tonight anyway. :-)

EDIT: I forgot, GNU screen would be nice too.

Pablo Tejada

@bnoordhuis There was actually no public email in your profile, so i sent it to the email on your site. Info@yoursite

Ben Noordhuis bnoordhuis closed this in c21c51a
Ben Noordhuis

Fixed in c21c51a, thanks @ptejada.

Turns out it's an obscure gcc bug. It didn't show up in standalone builds of V8 because it doesn't (by default) compile with the -fno-strict-aliasing flag that triggers the bug. I'll forward the patch to the V8 team.

Pablo Tejada

@bnoordhuis sure no problem

Ben Noordhuis bnoordhuis reopened this
Ben Noordhuis

Reopening. I've reverted the commit in 6b426a2, it was making mksnapshot segfault on sunos. Compiler bugs, you just can't win...

Ben Walding

What is the impact of compiling with --standalone?

We are seeing the same thing across a range of 0.7 versions on 64 bit EC2 hardware (m1.large) running Fedora core:

$ gcc -v
Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info
--with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared 
--enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit 
--disable-libunwind-exceptions --enable-gnu-unique-object 
--enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi 
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile 
--enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar 
--disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 
--build=x86_64-redhat-linux

Thread model: posix
gcc version 4.4.5 20101112 (Red Hat 4.4.5-2) (GCC) 

$ uname -a
Linux localhost 2.6.34.7-66.fc13.x86_64 #1 SMP Wed Dec 15 07:04:30 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

Our builds can be seen here:
https://cbruby.ci.cloudbees.com/job/build-all-nodes/

Ben Noordhuis

What is the impact of compiling with --standalone?

Standalone as in linking to an external libv8? It usually works but there are no guarantees - V8's API and ABI aren't stable.

If you mean --without-snapshot, there is no runtime performance overhead, only a slightly longer startup time.

Ben Walding

@bnoordhuis - Thanks for the info - unfortunately compiling using "--without-snapshot" did not improve the situation.

Still getting errors building >= 0.7.6 on x86_64

out/Release/node tools/installer.js install 
pure virtual method called
terminate called without an active exception
make: *** [install] Aborted
Ben Noordhuis

@bwalding You can apply the patch manually but I recommend you upgrade gcc.

Shawn Jonnet

I am receiving this error now with 0.8.2. I wasn't previously with 0.8.0 or 0.8.1 which is odd

pure virtual method called
terminate called without an active exception
/bin/sh: line 1: 32557 Aborted "/usr/local/n/node-v0.8.2/out/Release/mksnapshot" --log-snapshot-positions --logfile "/usr/local/n/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.log" "/usr/local/n/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.cc"
make[1]: *** [/usr/local/n/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.cc] Error 134
make[1]: Leaving directory `/usr/local/n/node-v0.8.2/out'
make: *** [node] Error 2

Sergey Kovalyov

Have the same with 0.8.2

Shawn Jonnet

Should I open a new ticket for this?

Ben Noordhuis

@sjonnet19 No need. Compile with -O2 for now (i.e. make CFLAGS+=-O2 CXXFLAGS+=-O2). That's going to be the default in v0.8.3 anyway.

If you have some time to kill, I'd be curious to know if compiling with either -fno-inline-functions or -fno-function-sections solves the issue for you.

Sergey Kovalyov

Thanks! Advice from #3675 works fine.

Shawn Jonnet

Thanks that works I am now dynamically patching n

Ben Noordhuis

Guys, can one of you test commit b40f813? -O3 but no -ffunction-sections.

Shawn Jonnet

No luck failed

Ben Noordhuis

@sjonnet19 What if you apply the patch and compile with make CFLAGS+=-fno-strict-aliasing CXXFLAGS+=-fno-strict-aliasing?

Shawn Jonnet

No luck either

Ben Noordhuis

This should be resolved now (where now = 0.8.4). I haven't had bug reports in a while so I'm going to assume that it is.

Ben Noordhuis bnoordhuis closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.