Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Can we get another beta? #333

Closed
luislavena opened this Issue · 54 comments

7 participants

@luislavena

Hello guys,

Lot of folks are having problems with 1.0.0.beta.4, Ruby 1.9.3-p194 and the dreaded select() and FD_SETSIZE issue.

Also, any attempt to compile from source with gem install eventmachine --pre --platform=ruby results in errors.

I just tried to compile from master and everything worked.

Can we get another beta out so people can compile it if required?

Thank you! :heart: :heart: :heart:

@luislavena luislavena referenced this issue in oneclick/rubyinstaller
Closed

Getting seg fault with 1.9.3p194 #116

@Azolo

Actually it's still broke, It's a Runtime error so it actually compiles fine. :tongue:

#319 still needs to resolved

@luislavena

@Azolo problem is that beta.4 (the one on RubyGems) does not compile at all, with or without FD_SETSIZE :tongue: :tongue:

@Azolo

@luislavena touché, my mistake. But interestingly enough the mingw32 compiled gem is at beta.4.1.

PS C:\Users\Justin> gem install eventmachine --pre
Successfully installed eventmachine-1.0.0.beta.4.1-x86-mingw32
1 gem installed
@garbagecat
Collaborator
@algritz

Any luck getting the new beta done?

@garbagecat
Collaborator
@algritz

Thanks, I'll come back once in a while to check if there's anything new.

@tmm1
Owner

I just pushed eventmachine 1.0.0.rc.1 to rubygems.

@stakach

@tmm1 @luislavena It's still an issue as FD_SETSIZE needs to be set at compile time otherwise it defaults to 64 sockets which is even worse then before.

All pre-compiled gems should have an FD_SETSIZE matching that of the ruby implementation they are compiled for.

FD_SETSIZE can only be removed completely if compilation occurs during gem installation.
We should come to an agreement that all ruby implementations running on Windows use a set size of 32767

@stakach

That or it needs to be compiled with the latest version of rubyinstaller ruby installed?

@garbagecat
Collaborator
@luislavena

@stakach latest RubyInstaller (1.9.3-p194) has FD_SETSIZE to 32767, so pre-compiled EventMachine should set to the same value.

The problem with that is that it will fail with previous versions of Ruby, like 1.8.7 (which has 256) or 1.9.2 (which has 64).

I can't go back in time and fix previous versions, but Ruby itself has been fixed so next release it will not matter how many FD_SETSIZE you set in your application, it will not segv due Ruby anymore.

So my proposal is remove the definition from the code and let extconf determine if FD_SETSIZE is required (by looking to previous $defs)

Thoughts?

@garbagecat
Collaborator
@stakach

What I'm trying to say is that the newly released eventmachine does not run on the latest version of RubyInstaller.
It segfaults as it was not compiled with FD_SETSIZE set to 32767.

As the next version of Ruby will run no matter what FD_SETSIZE is I vote we add it back to eventmachine, set to 32767 so it runs on the current version and supports more then 64 sockets when the next version of ruby is released.

@tmm1
Owner

I'm building these gems using rake-compiler on my Mac. AFAIK, there will be no FDSET_SIZE in $defs there.

How do I create a gem that works with 1.8.7, 1.9.2 and 1.9.3 on windows without segfaulting?

@luislavena

@stakach that is why I mentioned on previous comment that FD_SETSIZE definition should be checked on extconf.rb and if not found, defined.

I'm seem unable to locally compile EventMachine to try out my theory, but checking against RbConfig::CONFIG["cppflags"] for FD_SETSIZE should do the trick.

@stakach

Maybe we don't have a pre-compiled version then? That would work on all platforms?

@luislavena

@stakach a pre-compiled version is required since OpenSSL is not available for compilation on Windows (working on solve that issue)

@tmm1 can you check RbConfig::CONFIG["CPPFLAGS"]?

What about something like this?

if RbConfig::CONFIG["host_os"] =~ /mingw/
  found = RbConfig::CONFIG.values_at("CFLAGS", "CPPFLAGS").
    any? { |v| v.include?("FD_SETSIZE") }

  $defs << "-DFD_SETSIZE=32767" unless found
end

I just tested against RubyInstaller 1.8.7 and 1.9.3, and FD_SETSIZE is not defined if present in RbConfig.

Now, RubyInstaller 1.8.7 had FD_SETSIZE=256 and 1.9.3 now have 32767, so perhaps the size can be done using RUBY_VERSION accordingly.

When cross compiling RUBY_VERSION is set accordingly so should work.

Let me know otherwise.

@stakach

Setting setsize to 32767 won't segfault older versions of RubyInstaller.
The segfault only occurs if the setsize in Eventmachine is smaller then the setsize in Ruby

@luislavena

@stakach sorry, don't remember the details, been a few months since I dealt with this issue at Ruby-Core.

Perhaps the check for FD_SETSIZE is enough and check for version is not required. Agree?

@stakach

Agreed :)

@tmm1
Owner

Can someone start a pull request?

@luislavena

@tmm1 will give a try when I arrive home later today.

@luislavena

@stakach 1.0.0.rc2 is out, however binary gem doesn't work with RubyInstaller 1.9.3-p194, only with TCS release.

AARGGH

@tmm1 can you tell me if the generated Makefile in tmp/i386-mingw32 contains -DFD_SETSIZE=32767 ?

I see no logic based on previous @stakach comment that it wouldn't segfault.

@stakach

@luislavena it runs on all the versions of ruby installer prior to p194.

So my assumption here is that it was not compiled with -DFD_SETSIZE=32767

@luislavena

@stakach I'll get my ubuntu VM tomorrow and test cross-compilation, this entire issue is making me bald, so I guess @tmm1 head is already a lovely shinny one :grin:

@stakach

My logic about the segfault only occuring if the setsize in Eventmachine is smaller then the setsize in Ruby is valid.

The easy solution would be just to re-add the define in project.h as 32767 - might not be the most pretty or technically valid however it will run on all versions of ruby installer and rubinius.

@tmm1
Owner

It's possible I didn't fully clean before doing the rc2 build. I am seeing the -DFDSET_SIZE as part of the gcc command.

I just did a full clean and re-build the rc2 gem. Can you see if it works any better: http://cl.ly/1G0a210F2T2K473z0X13

@Azolo

It doesn't for me =( Here's the build I created using @luislavena's gem-compiler gem: http://ge.tt/9Ct7AeJ?c

It has the gem that's compiled on my local machine plus the tmp folder that gem-compiler created. I don't really know if that will help any but, it's there.

@stakach

I'm still seeing the seg-fault with your re-build @tmm1.
If I compile the gem on windows without SSL support it works.

@luislavena

@stakach I just checked on my mac. FD_SETSIZE gets defined for 1.8.7 and 1.9.3 Makefiles:

rubyeventmachine/1.8.7/Makfile

CPPFLAGS = -DWITHOUT_SSL -DBUILD_FOR_RUBY -DHAVE_RB_TRAP_IMMEDIATE -DHAVE_RBTRAP -DHAVE_RB_TIME_NEW -DOS_WIN32 -DFD_SETSIZE=32767 -DHAVE_WINDOWS_H -DHAVE_WINSOCK_H -DHAVE_MAKE_PAIR   

rubyeventmachine/1.9.3/Makfile:


CPPFLAGS = -DWITHOUT_SSL -DBUILD_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_RB_THREAD_CHECK_INTS -DHAVE_RB_TIME_NEW -DOS_WIN32 -DFD_SETSIZE=32767 -DHAVE_WINDOWS_H -DHAVE_WINSOCK_H -DHAVE_MAKE_PAIR  $(DEFS) $(cppflags)

Neither of my 1.8.7 or 1.9.3 cross-compiled Rubies had FD_SETSIZE

fastfilereader extension didn't receive this code change, so I don't know if this could be the cause.

I just setup the environment locally, built OpenSSL 1.0.0j (did some changes to extconf for it) and built a native gem.

Can you download and test it? (don't have my Windows machine with me now)

http://cl.ly/2u0s2f003s001j3N3P3Y

@luislavena

OK guys,

Except for the dependency on libgcc_s_sjlj-1.dll (for which you need to include DevKit (running devkitvars.bat) my build did work under:

  • 1.8.7-p358
  • 1.9.2-p290
  • 1.9.3-p194
  • 1.9.3-p232 (TCS build)

So there must be something interfering with the build.

@stakach if you can test and confirm then...

@tmm1 I'm going to send another pull request with the modifications I made. If you can build another mingw32 gem and upload to CloudApp for us to download and test, will be much appreciated.

I'm working really hard on getting gem-compiler working to provide a better building functionality, but free time is something scarce lately.

Thank you.

@Azolo

@luislavena I'm getting a pretty awesome error with your new download.

https://gist.github.com/2999990

I don't know what is going on there, I feel like it's probably my fault though.

@luislavena

@Azolo see previous comment, you need libgcc dll in the PATH for this to work, you need devkitvars, you should have gotten a popup indicating the missing DLL.

That is the problem with the mingw-w64 compiler I had in my Mac, need to go back to stable 4.5.4 to avoid that dependency issue.

@stakach

@luislavena works for me!

@Azolo

@luislavena Ahh, gotcha. I totally misinterpreted that statement.

Sorry about that, it's working now. :smiley:

@algritz

Working for me now, A big thanks for your hard work guys, its really appreciated !

@luislavena

Hello folks

OK, last try, here is another build of the gem:

http://cl.ly/1z0p2R2H162k1j1b3l0C

This time it doesn't require adding the DevKit to the PATH, it should work out of the box with normal gem installation.

Please uninstall the previous version of the gem I provided before, install this and try. I tested on 1.8.7 and 1.9.3 without issues.

If this work for others (to reduce the changes of works for me), @tmm1 the patches I applied are here:

https://github.com/luislavena/eventmachine/compare/minor-tweaks-cross-compile

And I will send a pull request once @stakach, @Azolo and @algritz confirm this new version works.

The step to build the gem where:

  • Use Ruby 1.8.7 natively
  • Have Ruby 1.8.7-p358 and 1.9.3-p125 cross compiled Rubies installed.
  • Build openssl_libs from ruby-pg with rake openssl_libs OPENSSL_VERSION=1.0.0j
  • Move the build artifacts of openssl into rake-compiler: mv build/builds/openssl-1.0.0j/ ~/.rake-compiler/builds/
  • Build the extensions with rake cross compile OPENSSL_VERSION=1.0.0j RUBY_CC_VERSION=1.8.7:1.9.3
  • Build the gem: rake cross native gem

Thank you guys for your patience while I replied, I was out sick (still).

Cheers.

@Azolo

@luislavena No weirdness here.

@luislavena

Thanks @Azolo once I get confirmation from @stakach that this new version works, will send the pull request.

@stakach

@luislavena clean environment, no devkit in the path - looks good! Epic work.

@tmm1
Owner

Pushed up 1.0.0.rc.3 gems to rubygems.

@stakach

@tmm1 Unfortunately I'm seeing an error on all RubyInstallers:

Unable to load the EventMachine C extension; To use the pure-ruby reactor, require 'em/pure_ruby'
LoadError: cannot load such file -- 1.9/rubyeventmachine

etc

@luislavena

@tmm1 seems is missing 1.9 folder into the gem, perhaps RUBY_CC_VERSION is not set to 1.8.7:1.9.3 ?

@tmm1
Owner

I copy-pasted that command.. I think maybe I never installed a cross-compiled 1.9.3 locally, only 1.9.2.

Yep, oops:

$ rake-compiler cross-ruby
Updating .rake-compiler/config.yml
Found Ruby version 1.8.7 (.rake-compiler/ruby/ruby-1.8.7-p334/lib/ruby/1.8/i386-mingw32/rbconfig.rb)
Found Ruby version 1.9.2 (.rake-compiler/ruby/ruby-1.9.2-p180/lib/ruby/1.9.1/i386-mingw32/rbconfig.rb)
@luislavena

@tmm1 ahh, this part:

Have Ruby 1.8.7-p358 and 1.9.3-p125 cross compiled Rubies installed.

Is what was missing :-P

Just changing RUBY_CC_VERSION to 1.8.7:1.9.2 should work.

Sorry for not being more clear, build against 1.9.2 or 1.9.3 will be the same, I'm always installing latest.

@tmm1
Owner

OK, try 1.0.0.rc.4.

@luislavena

@tmm1 tested against echo server (from README)

  • Ruby 1.8.7-p358
  • Ruby 1.9.2-p290
  • Ruby 1.9.3-p194
  • Ruby 1.9.3-p231 (TCS build)

Works on all of them.

Thank you! :heart: :heart: :heart:

@luislavena luislavena closed this
@tmm1
Owner

Excellent. Thanks for all the help everyone!

@luislavena

Thanks to you @tmm1 for your patience!

@ruilov

I was running into this too, and wanted to thank everyone here for the hard work getting to the bottom of the issue. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.