Wrong calling convention

[cfis]

rb-readline library has a pretty serious bug though that causes an MSVC instrumented build (I find it easiest to find bugs this way) to blow up. Its uses the cdecl calling convention to call the windows api. That causes the stack to get corrupted. Instead, it should use the stdcall calling convention.

diff --git a/lib/rbreadline.rb b/lib/rbreadline.rb
index 43894aa..b215aff 100644
--- a/lib/rbreadline.rb
+++ b/lib/rbreadline.rb
@@ -4288,7 +4288,7 @@ module RbReadline
def initialize(dllname, func, import, export = "0")
@proto = [import].join.tr("VPpNnLlIi", "0SSI").sub(/^(.)0*$/, '\1')
handle = DLL[dllname] ||= DL.dlopen(dllname)

•           @func = DL::CFunc.new(handle[func], TYPEMAP[export.tr("VPpNnLlIi", "0SSI")], func)
  

•           @func = DL::CFunc.new(handle[func], TYPEMAP[export.tr("VPpNnLlIi", "0SSI")], func, :stdcall)
       end
  
       def call(*args)
  

  -- 1.7.0.4.360.g11766c.dirty

[jmhodges]

Your patch came up all busted. Why not just make a fork, a new branch and push up the changes?

[cfis]

Yeah, guess so. Its such a simple change though:

@func = DL::CFunc.new(handle[func], TYPEMAP[export.tr("VPpNnLlIi", "0SSI")], func)

To:

@func = DL::CFunc.new(handle[func], TYPEMAP[export.tr("VPpNnLlIi", "0SSI")], func, :stdcall)

Anyway, sent Luis the patch file. I'll setup a fork if i don't hear from him.

[cfis]

Not sure how to reopen this...grrr...

[rdp]

appears this is fixed now?

[luislavena]

This has already been fixed. Closing this out.