New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Byebug debugger and readline problems #124

Closed
ccoupe opened this Issue May 18, 2015 · 19 comments

Comments

Projects
None yet
2 participants
@ccoupe
Contributor

ccoupe commented May 18, 2015

This bug report is mostly for documentation / discussion purposes.

The byebug gem doesn't work on cross compiled Shoes Windows (xwin7) but does in a Shoes built by Windows (win7)?! It complains about not having 'readline'. Indeed that Ruby extension is not built when cross compiling ruby for Windows. It's not built for i386 and x86_64 chrooted rubies either.

Turns out the RubyInstaller folks have a pure Ruby implementation (a gem named 'rb-readline', https://github.com/ConnorAtherton/rb-readline. Now that including gems into Shoes building is not as difficult it can can be added to the list of default gems of Shoes.

Byebug has context problems with being invoked from Shoes in an end user friendly way but it seems solvable. Perhaps a --debug flag on the command for Shoes maintainers that want to debug shoes.rb and cobbler.rb and the like and modify the Shoes irb.rb code to set the context properly for a loaded script.

I'm considering stealing the Alt-= to invoke the debugger on the current script and then inside the debugger one can invoke irb (if needed)

@ccoupe ccoupe added the Normal label May 18, 2015

@ccoupe ccoupe assigned ccoupe and unassigned ccoupe May 18, 2015

ccoupe added a commit that referenced this issue May 20, 2015

Updating win7 and xwin7 for many things: #72, #124
* move to Ruby 2.1.6
* Windows ruby gotcha's
* Added Rake constant SHOES_GEM_ARCH (for xwin7 and xarm6hf need this)
* Accomodate central directory of gems (old skool like sqlite3) and newer
  gems (copied by CopyGem.rb or extract1gem.rb or built by elfs with magic)
* Added byebug and deps to built in gems. No one is forced to use them and
  it's a bit clunky to use and developers only and there could be OSX huh?
  -- just getting started.
* Most built-in Shoes gems are no longer in 'specications/default.
  Sqlite3 is still in between - but it works.
* expect similar changes for other platforms and sqlite

@ccoupe ccoupe added Enhancement and removed Normal labels May 24, 2015

ccoupe added a commit that referenced this issue May 26, 2015

A debugger console - WIP (Work in progress) #124
* Does not work with byebug. And it's ugly and confusing to me.
* need Threading (where byebug is not the main thread)?
* did I mention IT DOES NOT WORK?
@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 26, 2015

Contributor

👍

Contributor

passenger94 commented May 26, 2015

👍

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 27, 2015

Contributor

So close and yet so far.
screenshot from 2015-05-26 23 23 21

I just know it's going to be "doh!" and I'll end up looking stupid and incompetent. I am. Help?

Contributor

ccoupe commented May 27, 2015

So close and yet so far.
screenshot from 2015-05-26 23 23 21

I just know it's going to be "doh!" and I'll end up looking stupid and incompetent. I am. Help?

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 28, 2015

Contributor

what's wrong ? This avant garde master piece reminds me a deconstructivist byebug session in shoes but better !!
Seriously, you're doing well apparently :-)

Contributor

passenger94 commented May 28, 2015

what's wrong ? This avant garde master piece reminds me a deconstructivist byebug session in shoes but better !!
Seriously, you're doing well apparently :-)

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 28, 2015

Contributor

Not so well. We have the Shoes thread and the byebug thread. And a common muxtex/condition variable. When the byebug thread does a gets() it goes to my readline() where it is supposed to wait for the Shoes thread (which is the console app) to get the keystrokes and when we get \n we tell the the signal the mutex/cv. Simple consumer/producer pattern.

Except the wait in readline from the byebug thread stops the Shoes thread. They are different threads. I'll commit the latest if you want to try it. Just install byebug, rb-readline in your Ruby and build Loose Shoes where you don't have to build shoes every time you change a line in /shoes/*.rb

Contributor

ccoupe commented May 28, 2015

Not so well. We have the Shoes thread and the byebug thread. And a common muxtex/condition variable. When the byebug thread does a gets() it goes to my readline() where it is supposed to wait for the Shoes thread (which is the console app) to get the keystrokes and when we get \n we tell the the signal the mutex/cv. Simple consumer/producer pattern.

Except the wait in readline from the byebug thread stops the Shoes thread. They are different threads. I'll commit the latest if you want to try it. Just install byebug, rb-readline in your Ruby and build Loose Shoes where you don't have to build shoes every time you change a line in /shoes/*.rb

@ccoupe ccoupe added this to the 3.2.23 milestone May 28, 2015

@ccoupe ccoupe self-assigned this May 28, 2015

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 28, 2015

Contributor

Is this another variation of the IO block nonblock issue ? i just had to deal with it recently while trying to communicate from irb to shoes via popen (programatically starting an irb session and calling shoes for the gui) turns out it works but i had a hard time with the shoes thread, i tried Fibers Threads with no luck ... i use a "read_nonblock" method in a loop @ shoes to wait for messages from irb, it was always freezing shoes even though i tried the task to be done in another Thread/Fiber
(might only be my short knowledge though ... but i remember _why saying to stay away from threads in shoes ? (maybe a pre ruby-1.9 issue))

Contributor

passenger94 commented May 28, 2015

Is this another variation of the IO block nonblock issue ? i just had to deal with it recently while trying to communicate from irb to shoes via popen (programatically starting an irb session and calling shoes for the gui) turns out it works but i had a hard time with the shoes thread, i tried Fibers Threads with no luck ... i use a "read_nonblock" method in a loop @ shoes to wait for messages from irb, it was always freezing shoes even though i tried the task to be done in another Thread/Fiber
(might only be my short knowledge though ... but i remember _why saying to stay away from threads in shoes ? (maybe a pre ruby-1.9 issue))

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 28, 2015

Contributor

In this case, I think byebug likes/wants/has-to pause threads that it didn't create or manage. The next commit has a simple get/put loop that is commented out but runs fine does show that the signal/wait I coded works. Just not with byebug.

I decided to implement the remote debugging server in Shoes if you type alt-; and then use another terminal. It works, even on OSX, so that's a win. See https://github.com/Shoes3/shoes3/wiki/Debugging-with-Byebug

The real problem is that nobody writes their gems expecting it will be used by an embedded MRI Ruby. They all assume command line ruby is being used and their gem is being used by clued-in folks, I don't think we can fix that in any meaningful way. Once I know enough about what byebug is doing with Threads then I'll put an issue up on their project asking for help/guidance.

Contributor

ccoupe commented May 28, 2015

In this case, I think byebug likes/wants/has-to pause threads that it didn't create or manage. The next commit has a simple get/put loop that is commented out but runs fine does show that the signal/wait I coded works. Just not with byebug.

I decided to implement the remote debugging server in Shoes if you type alt-; and then use another terminal. It works, even on OSX, so that's a win. See https://github.com/Shoes3/shoes3/wiki/Debugging-with-Byebug

The real problem is that nobody writes their gems expecting it will be used by an embedded MRI Ruby. They all assume command line ruby is being used and their gem is being used by clued-in folks, I don't think we can fix that in any meaningful way. Once I know enough about what byebug is doing with Threads then I'll put an issue up on their project asking for help/guidance.

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 28, 2015

Contributor

works on my side ! :-)

Contributor

passenger94 commented May 28, 2015

works on my side ! :-)

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 28, 2015

Contributor

Good to know, Yves. Thanks for reporting. Small wins are good. FIFA executive level awards would be better, but aren't there yet. Long ago and far away in another world I wrote a DEC VT100 terminal emulator for an Apple ][ and DEC Rainbow but I stopped when Kermit was released because it was so much better. Now days, no one knows about Kermit but I'm back to emulating a tty again.

I always have irony in the back pocket of my pants.

Contributor

ccoupe commented May 28, 2015

Good to know, Yves. Thanks for reporting. Small wins are good. FIFA executive level awards would be better, but aren't there yet. Long ago and far away in another world I wrote a DEC VT100 terminal emulator for an Apple ][ and DEC Rainbow but I stopped when Kermit was released because it was so much better. Now days, no one knows about Kermit but I'm back to emulating a tty again.

I always have irony in the back pocket of my pants.

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 28, 2015

Contributor

i don't know who is Irony or what Kermit is doing on github with all those tty ! and i'm not sure i want to know, .. though a bit of selfless sane curiosity ... :-D

Funny how we are all reproducing some personal pattern again and again, better be backed up with some serious irony (if that could possibly make any sense)

Glad to have helped a bit :-)

Contributor

passenger94 commented May 28, 2015

i don't know who is Irony or what Kermit is doing on github with all those tty ! and i'm not sure i want to know, .. though a bit of selfless sane curiosity ... :-D

Funny how we are all reproducing some personal pattern again and again, better be backed up with some serious irony (if that could possibly make any sense)

Glad to have helped a bit :-)

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 29, 2015

Contributor

A working console in Shoes that that can replace Ruby's stdin/stdout/stderr with enough fidelity would be a useful addition to the tool box if it could be written in a way that it could be extended with/by/for command line processors (like irb, byebug or even a primitive shell or something like 'expect' (and old Tcl program). I can imagine many uses in the embedded device world when a console and the serial port gem packaged (or some other gadget connector ) inside a Shoes program would be nice thing to provide to Shoes developers.

Contributor

ccoupe commented May 29, 2015

A working console in Shoes that that can replace Ruby's stdin/stdout/stderr with enough fidelity would be a useful addition to the tool box if it could be written in a way that it could be extended with/by/for command line processors (like irb, byebug or even a primitive shell or something like 'expect' (and old Tcl program). I can imagine many uses in the embedded device world when a console and the serial port gem packaged (or some other gadget connector ) inside a Shoes program would be nice thing to provide to Shoes developers.

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 30, 2015

Contributor

As I suspected, byebug does shut down Ruby threads and I've ask a question deivid-rodriguez/byebug#146.

It might be possible to have Gtk/Cocoa, C/Obj- C window that implements the terminal stdin/out/err that doesn't depend on Ruby being involved (other that some setup code). We did that for Windows and issue #110. There is a vte widget for gtk. and iTerm1/2 for OSX although that is pretty feature heavy for what we want. https://github.com/armadsen/ORSSerialPort looks less heavy for OSX. We just need to steal stdin/out/err beneath ruby and execute a command line program

Contributor

ccoupe commented May 30, 2015

As I suspected, byebug does shut down Ruby threads and I've ask a question deivid-rodriguez/byebug#146.

It might be possible to have Gtk/Cocoa, C/Obj- C window that implements the terminal stdin/out/err that doesn't depend on Ruby being involved (other that some setup code). We did that for Windows and issue #110. There is a vte widget for gtk. and iTerm1/2 for OSX although that is pretty feature heavy for what we want. https://github.com/armadsen/ORSSerialPort looks less heavy for OSX. We just need to steal stdin/out/err beneath ruby and execute a command line program

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 30, 2015

Contributor

"We just need to steal stdin/out/err beneath ruby and execute a command line program"

well, i don't have a clear vision of what you are building, and the specific needs/problems but why not Open3.popen3 and family ? i believe it's cross platform, totally Ruby though so maybe that's the problem

Contributor

passenger94 commented May 30, 2015

"We just need to steal stdin/out/err beneath ruby and execute a command line program"

well, i don't have a clear vision of what you are building, and the specific needs/problems but why not Open3.popen3 and family ? i believe it's cross platform, totally Ruby though so maybe that's the problem

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 30, 2015

Contributor

If you start Shoes from the Linux menus or Windows menus or OSX dbl click, there is no controlling terminal that responds to fgetc(stdin) calls and usually nothing attached for fputc(stdout, or stderr). Shoes can replace those at the Ruby level but not at C level. If byebug suspends Ruby threads (it does) and want a puts and readline from a terminal (it does) then it would have to be modified to play nice with Shoes (possible but unlikely) or Shoes could provide the teminal if the user was debugging or wanted pry or a better irb (or even less pretty log that behaves properly).

OSX is biggest problem, even when launched from the terminal Shoes is not connected to the launching terminal.

Contributor

ccoupe commented May 30, 2015

If you start Shoes from the Linux menus or Windows menus or OSX dbl click, there is no controlling terminal that responds to fgetc(stdin) calls and usually nothing attached for fputc(stdout, or stderr). Shoes can replace those at the Ruby level but not at C level. If byebug suspends Ruby threads (it does) and want a puts and readline from a terminal (it does) then it would have to be modified to play nice with Shoes (possible but unlikely) or Shoes could provide the teminal if the user was debugging or wanted pry or a better irb (or even less pretty log that behaves properly).

OSX is biggest problem, even when launched from the terminal Shoes is not connected to the launching terminal.

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 May 30, 2015

Contributor

ah yes, you mean it's only possible to pipe from another program to Shoes (i've done it, shoes reading on stdin and sending messages on stdout) but not from Shoes to another program if launched from the menus (don't know about osx) ?
Via a shell script of some sort maybe ? In any case if the needs of byebug, stops you from following that route ...

Contributor

passenger94 commented May 30, 2015

ah yes, you mean it's only possible to pipe from another program to Shoes (i've done it, shoes reading on stdin and sending messages on stdout) but not from Shoes to another program if launched from the menus (don't know about osx) ?
Via a shell script of some sort maybe ? In any case if the needs of byebug, stops you from following that route ...

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 31, 2015

Contributor

The easy demo would be to install Shoes 3.2.23 on Windows. Then note the start up difference from shoes.exe path-to-shoes-script and shoes.exe --console path-to-shoes-script. The latter opens a new console window (even when launched from the windows menus or a dbl click). It's a handy feature #110 and if you need it and in the case byebug it is needed for osx and linux.

The important spec is that it's all happens in one process. not two or three, no pipes. We want to start the byebug gem inside Shoes and we want to start the command version that wants stdin/stdout

https://github.com/alanszlosek/tesi/blob/master/examples/gtk-terminal.c is an reasonable example we might start with. Gtk.Keyevent gets written to the stdin fd. and anything written to the stdout fd is drawn on the window. Shoes/Linux would not know there was that new gtk level window title 'console' just like Shoes/Windows has no knowledge of the --console window. It's not a window that Shoes/Ruby knows about other than $stdin and $stdout.

Contributor

ccoupe commented May 31, 2015

The easy demo would be to install Shoes 3.2.23 on Windows. Then note the start up difference from shoes.exe path-to-shoes-script and shoes.exe --console path-to-shoes-script. The latter opens a new console window (even when launched from the windows menus or a dbl click). It's a handy feature #110 and if you need it and in the case byebug it is needed for osx and linux.

The important spec is that it's all happens in one process. not two or three, no pipes. We want to start the byebug gem inside Shoes and we want to start the command version that wants stdin/stdout

https://github.com/alanszlosek/tesi/blob/master/examples/gtk-terminal.c is an reasonable example we might start with. Gtk.Keyevent gets written to the stdin fd. and anything written to the stdout fd is drawn on the window. Shoes/Linux would not know there was that new gtk level window title 'console' just like Shoes/Windows has no knowledge of the --console window. It's not a window that Shoes/Ruby knows about other than $stdin and $stdout.

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe May 31, 2015

Contributor

This is as close I'm going to get to design specs - https://github.com/Shoes3/shoes3/wiki/Design:-New-Shoes-Console.

Contributor

ccoupe commented May 31, 2015

This is as close I'm going to get to design specs - https://github.com/Shoes3/shoes3/wiki/Design:-New-Shoes-Console.

ccoupe added a commit that referenced this issue Jun 11, 2015

stub method for #128 so OSX compiles and runs. Still TOTO.
* Restore byebug use of native (launch) terminal
* for OSX /long/path/Shoes.app/Contents/MacOS/shoes-launch keeps the controlling
  terminal and byebug runs fine when Shoes is launched that way. #124
* This should be moved from shoes.rb to a command line option. The splash menu
  should use the Shoes console (assuming that ever works).
@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe Jun 11, 2015

Contributor

Going to close this issue unless there are objections.

Contributor

ccoupe commented Jun 11, 2015

Going to close this issue unless there are objections.

@passenger94

This comment has been minimized.

Show comment
Hide comment
@passenger94

passenger94 Jun 11, 2015

Contributor

Niiiiiiiiiiiice , works here :-) 👍
time to dig inside byebug :-)

Contributor

passenger94 commented Jun 11, 2015

Niiiiiiiiiiiice , works here :-) 👍
time to dig inside byebug :-)

ccoupe added a commit that referenced this issue Jun 13, 2015

@ccoupe

This comment has been minimized.

Show comment
Hide comment
@ccoupe

ccoupe Jun 13, 2015

Contributor

I'm going to close this issue. I still want a replacement console like #110 for OSX and Linux but that's a different but related issue.

Contributor

ccoupe commented Jun 13, 2015

I'm going to close this issue. I still want a replacement console like #110 for OSX and Linux but that's a different but related issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment