ruby-send-region fails on long single command

[mikelygee]

I ran into this using org babel to pass a table to a ruby block.

I had no problems with it on a Windows machine, but the operation hung on my linux system.

org generates the ruby code below to pass the table. If I paste this into an inf-ruby buffer, it is interpreted correctly. However, when I select it and run (ruby-send-region), I get an "unexpected end-of-input". As far as I can tell, process-send-region is sending the region in chunks. As soon as it sends the first 4k characters of the region, it look s like inf-ruby is trying to interpret it, and fails because it is missing the full line.

raw_data=[["Column 1", "Column 2", "Notes"], [0, 0, "This is note #0. It just takes up space."], [1, 1, "This is note #1. It just takes up space."], [2, 2, "This is note #2. It just takes up space."], [3, 3, "This is note #3. It just takes up space."], [4, 4, "This is note #4. It just takes up space."], [5, 5, "This is note #5. It just takes up space."], [6, 6, "This is note #6. It just takes up space."], [7, 7, "This is note #7. It just takes up space."], [8, 8, "This is note #8. It just takes up space."], [9, 9, "This is note #9. It just takes up space."], [10, 10, "This is note #10. It just takes up space."], [11, 11, "This is note #11. It just takes up space."], [12, 12, "This is note #12. It just takes up space."], [13, 13, "This is note #13. It just takes up space."], [14, 14, "This is note #14. It just takes up space."], [15, 15, "This is note #15. It just takes up space."], [16, 16, "This is note #16. It just takes up space."], [17, 17, "This is note #17. It just takes up space."], [18, 18, "This is note #18. It just takes up space."], [19, 19, "This is note #19. It just takes up space."], [20, 20, "This is note #20. It just takes up space."], [21, 21, "This is note #21. It just takes up space."], [22, 22, "This is note #22. It just takes up space."], [23, 23, "This is note #23. It just takes up space."], [24, 24, "This is note #24. It just takes up space."], [25, 25, "This is note #25. It just takes up space."], [26, 26, "This is note #26. It just takes up space."], [27, 27, "This is note #27. It just takes up space."], [28, 28, "This is note #28. It just takes up space."], [29, 29, "This is note #29. It just takes up space."], [30, 30, "This is note #30. It just takes up space."], [31, 31, "This is note #31. It just takes up space."], [32, 32, "This is note #32. It just takes up space."], [33, 33, "This is note #33. It just takes up space."], [34, 34, "This is note #34. It just takes up space."], [35, 35, "This is note #35. It just takes up space."], [36, 36, "This is note #36. It just takes up space."], [37, 37, "This is note #37. It just takes up space."], [38, 38, "This is note #38. It just takes up space."], [39, 39, "This is note #39. It just takes up space."], [40, 40, "This is note #40. It just takes up space."], [41, 41, "This is note #41. It just takes up space."], [42, 42, "This is note #42. It just takes up space."], [43, 43, "This is note #43. It just takes up space."], [44, 44, "This is note #44. It just takes up space."], [45, 45, "This is note #45. It just takes up space."], [46, 46, "This is note #46. It just takes up space."], [47, 47, "This is note #47. It just takes up space."], [48, 48, "This is note #48. It just takes up space."], [49, 49, "This is note #49. It just takes up space."], [50, 50, "This is note #50. It just takes up space."], [51, 51, "This is note #51. It just takes up space."], [52, 52, "This is note #52. It just takes up space."], [53, 53, "This is note #53. It just takes up space."], [54, 54, "This is note #54. It just takes up space."], [55, 55, "This is note #55. It just takes up space."], [56, 56, "This is note #56. It just takes up space."], [57, 57, "This is note #57. It just takes up space."], [58, 58, "This is note #58. It just takes up space."], [59, 59, "This is note #59. It just takes up space."], [60, 60, "This is note #60. It just takes up space."], [61, 61, "This is note #61. It just takes up space."], [62, 62, "This is note #62. It just takes up space."], [63, 63, "This is note #63. It just takes up space."], [64, 64, "This is note #64. It just takes up space."], [65, 65, "This is note #65. It just takes up space."], [66, 66, "This is note #66. It just takes up space."], [67, 67, "This is note #67. It just takes up space."], [68, 68, "This is note #68. It just takes up space."], [69, 69, "This is note #69. It just takes up space."], [70, 70, "This is note #70. It just takes up space."], [71, 71, "This is note #71. It just takes up space."], [72, 72, "This is note #72. It just takes up space."], [73, 73, "This is note #73. It just takes up space."], [74, 74, "This is note #74. It just takes up space."], [75, 75, "This is note #75. It just takes up space."]]'''

[mikelygee]

The same test behaves just fine when I use pry instead of irb, so I now suspect inf-ruby is not be to blame.

[yantar92]

For context, these 4k blocks may be connected with PTY process connection type. Some terminal implementations forcefully chunk input into 4k segments. So, changing the process connection type (if is it at all possible), might be the solution.

See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=6149 for details.

[dgutov]

It should use pty already, as long as process-connection-type is at its default value (t).

I wonder what difference could using Pry make.

Otherwise, yes, the string length is just above this limit above, so it seems related.

[yantar92]

It should use pty already, as long as process-connection-type is at its default value (t).

In the original report [1], the problem only occured in NixOS. @mikelygee, can you check the value of process-connection-type in Emacs that use run in NixOS? May it be not t?

[1] https://list.orgmode.org/orgmode/980379b9-3b1c-7671-839a-60fa488eed7c@gmail.com/

[dgutov]

I also see two bindings for process-connection-type inside the Org codebase, but they seem to be on different execution paths. Correct me if I'm wrong, though.

[yantar92]

Dmitry Gutov ***@***.***> writes:

I also see two bindings for `process-connection-type` inside the Org codebase, but they seem to be on different execution paths. Correct me if I'm wrong, though.

You are right. They are for (1) connecting to child Emacs process during export; (2) running xdg-open when opening file links, which had some bugs with the default t value. ob-ruby simply relies upon `comint-send-input`.

…

-- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>

[mikelygee]

It should use pty already, as long as process-connection-type is at its default value (t).

In the original report [1], the problem only occured in NixOS. @mikelygee, can you check the value of process-connection-type in Emacs that use run in NixOS? May it be not t?

[1] https://list.orgmode.org/orgmode/980379b9-3b1c-7671-839a-60fa488eed7c@gmail.com/

It is t.

I have found that (comint-send-string) fails on both irb and pry, apparently truncating the string at ~4096 characters. I do not know if this is related to (ruby-send-region).

[mikelygee]

Using irb, (ruby-send-region)` reports:

irb(main):013:0" irb(main):014:0" (irb):12:in `eval': /home/mike/AmberleyFarmersMarket/test.def:1: unterminated string meets end of file (SyntaxError)
...t takes up space.'], [74, 74, '
...                               ^
/home/mike/AmberleyFarmersMarket/test.def:1: syntax error, unexpected end-of-input, expecting ']'
...t takes up space.'], [74, 74, '
...                               ^

	from (irb):12:in `<main>'
	from /nix/store/5znr7dy56x68lp25ys45699zzmp15zgb-ruby-3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'
	from /nix/store/5znr7dy56x68lp25ys45699zzmp15zgb-ruby-3.2.2/bin/irb:25:in `load'
	from /nix/store/5znr7dy56x68lp25ys45699zzmp15zgb-ruby-3.2.2/bin/irb:25:in `<main>'
irb(main):015:0> 

In contrast, (comint-send-string) leaves the ruby process ready for more input, so if I type in a closing quote and square brackets (']]), the process accepts the completed line.

[dgutov]

I have found that (comint-send-string) fails on both irb and pry, apparently truncating the string at ~4096 characters. I do not know if this is related to (ruby-send-region).

ruby-send-region calls comint-send-string internally.

In contrast, (comint-send-string) leaves the ruby process ready for more input, so if I type in a closing quote and square brackets (']]), the process accepts the completed line.

The difference is likely due to the method: ruby-send-region constructs a heredoc expression and calls eval on it, together with file name and line (so that all definitions are evaluated with proper locations). But when a heredoc is truncated, it's much harder to close than sending some closing brackets.

[dgutov]

I have found that (comint-send-string) fails on both irb and pry, apparently truncating the string at ~4096 characters.

So... there is no difference between Irb and Pry, in the context of this problem?

Unless somebody has better ideas, we could indeed revert to measuring the spans chars without newlines, and if one exceeds 4096, show an error to the user instead. Maybe upstream (Emacs) does that first, though.

[bo-tato]

It should use pty already, as long as process-connection-type is at its default value (t).

In the original report [1], the problem only occured in NixOS. @mikelygee, can you check the value of process-connection-type in Emacs that use run in NixOS? May it be not t?

[1] https://list.orgmode.org/orgmode/980379b9-3b1c-7671-839a-60fa488eed7c@gmail.com/

I can confirm I get the same issue on emacs on debian, with both irb and pry. with process-connection-type as its default t. But I think there is a misunderstanding here, it is pty on linux that truncates lines to 4096. Setting process-connection-type to nil, meaning using plain pipes with no pty, and it works fine sending longer lines. But with plain pipes there is the problem that it echoes all input. Setting comint-process-echoes fixes that for stuff you type into the repl, which calls comint-send-input which takes comint-process-echoes into account, but send-region which calls comint-send-string and comint-send-region directly call the process-send-region or string function and aren't affected by comint-process-echoes.

[dgutov]

Should be fixed with #175.