REPL window gets sluggish with large data structures

[PEZ]

When the repl window is handed a somewhat large dataset, it gets really slow and sluggish.

Reproduce by either evaluating something like:

(map (fn [x] {x (range 7)}) (range 1000))

You can also paste a large data structure. For instance copy the last result after evaluationg the above, ctrl+alt+c ctrl+c. And then paste it.

It takes a too long while for the results to appear.

This could probably partially be handled by coming enhancements to nrepl: nrepl/nrepl#106

But it would still leave the case with pasting large chunks of data.

[Gnurdle]

perhaps give some thought to something akin to 'cider-inspect' which would be useful in its own right.

[PEZ]

I am not familiar with cider-inspect. Can you describe it some, and how you think it would help here?

I started to think about this problem yesterday before falling asleep. My ideas in that stage are mostly crap, but maybe this one could be an interim solution until someone finds out what would bring the REPL window the performance it needs:

1. Detect when the output is too large for comfort.
2. Print only part of the output, and add a button to click at to see the rest. The rest could then open in a regular editor that we enable Clojure mode in.

Obviously needs some more thought around the UI/Ux, but printing results to a regular Clojure editor should be handy for many situations.

[brunchboy]

cider-inspect allows you to interactively traverse Clojure and Java data structures, paging through lists and vectors, and diving into and out of objects. It can even handle structures with cycles in them, which would never finish printing at the REPL, because the user guides the search through the object space, and nodes are only expanded as they are reached. I came to this issue trying to determine if Calva had a similar feature, to see if I could recommend it as an alternative to my users (because some of my data structures do have cycles), but so far it is looking like CIDER is the only option.

[PEZ]

Thanks, @brunchboy. It sound a bit like REBL, have I understood it correctly if I think like that?

[brunchboy]

Hmm, I had not heard of REBL before, but it does look similar, except that cider-inspect works over the nREPL server so you can do it to a remote process (which is the use case that I need, because my users are running my Beat Link Trigger as a standalone Jar file or native application with embedded Java runtime).

[PEZ]

Thanks again! There is nREPL middleware for REBL. If you have some little time to spend on it, I would like to know if a tighter/easier REBL integration would be a feasible way to compensate for Calva's lack of inspector. And also, of course if it even works in the setup you describe there. Check out this page on using REBL with Calva: https://github.com/BetterThanTomorrow/calva/wiki/Using-with-REBL (If you have the time, that is.)

[brunchboy]

I’m afraid I probably won’t have time to research anything further. I don’t use Calva myself, I was just trying to see if I could point my users to it as an alternative when they were picking Clojure IDEs to get more serious about writing extensions. But I don’t have enough time as it is to implement and document the features I want to work on. 😄

However, even without doing any further research, I do see a big problem with REBL, which is the license. I allow my users to use my software freely for whatever purpose, but they would not be able to use REBL to develop their shows (that would be considered a commercial use, because they are getting paid to perform lights, lasers, video, or music), and none of them will be Datomic customers (as they are not programmers), nor will necessarily want to contribute on the REBL Patreon. I would not be comfortable telling them to do that, when I don’t tell them to pay me for my own work.

[PEZ]

Yeah, the license. Thanks for sharing these thoughts and weighing in on this issue. Seem's we'll need to fix this ASAP. Watch this space.