Common Lisp/ros/swank not behaving as expected

[kflak]

Hi,

Don't know quite how to formulate this, as I'm new to the CL ecosystem, but I am able to run this code within a ros repl with no issue:

(ql:quickload :cl-collider)
(in-package :sc-user)
(named-readtables:in-readtable :sc)

(setf *s* (make-external-server "localhost" :port 48800))
(server-boot *s*)

(defvar *synth*)
(setf *synth* (play(sin-osc.ar [320 321] 0 0.2)))

However, when running it in nvim/conjure with

❯ ros run --eval '(ql:quickload :swank)' --eval '(swank:create-server :dont-close t)'

I get this:

; 127.0.0.1:4005 (connected)
; with-protoent
; --------------------------------------------------------------------------------
; eval (root-form): (ql:quickload :cl-collider)
; To load "cl-collider":
;   Load 1 ASDF system:
;     cl-collider
; ; Loading "cl-collider"
; 
; 
(:CL-COLLIDER)
; --------------------------------------------------------------------------------
; eval (root-form): (in-package :sc-user)
#<PACKAGE "SC-USER">
; --------------------------------------------------------------------------------
; eval (root-form): (named-readtables:in-readtable :sc)
#<NAMED-READTABLE :SC {1005AB0633}>
; --------------------------------------------------------------------------------
; eval (root-form): (setf *s* (make-external-server "localhost...
; The function COMMON-LISP-USER::MAKE-EXTERNAL-SERVER is undefined.
; --------------------------------------------------------------------------------
; eval (root-form): (server-boot *s*)
; The variable *S* is unbound.
; --------------------------------------------------------------------------------
; eval (root-form): (defvar *synth*)
*SYNTH*
; --------------------------------------------------------------------------------
; eval (root-form): (setf *synth* (play(sin-osc.ar [320 321] 0...
; The function COMMON-LISP-USER::SIN-OSC.AR is undefined.

Running on arch linux with ros-installed sbcl:

❮ ros config  
sbcl.version=2.4.2
setup.time=3918183544
sbcl-bin.version=2.4.1
default.lisp=sbcl

Not sure if it's got to do with cl-collider only, or if it's a general issue... Any help much appreciated!

[Olical]

Hey sorry about the bug, it's because Conjure is swapping the package to "common-lisp-user" on each eval because it's looking for a defpackage form and not finding it. It's related to #557

I'm just trying to add a fix now that is a best effort contextual eval that should support multiple defpackage / in-package forms in a file. Conjure's context system was designed with Clojure in mind with one ns form at the top and you generally keep the link of file-namespace name. It looks like in CL people swap the package a lot in some files, so I haven't accounted for that yet.

I'm not a CL user so it's a bit unfamiliar to me, but I'll add a best effort fix for this to develop now.

[kflak]

Great, thanks! I'm also brand new to the whole world of CL, so it's all a bit confusing :-)

[Olical]

Pushed a change to develop that I think addresses this? We now search up through the lines from your cursor position for the nearest in-package or defpackage then try to assume that package name with the eval you're doing. So it should support multiple regions in your files where you swap between packages for different calls.

I think that's useful for CL? But I'm really not sure! Let me know if that works for you. I can call the functions you listed in your example now but I do run into some other kind of errors, I think because I don't have whatever dependencies it needs installed.

[kflak]

Wow, that seems to work! Thanks a lot.

A couple of initial tests:

(in-package :sc-user)
(named-readtables:in-readtable :sc)

(setf *s* (make-external-server "localhost" :port 48800))
(server-boot *s*)

(defvar *synth*)

(setf *synth* (play(sin-osc.ar [320 321] 0 0.2))) ;; evaluating root form here...
(free *synth*) ;; ... also evaluates this.

(server-quit *s*) ;; evaluating root form on this also evaluates the previous two forms.

Not sure if this is expected behavior....

[Olical]

Cool! That eval capturing other forms is definitely not expected though. Are you using Tree Sitter? If not, well worth a try, it's so much more robust.

…

On Fri, 1 Mar 2024, 14:02 Kenneth Flak, ***@***.***> wrote: Wow, that seems to work! Thanks a lot. A couple of initial tests: (in-package :sc-user) (named-readtables:in-readtable :sc) (setf *s* (make-external-server "localhost" :port 48800)) (server-boot *s*) (defvar *synth*) (setf *synth* (play(sin-osc.ar [320 321] 0 0.2))) ;; evaluating root form here... (free *synth*) ;; ... also evaluates this. (server-quit *s*) ;; evaluating root form on this also evaluates the previous two forms. Not sure if this is expected behavior.... — Reply to this email directly, view it on GitHub <#561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACM6XJ6NIBX7B4WX66F5NTYWCC6ZAVCNFSM6AAAAABD7POJBGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZTGI3DAMRYGE> . You are receiving this because you commented.Message ID: ***@***.***>

[kflak]

Yup, using treesitter. I did notice, however, that the highlighting is off in a way that suggests something is not quite right here...:

[Olical]

Good spot! That's 100% related, the highlighting and Conjure's view of your code tree are interlinked in tree sitter. So if what you're seeing is wrong, Conjure is seeing the same when it queries the tree. You might have to update your tree sitter parsers, I've had to do that between Neovim versions before otherwise I saw similar problems.

[kflak]

OK, so treesitter is to blame here, I think :-) Tried updating it, but seems it hasn't seen an update for 4 months, so there may be some breaking changes in the treesitter since then. When I disable treesitter for commonlisp the evaluation works as expected. If I have a bit more time I'll open an issue over at cl-treesitter!

[kflak]

Ah, figured out where treesitter stumbles: it doesn't like the square brackets in this expression:

(setf *synth* (play(sin-osc.ar [320 321] 0 0.2))) ;; evaluating root form here...

When I change those to regular parentheses it all works fine.

[russtoku]

@Olical commented:

I'm just trying to add a fix now that is a best effort contextual eval that should support multiple defpackage / in-package forms in a file. Conjure's context system was designed with Clojure in mind with one ns form at the top and you generally keep the link of file-namespace name. It looks like in CL people swap the package a lot in some files, so I haven't accounted for that yet.

I think that it should be the nearest preceeding in-package form from the form or item being evaluated. I've put a comment regarding this on the discussion #557.

But I will try the fix added to the develop branch to see how things works so far.

[russtoku]

I tried the fix added to the develop branch (20df0e0) and it seems to work.

[Olical]

Oh okay, so defpackage does NOT swap your current package like Clojure's na form? You NEED both?

…

On Sat, 2 Mar 2024, 08:03 Russ Tokuyama, ***@***.***> wrote: I tried the fix added to the develop branch (20df0e0 <20df0e0>) and it seems to work. — Reply to this email directly, view it on GitHub <#561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACM6XM3I2VW6Q6CD7CTBN3YWGBTLAVCNFSM6AAAAABD7POJBGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZUGY4TQMZTGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[russtoku]

No, just in-package should be OK.

[Olical]

So are we happy with this now? I'm going to merge anyway and get it in more hands, I think it's solid theoretically from what I understand of CL now. Thanks for doing research, sharing it and testing this @russtoku ❤️

[Olical]

Closing this issue because I think we did it! Please re-open if you run into issues!