Lem with other CL implementations #686

Sasanidas · 2023-06-05T03:20:13Z

So, this is the first try of making Lem compile/work with other implementations, I manage to make it compile the ncurses front-end with abcl,:

(It should of course also work with SBCL)

Compilation progress (make the :lem system compile)

ABCL
CCL
ECL
Clasp

Most tests green

ABCL
CCL
ECL
Clasp

Front-end ncurses progress (run the system)

ABCL
CCL
ECL
Clasp

"Usable state" (it can be used like sbcl)

ABCL
CCL
ECL
Clasp

- Modify the mailbox implementation - Add +sbcl macro to a lot of places - It compiles with ABCL v1.9.0

Sasanidas · 2023-06-05T05:00:49Z

I had better luck with CCL, which works quite better, except for some errors here and there, it's usable 👍

Sasanidas · 2023-06-05T05:18:21Z

So, this is the first try of making Lem compile/work with other implementations, I manage to make it compile the ncurses front-end with abcl,:

Buut sadly, when launching (lem:lem), it just shows the tmp buffer and doesn't respond to any command:

The good news, is that it also is working with sbcl, but of course more testing is needed +1

Same thing with ECL, it compiles but it doesn't register any input

Sasanidas · 2023-06-05T06:05:46Z

So, loading the ncurses front-end, I realize that ABCL and ECL have the same problem, no response from the user input.

I debug the *editor-event-queue* and seems like the events are indeed been receive (it shouldn't be a problem in the change of timer because SBCL-Lem does work)

Maybe a library that I missed ❓

cxxxr · 2023-06-05T06:32:04Z

That's wonderful, thank you!

ECL should have worked before, maybe there is a problem around threads
I'll dig a little deeper tonight (at JST) and see what I can find.

- These need to be adapted for the new bt library

cxxxr · 2023-06-05T14:12:13Z

ECL is failing a lot of tests, so it might not work without going through that one.
Of course, I'll leave the policy up to you.

Sasanidas · 2023-06-05T18:48:11Z

Yeah, I had some problems with ECL, specially the annoying ASDF version error (not related to Lem), so I think I'm going to focus on ABCL, I think it's the one with more potential uses (open the Java ecosystem)

- Modify timer and the tests to reflect the changes

Sasanidas · 2023-06-06T10:20:18Z

I did a couple of things in the last commit c047386 :

I adapt the library mailbox from SBCL to a CL ANSI version, all the tests seems to pass but still there is an ignore errors that I'm not sure how to fix (I'll add a TODO in it)
Modify again timer to reflect these changes
Also I add all the tests from the mailbox SBCL to this ANSI version to the rove framework, and all green for now.

I still didn't manage to make it work with ABCL tho, but I think I'm in the right direction here 👍

Sasanidas · 2023-06-06T10:52:55Z

Maybe the mailbox library can be separated in a separate repository?
I couldn't found anything similar online (there is mailbox, but it's quite limited compare to this version), and it can be used for other people/projects

cxxxr · 2023-06-06T11:40:45Z

Maybe the mailbox library can be separated in a separate repository?

It's might be good.

Sasanidas · 2023-06-06T14:09:36Z

I added some documentation more detail on how the progress in the tests is going (using this branch as reference)
lem-project/lem-project.github.io@8728344

Sasanidas · 2023-06-10T13:08:42Z

OK, I think I found THE problem:

So, the thing is when calling collecting-subclasses, in signal condition in signal handler, it breaks because the subclass cannot be coerce into a simple condition.

So, why it works in SBCL? Because is seems like it's doing some wrapping around the classes, let's see the call to the function (this functions returns a list, in this case, I'm inspecting the first element):

(collect-subclasses 'lem-core::before-executing-command :include-itself nil)

SBCL:

Clasp:

In ABCL happens the same as in Clasp.

So, in the first case, it can be signal without any problem, but in the second it cannot be coerce. I guess the odd (or clever) behaviour is in SBCL.

Not sure what to do here, so calling for some help (@cxxxr)

cxxxr · 2023-06-10T13:21:51Z

Hmm, this seems to be a deep-seated problem.
The signal-holder mechanism is unnecessarily complicated and may be better replaced by a hook.

cxxxr · 2023-06-11T09:44:29Z

@Sasanidas
#716
I think I've merged the PR and this problem has been resolved, what do you think?

Sasanidas · 2023-06-11T09:46:32Z

@Sasanidas #716 I think I've merged the PR and this problem has been resolved, what do you think?

Oh! seems very interesting, I'll merge the branch and try it 👍

Sasanidas · 2023-06-17T13:40:56Z

Even tho ECL works really well (maybe the best aside from SBCL) I wasn't been able to dump the image to an executable.

Running https://ecl.common-lisp.dev/static/manual/System-building.html#Compiling-with-ASDF does creates an executable, but it the stack crashes when I try to run it.

Sasanidas · 2023-06-18T15:45:30Z

Using (#756) to document different implementations, some of the results are now in https://lem-project.github.io/lem-page/development/implementations-details/

Sasanidas · 2023-06-20T08:45:08Z

OK, seems like the CCL display bug was because in the micros backend, the function-name implementation was returning a symbol instead of a string, to ensure that this also works with other implementations, I format the function name to a string 👍

Sasanidas · 2023-06-20T08:45:54Z

At this point, CCL works as well as SBCL (except for lsp, but that's because of the sb-concurrency library usage, which I think it's a problem for other issue 👍 )

Sasanidas · 2023-06-22T10:55:49Z

I think what is left in this issue is to test both ECL and Clasp, upload the results and create a few specific issues for implementation (here we can also use github labels), let me know what do you think.

cxxxr · 2023-06-22T11:00:48Z

Yes, it might be a good idea to merge at a reasonable point for now, and separate the pull requests.

Sasanidas · 2023-06-22T11:06:29Z

Yes, it might be a good idea to merge at a reasonable point for now, and separate the pull requests.

Great! I'll ping you when the test are ready 👍

Sasanidas · 2023-06-22T14:53:09Z

OK, so I have to tweak a little bit the script to make it work with Clasp, but I think this issue is ready to merge 👍 (ping @cxxxr )

cxxxr

This is a very attractive change!
I am very thankful.

I just need a few more corrections 🙏

cxxxr · 2023-06-22T15:06:06Z

src/system.lisp

@@ -5,7 +5,7 @@
 (defun lem-relative-pathname (pathname)
  (when *deployed*
    (truename (merge-pathnames pathname
-                               (uiop:pathname-directory-pathname (first sb-ext:*posix-argv*))))))
+                               (uiop:pathname-directory-pathname "/home")))))


Isn't this environment-dependent?

cxxxr · 2023-06-22T15:07:00Z

src/version.lisp

+	 #+sbcl
         (with-output-to-string (stream)
           (uiop:run-program "git rev-parse --short HEAD"
-                             :output stream)))))))
+                             :output stream))


Isn't the run-program portable?

cxxxr · 2023-06-22T15:08:20Z

src/common/timer.lisp

  (start-timer-thread timer ms repeat-p)
  timer)

 (defmethod stop-timer ((timer timer))
  (stop-timer-thread timer))

 (defun start-timer-thread (timer ms repeat-p)
-  (let ((stop-mailbox (sb-concurrency:make-mailbox))
+  (let ((stop-mailbox (lem-mailbox::make-mailbox))


Don't you export symbols for mailboxes?

Sasanidas · 2023-06-23T10:29:54Z

Let me know if there's anything more to change 👍 (@cxxxr )

cxxxr · 2023-06-23T10:49:03Z

Thank you very much.
I think it's very good.
I appreciate your efforts over the past month.

First try

40edc5f

- Modify the mailbox implementation - Add +sbcl macro to a lot of places - It compiles with ABCL v1.9.0

Sasanidas changed the title ~~First try~~ Lem with other CL implementations Jun 5, 2023

Fix compilation error

2166bd1

Sasanidas mentioned this pull request Jun 5, 2023

Run Lem with other Common Lisp implementations #680

Closed

Sasanidas added 3 commits June 5, 2023 12:15

Make tests compile

4e63bd1

Comment these two tests for now

a4fe3b3

- These need to be adapted for the new bt library

Fix (mostly) the timer tests

76f571f

Adapt the mailbox sbcl library to a CL ANSI version

c047386

- Modify timer and the tests to reflect the changes

Sasanidas added 6 commits June 6, 2023 16:10

ABCL uses NIL instead of 0 for the emptiness in the array

fdf2019

Mark this tests as only SBCL for now

d878fa9

Merge branch 'main' into other-lisp-implementations

2b7ac45

Remove third-party directory

24e26a1

Move mailbox to submodules folder

296d93c

Fix loading lem-scheme-mode typo

476e4e2

Merge branch 'lem-project:main' into other-lisp-implementations

50e67cb

cxxxr mentioned this pull request Jun 11, 2023

To deprecated signal-hanlder mechanism #716

Merged

Add ccl to the build binary

0876d42

Fix CCL display bug

6c012c9

Sasanidas added 8 commits June 20, 2023 10:51

Remove ccl macros in timer

651b8bf

Keep loading sb-concurrency for sbcl

7e2c034

Add sbcl macros for other implementations

910cf9a

Add sbcl macros to documentation-mode/utils

a62f2c5

Change generate-markdown-file to a generic

8288168

Add documentation-tests file

d0e23be

Add rove to dependencies

070d82b

Move documentation test file to scripts

5cfbfaf

Sasanidas added 3 commits June 22, 2023 15:26

Merge branch 'main' into other-lisp-implementations

716cad4

Fix merge

f44ea18

Enable documentation-mode for other implementations

5c12b6d

Sasanidas marked this pull request as ready for review June 22, 2023 14:53

cxxxr reviewed Jun 22, 2023

View reviewed changes

Sasanidas added 2 commits June 22, 2023 17:20

Add workaround for non-sbcl implementations

897b15e

Use exported mailbox symbols

6fda972

cxxxr merged commit dc568f3 into lem-project:main Jun 23, 2023
1 check passed

cxxxr mentioned this pull request Jul 11, 2023

Release v2.1 #823

Closed

6 tasks

dariof4 mentioned this pull request Aug 27, 2023

Package request: lem NixOS/nixpkgs#250777

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lem with other CL implementations #686

Lem with other CL implementations #686

Sasanidas commented Jun 5, 2023 •

edited

Loading

Sasanidas commented Jun 5, 2023

Sasanidas commented Jun 5, 2023

Sasanidas commented Jun 5, 2023 •

edited

Loading

cxxxr commented Jun 5, 2023

cxxxr commented Jun 5, 2023

Sasanidas commented Jun 5, 2023

Sasanidas commented Jun 6, 2023

Sasanidas commented Jun 6, 2023

cxxxr commented Jun 6, 2023

Sasanidas commented Jun 6, 2023

Sasanidas commented Jun 10, 2023 •

edited

Loading

cxxxr commented Jun 10, 2023

cxxxr commented Jun 11, 2023

Sasanidas commented Jun 11, 2023

Sasanidas commented Jun 17, 2023

Sasanidas commented Jun 18, 2023

Sasanidas commented Jun 20, 2023

Sasanidas commented Jun 20, 2023

Sasanidas commented Jun 22, 2023 •

edited

Loading

cxxxr commented Jun 22, 2023

Sasanidas commented Jun 22, 2023

Sasanidas commented Jun 22, 2023 •

edited

Loading

cxxxr left a comment

cxxxr Jun 22, 2023

cxxxr Jun 22, 2023

cxxxr Jun 22, 2023

Sasanidas commented Jun 23, 2023

cxxxr commented Jun 23, 2023

Lem with other CL implementations #686

Lem with other CL implementations #686

Conversation

Sasanidas commented Jun 5, 2023 • edited Loading

Sasanidas commented Jun 5, 2023

Sasanidas commented Jun 5, 2023

Sasanidas commented Jun 5, 2023 • edited Loading

cxxxr commented Jun 5, 2023

cxxxr commented Jun 5, 2023

Sasanidas commented Jun 5, 2023

Sasanidas commented Jun 6, 2023

Sasanidas commented Jun 6, 2023

cxxxr commented Jun 6, 2023

Sasanidas commented Jun 6, 2023

Sasanidas commented Jun 10, 2023 • edited Loading

cxxxr commented Jun 10, 2023

cxxxr commented Jun 11, 2023

Sasanidas commented Jun 11, 2023

Sasanidas commented Jun 17, 2023

Sasanidas commented Jun 18, 2023

Sasanidas commented Jun 20, 2023

Sasanidas commented Jun 20, 2023

Sasanidas commented Jun 22, 2023 • edited Loading

cxxxr commented Jun 22, 2023

Sasanidas commented Jun 22, 2023

Sasanidas commented Jun 22, 2023 • edited Loading

cxxxr left a comment

Choose a reason for hiding this comment

cxxxr Jun 22, 2023

Choose a reason for hiding this comment

cxxxr Jun 22, 2023

Choose a reason for hiding this comment

cxxxr Jun 22, 2023

Choose a reason for hiding this comment

Sasanidas commented Jun 23, 2023

cxxxr commented Jun 23, 2023

Sasanidas commented Jun 5, 2023 •

edited

Loading

Sasanidas commented Jun 5, 2023 •

edited

Loading

Sasanidas commented Jun 10, 2023 •

edited

Loading

Sasanidas commented Jun 22, 2023 •

edited

Loading

Sasanidas commented Jun 22, 2023 •

edited

Loading