Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I use april-c to bind varible,however Cannot save core with multiple threads running. #266

Closed
secwang opened this issue Dec 28, 2022 · 19 comments

Comments

@secwang
Copy link

secwang commented Dec 28, 2022

(sb-ext:save-lisp-and-die #p"my-app-binary" :toplevel #'b:run :executable t)

debugger invoked on a SB-IMPL::SAVE-WITH-MULTIPLE-THREADS-ERROR in thread
#<THREAD "main thread" RUNNING {1001090003}>:
Cannot save core with multiple threads running.

Interactive thread (of current session):
#<THREAD "main thread" RUNNING {1001090003}>

Other threads:
#<THREAD "april-language-kernel" waiting on: # {1007A909B3}>,
#<THREAD "april-language-kernel" waiting on: # {1007A90663}>,
#<THREAD "april-language-kernel" waiting on: # {1007A916F3}>,
#<THREAD "april-language-kernel" waiting on: # {1007A91053}>,
#<THREAD "april-language-kernel" waiting on: # {1007A90D03}>,
#<THREAD "april-language-kernel" waiting on: # {1007A913A3}>,
#<THREAD "april-language-kernel" waiting on: # {1007A91A43}>,
#<THREAD "april-language-kernel" waiting on: # {1007A91D93}>
See also:
The SBCL Manual, Node "Saving a Core Image"

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.

(SB-IMPL::DEINIT)

@phantomics
Copy link
Owner

I tried this, is this similar to what you're doing?

* (asdf:load-system 'april)

* (defvar bla (april:april-c "{⍵×⍳10}" 15))

* (defun dorun () (loop :for i :while t :do (sleep 0.1)))

* (sb-ext:save-lisp-and-die #p"/tmp/lbinary" :toplevel #'pkg:dorun :executable t)

Then I got the sb-impl::save-error but no language telling me that it was because of the multiple threads.

@secwang
Copy link
Author

secwang commented Dec 28, 2022

I tried to use april with postmorden. Here is my code.
The strange thing here is that if I load this code, it can be executed, so according to my experience, I think it can generally be build into exe.

(load "~/quicklisp/setup.lisp")

(ql:quickload '(postmodern april)  :silent t)

(defpackage :b (:use :cl :postmodern :april) (:export :run))
(in-package :b)

(defparameter cns '("postgres" "postgres" "password" "host"))


(defun list-to-2d-array (list)
  (make-array (list (length list)
                    (length (first list)))
              :initial-contents list))

(defun run ()
    (with-connection cns
      (let* ((a  (query "select * from table ;"))
	     (av (list-to-2d-array a)))
	(format t  "~A~%" (april-c "{[a]  +/ a[;6] > 0 }" av))
	(format t "~A~%" (april-c "{[a] +/ a[;6] < 0 }" av))
	(format t "~:d~%" (round (april-c "{[a] +/ a[;7] }" av)) )
	)))

@secwang
Copy link
Author

secwang commented Dec 28, 2022

I regenerate error by follow code.

(asdf:load-system 'april)
(defvar bla (april:april-c "{⍵×⍳10}" 15))
(defun dorun () bla)  
(sb-ext:save-lisp-and-die "app" :toplevel #'dorun :executable t)

debugger invoked on a SB-IMPL::SAVE-WITH-MULTIPLE-THREADS-ERROR in thread
#<THREAD "main thread" RUNNING {10044601E3}>:
Cannot save core with multiple threads running.

Interactive thread (of current session):
#<THREAD "main thread" RUNNING {10044601E3}>

Other threads:
#<THREAD "april-language-kernel" waiting on: # {1002DE89B3}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE8663}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE96F3}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE9053}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE8D03}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE93A3}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE9A43}>,
#<THREAD "april-language-kernel" waiting on: # {1002DE9D93}>
See also:
The SBCL Manual, Node "Saving a Core Image"

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.

SBCL 2.2.11, On OS X. Install april from quicklisp locally dir.

@phantomics
Copy link
Owner

I've been trying this myself but on Linux kernel 5.4.0-17.1 with SBCL 2.1.7 I just get:

Could not save core.
   [Condition of type SB-IMPL::SAVE-ERROR]

I can try updating the SBCL version and seeing if that works.

Also,. can you try running this as the last line in your test instead of the save-lisp-and-die and see what it returns?

(loop :for worker :across (lparallel.kernel::workers lparallel::*kernel*) :collect (lparallel.kernel::running-category worker))

This will tell you if any threads in the system are busy; if not it should just return a list of NILs.

@secwang
Copy link
Author

secwang commented Jan 9, 2023

run with repl

(asdf:load-system 'april)
(defvar bla (april:april-c "{⍵×⍳10}" 15))
(defun dorun () bla) 
(loop :for worker :across (lparallel.kernel::workers lparallel::*kernel*) :collect (lparallel.kernel::running-category worker))
* BLA
* DORUN
* (NIL NIL NIL NIL NIL NIL NIL NIL)

in addition, my mac sbcl version is 2.3.0

@phantomics
Copy link
Owner

Thanks, looks like nothing's out of the ordinary in your threading kernel. I'll check into possible issues with SBCL and lparallel; for some reason I get the same Could not save core. error even if I try to save a bare system state just after starting up without loading April or any other package.

@secwang
Copy link
Author

secwang commented Jan 9, 2023

I suggest we can try the sbcl2.30 test? In my limited experience of using sbcl, this kind of failure to save to executable is relatively rare.

@phantomics
Copy link
Owner

I'm building 2.3.0 now, I'll see how it handles saving binaries.

@phantomics
Copy link
Owner

2.3.0 is built and I'm seeing the same error you do. Building binaries without April loaded is working. I'll see what I can find from this point.

@secwang
Copy link
Author

secwang commented Jan 9, 2023 via email

@phantomics
Copy link
Owner

Try putting this in a file and loading it:

(asdf:load-system 'april)
(defvar bla (april:april-c "{⍵×⍳10}" 15))
(defun dorun () bla)
(lparallel::end-kernel :wait t)
(sb-ext:save-lisp-and-die "/tmp/app" :toplevel #'dorun :executable t)

The (end-kernel) will shut down all lparallel workers. This won't work if you do it from a REPL because there will be another kernel with Swank-related tasks still running.

@secwang
Copy link
Author

secwang commented Jan 16, 2023

It works , great work!

@secwang secwang closed this as completed Jan 16, 2023
@phantomics
Copy link
Owner

Great, I'm interested to hear more about your April use case if possible. Let me know if there are any further issues.

@secwang
Copy link
Author

secwang commented Jan 16, 2023 via email

@phantomics
Copy link
Owner

Interesting. If you encounter any situations where April seems particularly slow let me know, there are places where significant optimization is possible but it's a matter of finding these cases in order to make improvements.

@secwang
Copy link
Author

secwang commented Jan 17, 2023

Under my test, the same task, the compiled runtime is almost twice longer that of dyalogapl. If it is to explain execution, the quickload of cl will take more time. To be honest, I haven't found out why april is slow. The current conclusion is like this.

However, because I am familiar with cl, april is actually the most convenient apl to process input and output and refer to other array languge. The same task, in dyalogapl and k/q, needs to be very painfully built from the bottom and try various environmental problems. I think even if april is not the fastest apl, it is actually a space in engineering.

Another problem for me is that I still need time to learn apl. Personally, I think dyalog actually provides very good documentation and entry-level environment and development support because of their revenue models. This is a good place to learn apl. This may be a very difficult place for april as open source software. In fact, I don't think there are many people familiar with common lisp and array language in my background. April may need to provide more direct repl. Of course, this is a very good project. Thank you. Cheers.

On the other point, I think the reason why common lisp is not so popular is that people always have to learn emacs. Learning the learning curve of lisp and emacs at the same time is crazy. Of course, emacs is a good environment, but it is not short/simple enough. A short enough environment, similar to kdb+ repl, I think it makes sense to attract novices.

@phantomics
Copy link
Owner

April now has a REPL mode for Emacs, see: https://github.com/phantomics/april/tree/master/aprepl

Let me know if the installation instructions there make sense. Once it's installed, you can enter M-x aprepl to enter the REPL. This is still alpha software, hence it's not mentioned in the main README.

April's core lexical functions and operators are almost exactly the same as Dyalog's so you can use Dyalog's user manual as a reference for April. This section of the README summarizes the main differences between Dyalog and April. April notably lacks Dyalog's control statements like :If but it includes k-style if statements as a more flexible alternative to guards.

I agree with you about the learning curve of Emacs and I have some ideas about how to create a better CL end-user experience. That's a whole different discussion but these videos of mine (https://vimeo.com/237947324) (https://vimeo.com/269495385) may provide some food for thought.

@secwang
Copy link
Author

secwang commented Jan 17, 2023 via email

@justin2004
Copy link
Contributor

@secwang I use April from a REPL in vim. Let me know if you'd like the details on how I do that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants