- Improve the reader, add tests
- Make commands more declarative (see pattern.lisp)
- Improve the documentation
- Implement better refactor commands
- Improve breeze-eval function, add tests
- Reduce breeze’s dependencies
- Add support for other editors
**Regularly review the tasks** it helps to see the big picture and prioritize accordingly.
I’ve been using slime only for a long time… Today I tried sly and it crashed when I tried to use quickfix.
Here’s the problem:
(EVAL (SLYNK:EVAL-AND-GRAB-OUTPUT "(breeze.command:continue-command 1 (1 bit, #x1, #o1, #b1))"))
Which comes from breeze.el’s breeze-interactive-eval function,
which calls slynk:interactive-eval.
With slime:
(breeze-interactive-eval "1")
;; => "1"With sly:
(breeze-interactive-eval "1")
;; => "1 (1 bit, #x1, #o1, #b1)"- OS version
- lisp version
- editor version
- quicklisp
- client version
- distributions
- for each dependency
- from which distribution the system come from
- version of the system
Gavinok tried breeze, but ran C-. on breeze.el, this causes
confusion for everybody.
Here’s some ideas to help with this:
- check the extension of the buffer or file
- check the mode of the buffer
- make sure it’s not in the repl (for now)
Instead of having 2 definitions
https://github.com/phoe/with-branching
e.g. “There is no applicable quickfixes in the current context”
e.g. If the common-lisp side ask the editor to insert a string, but the user changed buffer, the editor will insert the string in the wrong buffer.
OR the “client” should be responsible to always execute the commands with on the “initial” buffer, unless stated otherwise (but that would add even more state on the protocol).
I introduced the command backward-char as a hacky workaround: the
integration with the editor edits the buffer incrementally, but then
some other editor configuration would interfere with that process (for
example aggressive-indent-mode in emacs.
I would prefer to improve the whole integration with the editor than using “backward-chard” as a crutch.
Something like relative-move would be better.
e.g. you made a “capture” with breeze, and now you want to turn that into a project.
To help revision
Don’t re-implement it, just figure out which how to configure the editor to do that.
Instead of a lighter
The lighter could still be used though, it could be a ✓ or χ.
As opposed to explain the principles, goals, non-goals, etc.
Actually explain what breeze does.
Start by documenting how to start using breeze, then automate it.
N.B. Since I remove the test-runner and etc. breeze doesn’t really need to be “started”. But it still needs to be loaded.
grep TODO -h | to_github_url | to_html
e.g not suggest “slot-exists-p” when trying to eval “this-really-doesnt-exists”.
- Maybe look into VP-trees (Vantage-Point trees)
How would that help the user? Adding a restart for each candidate would really pollute the list of restarts. Perhaps we could keep the current restart, and add another one that shows more candidate (restarts).
Maybe we can refine that list of candidate based on other, perhaps heavier criteria?
When you redefine many function (e.g. when reloading a file), it doesn’t run all the tests each time a something is redefined.
Really not a priority, even though generating the documentation locally and committing the result is less than ideal, it works well.
The main problem is that, AFAIK, you can’t host something in github pages without committing it into a repository. Which means that you have to hack a CI pipeline that commit its results back into the repository. This sucks IMO.
An alternative would be to use GitLab pages, which are way more sane as they allow (require, really) that your pages’ content be generated from the CI pipeline (from a job called “pages” to be exact). On the other hand (again), I’ve had really janky load time with GitLab, but that might just have been the Authentication + boatload of js.
https://github.com/bard/emacs-director
makem.sh - Makefile-like script for building and testing Emacs Lisp packages
The goal would be to warn the user “hey, you’re trying to evaluate that function, but the package declared in this file/buffer doesn’t exists”.
Could pass the file to br:next. What if
- it’s a buffer that’s not visiting a file?
- it’s not a buffer in lisp-mode
- the buffer is empty
- the buffer requires some reader-macro
- the buffer is visiting a file, but there are unsaved modifications
When the user just initialized breeze, try to find out if the user has any lisp project(s) already opened, help him work on it.
If not projects are found guide him through breeze-scaffold
We already suggest stuff when there’s, for example, an undefined-function error. We could go one step further and suggest a quickfix in the editor. We should probably suggest that quickfix only when the edit-distance is not too great, or we would get some wild quickfixes.
already has a prototype in emacs lisp
already has a prototype in emacs lisp
A.k.a transpose-forms, but keep the cursor at the start of the form that we just moved.
or betweew test system and the system under test
I consider this task “DONE” because I did try to find discrepancies
between the package breeze.refactor and breeze.test.refactor. I
used the convention that each “command” defined in breeze.refator
should have a test with the same name (i.e. the same symbol-name). I
have a test that fails if this “invariant” is not held.
In the future, I would like to
Not everyone is going to have the same conventions.
E.g package a has an exported symbol s, it’s corresponding test
package is a.test.
The current implementation would try to find a test named s in
a.test (for example a.test::s, or ~”s”~ (test names can be string
in parachute), it would be nice to have it also considers tests that
have the prefix s.
Why? Because I have some automatically generated test (a bit like snapshot tests), it’s very convenient that they have the same name as the thing they are testing. Using a prefix would let me have multiple kind of tests for each (automatically generated or not).
Do I want to check if each “types” of tests are implemented?
Can parachute’s `deftest`’s be easily augmented with some metadata? That might help too.
See @phoe’s phoe/protest.
In Slime
In 40ants/ci
> I was sure I already had a note about these…
Holes, in programming, are something used to tell the language that
a part of the program is incomplete. Some languages like Idris and
Agda natively support typed holes. The way I see it, holes are used
to falicitate the conversation between the programmer and the
compiler.
But, for languages like common lisp that doesn’t support holes
out-of-the box, how could we do that? In general, there are no symbol
name that will never clash with other symbols, because symbols in
common lisp can be any string. One idea is to use inline comments,
like #| hole-name |#. Breeze’s parser would be able to recognize
them and manipulate them.
But what for?
Holes can be used to both tell the user what he is expected to enter in a snippet and tell the editor where the user is expected to enter stuff.
A user could use a hole to tell the editor to infer the type of an expression or function and replace the hole by the appropriate declaration.
A user could use a hole to tell the editor to find the right expression where the hole is. This probably requires that the user specify some more constraints, by giving types, writing tests, etc.
| Tag name | Tag description |
|---|---|
| 3rd_parties | Relating to a third-party, e.g. an external library. |
| obsolete | This task is now obsolete. |
| doc | Relating to the documentation. |
| ux | This task is about improving the user experience |
| test | This task is about testing |
| ops | This task is about CI, releases, deploying docs, etc. |
| easy | This task should be easy |
| bug | This is an unintended bug |
| editor | This task relates to the integration with an editor. |
| config | Relating to breeze’s configuration and setup. |
| refactor | Relating to breeze’s refactoring facilities. |
| capture | Relating to breeze’s capture feature. |
| quickproject | Relating to quickproject integration. |
| tech_debt | Due to an ongoing refactoring, to an old hack, incomplete implementation, missing tests, etc. |
| reader | Relating to breeze.reader. |
| noexport | org-mode internal tag |
;; (prin1-to-string (org-get-buffer-tags))
;; (prin1-to-string tags)
;; Find tags that have no descriptions
(let ((unknown-tags
(cl-set-difference
(mapcar #'car (org-get-buffer-tags))
(mapcar #'car tags)
:test #'string=)))
(or unknown-tags
"All good, no tags without description found."))(let ((result))
(org-map-entries (lambda ()
;; (org-entry-is-todo-p)
(cl-destructuring-bind (level reduced-level todo priority headline _tags)
(org-heading-components)
;; _tags does not contain the inherited tags
(when (and
todo
;; todo could be "DONE" for example
(string= todo "TODO")
(not (org-get-tags)))
(push (list headline) result)))))
(nreverse result))As much as possible, so that breeze can easily be ported to different platforms and editors.
Decision: Create wrapper macros (e.g. br:defun) to keep the original
forms for later analysis.
This decision is really not definitive.
This decision is less than ideal, especially for existing systems, but it was the easiest to start with.
Advising swank’s eval function is “a good start” in that direction.
- Might be hard, but eclector could make this easy.
- hyotang666/read-as-string is another candidate
The test framework and the “wrap definition” parts always were proof-of-concepts: I wanted to be able to define some tests, and run them when either the test of the system-under-test was redefined. It worked, but now that I have a more and more complete common lisp parser, I can do the things properly. So I’ve move the concerned code into the folder “scratch-files” and I’ll re-introduce them slowly in the future. (Because I really want something to run the tests in the background, for example.)
I did some tests and the code was like 100x faster when reading from string instead of reading from streams. There are multiple reasons: to extract the “raw” text from the stream require consing new strings and abusing file-position to move back and forth in the stream, both of these are very inefficient. Instead, we use displaced arrays which results in way less consing and no “stream state” to manage. This made both the code faster and simpler.
From another point of view: why not? we were already copying the whole stream into the resulting tree, now we just have references to one string.
This is a very tiny decision, but I know I’ll forget it.
What made me decide between the two: licence is what asdf use, and
it’s what the user will see in their project.
This project is not in quicklisp, and I don’t plan to add it to quicklisp until it stabilize (which might take years). But I make sure that I only use dependencies from quicklisp so that if somebody wants to try it out they’ll just need to clone this repository in quicklisp’s local-projects folder.
- https://github.com/sharplispers/log4cl/blob/master/log4cl.log4slime.asd
- https://github.com/sharplispers/log4cl/blob/master/log4cl.log4sly.asd
- https://github.com/sharplispers/log4cl/tree/master/elisp
- https://github.com/cbaggers/cepl/blob/master/docs/single-thread-swank.md
- https://github.com/cbaggers/livesupport
- https://github.com/cbaggers/swank.live
https://github.com/archimag/cl-routes/blob/master/src/routes-swank.lisp
https://www.reddit.com/r/lisp/comments/1iatcd/fswatcher_watches_filesystem_changes/
http://eradman.com/entrproject/
https://github.com/Ralt/fs-watcher (polls)
https://github.com/Shinmera/file-notify <===
2023-09-25 I briefly talked with Shinmera this summer, and they mentioned that this project doesn’t currently work.
(defun print-x (x) “print (* x x)” (print (* x x))
See function-fingerprinting.lisp
- The more we know the types of the expression, the more we can narrow down the search.
- It would be easier if we knew which expression are safe to execute
https://github.com/webyrd/Barliman
- The more we know the types of the expression, the more we can narrow down the search.
- It would be easier if we knew which expression are safe to execute
- The linter can help choose better results
- Using e-graph to refactor candidates can help suggest helper functions
Which is one of the reason behind wrapping the definitions (e.g. breeze:defun)
- fault injection
- program (dynamic) slicing
- Stepping though code
- profiling
- test coverage
- coverage guided
- profile-guided optimization
It would be nice to be able to search for something (e.g. calls to make-instance) only in a certain slice (e.g. from the “call tree” of foo).
If we have multiple tests on the same piece of code, we can use the slices from the tests that pass and the tests that fail to narrow down which slice is probably the source of the failure.
Main resource: E-Graphs Good
This might be hard and complicated, I was thinking that I should start by making this work on a very small scope. For example, if the user ask to suggest some refactors, we can look for forms that contains only arithmetic (again, just an example) and nothing else, that use equality saturation to find interesting equivalent forms and propose them to the user.
The important bit:
egg is great for algebraic rewrites, but doesn’t have good builtin tools for associative/commutative operations, nor for alpha-equivalence and rewriting under binders. I deliberately left names out of my syntax so that rewriting would be easier; this won’t be as simple for Lisps in general.
A while ago I started by writing a disjoint sets data structure (also known as union-find, based on the 2 mains operations it supports).
https://github.com/fstamour/disjoint-sets
uses locative: http://quickdocs.org/mgl-pax/
2022-03-17 - I read most of the readme, this system looks awesome
It’s mostly for documentation, but it also expand slime/swank for easier navigation (using the concept of locative).
> Clozure CL provides a way for lisp objects to be watched so that a condition will be signaled when a thread attempts to write to the watched object
Very useful for debugging.
> The advise macro can be thought of as a more general version of trace.
I think I kept this link just for the general interface (advise,
unadvise and advisep)
https://github.com/melisgl/mgl-pax for more emacs/slime integration
Superior Lisp Interactive Mode for Atom
https://github.com/neil-lindquist/SLIMA
- https://github.com/cxxxr/cl-lsp
- related: https://marketplace.visualstudio.com/items?itemName=ailisp.commonlisp-vscode
- https://egraphs-good.github.io/
- https://colab.research.google.com/drive/1tNOQijJqe5tw-Pk9iqd6HHb2abC5aRid?usp=sharing
- https://arxiv.org/pdf/2004.03082.pdf
https://gist.github.com/fstamour/2d7569beaf42c0a0883dc0ae559c6638
PROTEST is a tool for defining protocols and test cases written in and for Common Lisp.
https://github.com/s-expressionists/Concrete-Syntax-Tree This library is intended to solve the problem of source tracking for Common Lisp code.
A fresh implementation of Common Lisp https://github.com/robert-strandh/SICL
I’m sure there are tons of other user-case:
- infer types
- interpret code (symbolically or not)
http://web.media.mit.edu/%7Elieber/Lieberary/Tinker/Tinker/Tinker.html
[Image based development](https://www.informatimago.com/develop/lisp/com/informatimago/small-cl-pgms/ibcl/index.html)
https://comby.dev/ (and https://github.com/s-kostyaev/comby.el) https://github.com/reviewdog/reviewdog
https://github.com/hyotang666/trivial-formatter https://github.com/yitzchak/cl-indentify https://github.com/vindarel/colisper (uses comby)
- its catalog of rewrites: https://github.com/vindarel/colisper/tree/master/src/catalog/lisp
https://github.com/cxxxr/sblint https://github.com/g000001/lisp-critic/ https://github.com/eschulte/lisp-format
https://github.com/facebookarchive/codemod replaced by https://github.com/facebook/jscodeshift, which uses https://github.com/benjamn/recast
Examples: https://github.com/cpojer/js-codemod
https://github.com/whitequark/parser https://github.com/seattlerb/ruby_parser https://github.com/seattlerb/ruby2ruby/ https://docs.rubocop.org/rubocop-ast/node_pattern_compiler.html https://nodepattern.herokuapp.com/ https://github.com/mbj/unparser
Probably Rosely for C# and clang for C/C++. I’m sure there are tons of tools/libraries for Java.
For python, there’s the ast module, but I don’t know if it can preserve the formatting. There’s a bunch of tools to format the code.
- https://github.com/Zulu-Inuoe/clution
- https://github.com/Zulu-Inuoe/clution.lib
- https://github.com/Zulu-Inuoe/lob
https://40ants.com/lisp-project-of-the-day/2020/08/0166-breeze.html
https://old.reddit.com/r/Common_Lisp/comments/pgtfm3/looking_for_feedbackhelp_on_a_project/
> testing features along with workers and a file watcher? Shouldn’t they be different projects?
What annoys you when developing in lisp?
I find that setting up a test framework is more difficult than it should be, so any effort on this area is appreciated. I mean: starting with 5am is ok (but could be easier with an editor command), running it from the CLI/a CI is less OK, getting the correct return code of the tests needs more work, etc.
- Chrome DevTools Protocol
- Slime/Sly
- LSP (Language Server Protocol)
- LSIF (Language Server Index Format)
- Debug Adapter Protocol
https://quickdocs.org/cl-scripting https://quickdocs.org/repl-utilities slime issue #532: Rename package and all the symbol prefixes https://blog.cddr.org/posts/2021-11-23-on-new-ides/ https://common-lisp.net/project/slime/doc/html/Contributed-Packages.html
https://quickdocs.org/external-symbol-not-found https://github.com/Bike/compiler-macro https://quickdocs.org/dotenv
https://quickdocs.org/slite - SLIME based Test-runner for FiveAM tests (and possibly others in the future)
In SLIME’s debugger, press v to navigate to its definition.
https://github.com/melisgl/journal - for logging and trace-based testing https://github.com/melisgl/try/ - for a test framework that looks a lot with what I want from a test framework.
For a pretty nice review of existing testing framework: https://sabracrolleton.github.io/testing-framework
https://github.com/ruricolist/moira - Monitor and restart background threads.
https://github.com/pokepay/cl-string-generator - Generate string from regular expression
Emacs supports (declare (pure t) (side-effect-free t))
https://github.com/mmontone/duologue - High level user interaction library for Common Lis
Discussion about #:
https://discord.com/channels/297478281278652417/569524818991644692/915330555334234192
https://stackoverflow.com/questions/869529/difference-between-set-setq-and-setf-in-common-lisp
They don’t actually ask that, they usually just don’t think or know about it.
Here’s something that does an OK job at explaining the importance: https://technotales.wordpress.com/2007/10/03/like-slime-for-vim/
- variables/symbol macros
- functions/macros
- classes/conditions/types
- method combinations
- block names
- catch tags
- tagbody tags
- restarts
- packages
- compiler macros
- slot names
- compiler macros
http://www.lispworks.com/documentation/lw50/LWUG/html/lwuser-90.htm
https://flownet.com/ron/packages.pdf
- Common Lisp Quick Reference
- Ultraspec (dead)
- Simplified something something
- The lisp cookbook
- Don’t forget trailing backslashes for directories.
- Use the functions that operate on sequences.
- Use libraries, like alexandria, split-sequences, serapeum, etc.
- More often called “lisp repl”.
- I use this term to try to avoid confusion with an hypothetical future actual REPL.
- You could describe that as a “client-server REPL”.
- Stands for “Read-Eval-Print-Loop”
- Most people think about “command line” when they hear REPL, but in the case of lisp, it usually means a “listener”.
- [alexandria](https://alexandria.common-lisp.dev/draft/alexandria.html)
- [log4cl](https://github.com/7max/log4cl)