Interest

[jojojames]

Just commenting and throwing my interest out there. This looks really cool and I'll definitely be trying this soon.

Good luck!

[piotrklibert]

Thank you! I'm getting more and more motivated to actually polish the package and make it useful for others :) I'm currently using it almost exclusively for jumping to files in projects - it already 100% replaced counsel-jump-file for me - but it's still only on the "works for me" level, it will require more work before it can be easily reused by others (I need to at least prepare binary downloads, among other things, so that you don't have to install Nim and compile the module yourself).

If you plan to try it out anyway, please be careful: although I haven't seen it for some time, it's still possible for the module to cause a segfault and a hard crash of the whole Emacs. Also, don't recompile the module while it's loaded into running Emacs: I got a few crashes before I realized this. So, if you change the Nim code, close Emacs before recompiling. I'll update the README with a warning about that later. Also, remember to please let me know about any problems you might have while trying to make it work on your system! I don't have that much experience with native DLLs, so I expect there's a lot to improve in terms of building and configuring the package.

[piotrklibert]

I cleaned up the ffipf.el and updated the README accordingly. Next, I'm going to add customization options on the Elisp side and make sure they're readable on the Nim side.

[piotrklibert]

Couldn't work on this last week due to falling ill a bit, but there's progress today. I rewrote the helpers for calling Emacs API from Nim templates to macros, which allowed me to hide the need for passing around the emacs_env struct everywhere. Further, this will allow me to easily check for signals and throws from the Emacs side, and to convert them into Nim exceptions, hence simplifying error handling on the Nim side significantly. This was also a prerequisite for easily and reliably getting the values of Emacs variables from Nim, but I think I'll finish error/exception-handling first.

[jojojames]

Nice, I had a chance to try it on OSX and building fails (readme says it's linux only but I thought I'd try on OSX).

ffipf/ $ make dist
nim c \
	  -d:nimMaxHeap=120 \
	  --nimcache:nimcache \
	  --app:lib \
	  --out:ffipf_backend.so \
	  --opt:speed \
	  --gc:arc \
	  src/ffipf_backend.nim
Hint: used config file '/usr/local/Cellar/nim/1.4.2/nim/config/nim.cfg' [Conf]
Hint: used config file '/usr/local/Cellar/nim/1.4.2/nim/config/config.nims' [Conf]
...............................CC: ffipf_backend.nim
/Users/james/Code/ffipf/nimcache/@mffipf_backend.nim.c:518:28: error: conflicting types for 'NimMain'
N_LIB_EXPORT N_CDECL(void, NimMain)(void) {
                           ^
/Users/james/Code/ffipf/nimcache/@mffipf_backend.nim.c:187:3: note: previous implicit declaration is here
  NimMain(); // execute module-global statements
  ^
1 error generated.

Error: execution of an external compiler program 'clang -c  -w -ferror-limit=3 -O3  -fPIC  -I/usr/local/Cellar/nim/1.4.2/nim/lib -I/Users/james/Code/ffipf/src -o /Users/james/Code/ffipf/nimcache/@mffipf_backend.nim.c.o /Users/james/Code/ffipf/nimcache/@mffipf_backend.nim.c' failed with exit code: 1


make: *** [ffipf_backend.so] Error 1

[piotrklibert]

Oh, that's interesting. I don't have a macOS available at the moment, but I know Nim should be able to compile dylibs on it (as well as dlls on Windows), so in theory, it should be possible to make it work. I've had a similar problem on Linux once ("previous implicit declaration", etc.), though I don't remember exactly what it was about. I'll look into it and ask around for hints on the Nim forum.

[Trisk3lion]

Indeed really interesting, trying it out now!
Just throwing something in here for anyone like me who is a illiterate in Nim and running Ubuntu; if you install Nim with sudo apt install nim you get version 1.0.6 which doesn't have arc garbage collector which will make the install fail.
Took me some time to understand.

[piotrklibert]

Thanks @Trisk3lion, I'll add a note to the README about requiring Nim 1.4. I'm not sure if we really need it, but I was getting some mysterious crashes while using the default GC which went away after switching to ARC. The performance is also better, and I don't have any cycles in the data, so there are no demerits for using it, other than needing the latest Nim.

As for the status update this time: I just implemented proper interop with Elisp signals and throws (they're translated to Nim exceptions and back). I will need to add a few more helpers for getting and decomposing values from the Emacs side (car, cdr, cons, and Elisp list to Nim seq conversion) to finally be able to set blacklists and other configuration on the Elisp side. As for the "other configuration", I added an extension-to-weight table which allows for fine-tuning how the results are sorted. My use case is that I have ffip_backend.nim and ffip_backend.so, the latter in two places, and when I input "ba" I obviously want the former selected.

It's going a bit slower now that I'm back at work, but there's good progress. I want to focus on moving the config to the Emacs side, then on returning the full MatchResult objects with additional data instead of simple strings. This will allow highlighting the groups of characters that were actually matched, which is one feature common to other Ivy commands the interface is currently lacking. It will also enable further sorting and filtering from Elisp if needed. After that, I want to investigate compilation on non-Linux systems, and possibly set up a CI process for producing binaries for each OS. I wonder if there are any cheap options out there... Then, finally, I'll make a proper Elisp package and push it to MELPA. It shouldn't take more than a month or two to get there, hopefully.

[piotrklibert]

I moved most of the configuration to the Elisp side. I found a nice trick for implementing fromEmacs in a Nimler package, which uses typedesc[...] parameter to overload its definition(s) to support various shapes of the data on the other side. Neat.

Next: a bit of a cleanup, then returning full MatchResults to Emacs. Some tuning for the scoring of links.

Everything works for me, but I did a bit of refactoring to implement this, so the code could be potentially less stable for a while. Should be back to rock-solid after the next round of cleanups.