[Support]: How to rebuild a package at init if source is newer than compiled

[meedstrom]

Confirmation

• I have checked the documentation (README, Wiki, docstrings, etc)
• I am checking these without reading them.
• I have searched previous issues to see if my question is a duplicate.

Elpaca Version

Elpaca a65fcc5 HEAD -> master, origin/master, origin/HEAD
installer: 0.7
emacs-version: GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.42, cairo version 1.18.0)
of 2024-06-30, modified by Debian
git --version: git version 2.45.2

Operating System

Debian sid

Description

I decided I'd like to avoid using M-x elpaca-rebuild all the time on the packages I develop. Better if my init process will do it automatically.

I took a stab at this, but I don't quite understand how to design build steps to take this into account. Here's my attempt:

(defun +elpaca/build-if-new (e)
  (let ((compiled-main (file-name-concat
                        (elpaca<-build-dir e)
                        (concat (file-name-base (elpaca<-main e))
                                ".elc"))))
    (if (file-newer-than-file-p (elpaca<-main e) compiled-main)
        (elpaca--build-steps (elpaca<-recipe e))
      elpaca--pre-built-steps)))

(use-package org-node
  :ensure (:build (+elpaca/build-if-new))
  :after org)

I got this backtrace:

Debugger entered--Lisp error: (wrong-type-argument stringp nil)
  file-name-base(nil)
  (concat (file-name-base (progn (progn (nth 8 e)))) ".elc")
  (file-name-concat (progn (progn (nth 6 e))) (concat (file-name-base (progn (progn (nth 8 e)))) ".elc"))
  (let ((compiled (file-name-concat (progn (progn (nth 6 e))) (concat (file-name-base (progn (progn ...))) ".elc")))) (if (file-newer-than-file-p (progn (progn (nth 8 e))) compiled) (elpaca--build-steps (progn (progn (nth 11 e)))) elpaca--pre-built-steps))
  +elpaca/build-if-new((elpaca org-node "org-node" (org-node :build (+elpaca/build-if-new)) (queued) "/home/me/.config/emacs/elpaca/repos/org-node/" "/home/me/.config/emacs/elpaca/builds29.4/org-node" nil nil nil nil (:package "org-node" :fetcher github :repo "meedstrom/org-node" :files ("*.el" "*.el.in" "dir" "*.info" "*.texi" "*.texinfo" "doc/dir" "doc/*.info" "doc/*.texi" "doc/*.texinfo" "lisp/*.el" "docs/dir" "docs/*.info" "docs/*.texi" "docs/*.texinfo" (:exclude ".dir-locals.el" "test.el" "tests.el" "*-test.el" "*-tests.el" "LICENSE" "README*" "*-pkg.el")) :source "MELPA" :protocol https :inherit t :depth treeless :build (+elpaca/build-if-new)) nil nil nil nil 3 (26366 35685 603907 107000) init nil ((queued (26366 35686 81696 661000) "Continued by: elpaca--process" 2) (queued (26366 35685 603914 562000) "Package queued" 1)) t))
  elpaca--continue-build((elpaca org-node "org-node" (org-node :build (+elpaca/build-if-new)) (queued) "/home/me/.config/emacs/elpaca/repos/org-node/" "/home/me/.config/emacs/elpaca/builds29.4/org-node" nil nil nil nil (:package "org-node" :fetcher github :repo "meedstrom/org-node" :files ("*.el" "*.el.in" "dir" "*.info" "*.texi" "*.texinfo" "doc/dir" "doc/*.info" "doc/*.texi" "doc/*.texinfo" "lisp/*.el" "docs/dir" "docs/*.info" "docs/*.texi" "docs/*.texinfo" (:exclude ".dir-locals.el" "test.el" "tests.el" "*-test.el" "*-tests.el" "LICENSE" "README*" "*-pkg.el")) :source "MELPA" :protocol https :inherit t :depth treeless :build (+elpaca/build-if-new)) nil nil nil nil 3 (26366 35685 603907 107000) init nil ((queued (26366 35686 81696 661000) "Continued by: elpaca--process" 2) (queued (26366 35685 603914 562000) "Package queued" 1)) t))

Now I'm considering an alternative trick of using a :pre-build command to simply delete the built directory if it's older. Maybe that would be easier.

[progfolio]

Try using elpaca--main-file in your function instead of accessing the elpaca<-main slot directly.
It will compute the file name if necessary, or use the elpaca<-main cache slot once computed. It may have an impact on startup time if used globally, but shouldn't be too bad if you limit it to packages you're actively developing as in your example.

You'll also want to alter the build steps in the case where you want a rebuild done.
elpaca-build-steps by default will include things like cloning the repository which will fail since the repository is already on disk. See the source of elpaca-rebuild for how I remove build steps in that case.

Once you get the function right, you could experiment with adding your own recipe syntax if you like, too. e.g. use elpaca-order-functions to translate something like :build auto to the syntax in your example declaration. Should be straightforward.

Now I'm considering an alternative trick of using a :pre-build command to simply delete the built directory if it's older. Maybe that would be easier.

That could work, but you'd want to explicitly set the build steps similar to how you are in your function.
The build steps have already been determined by the time :pre-build is run.
I plan to eventually remove :pre-build and :post-build, since they're both special cases of :build.
I need to come up with a better DSL or utilities for manipulating :build steps in general first, though.

[meedstrom]

Thanks for info! Now I'm at this version:

(defun +elpaca/build-if-new (e)
  (let* ((main (elpaca--main-file e))
         (compiled (file-name-concat (elpaca<-build-dir e)
                                     (concat (file-name-base main) ".elc"))))
    (if (file-newer-than-file-p main compiled)
        (setf (elpaca<-build-steps e) elpaca-build-steps)
      (setf (elpaca<-build-steps e) elpaca--pre-built-steps))))

however, it does not seem to re-build when the source is newer.

Still get the message:

Source file ‘/home/me/.config/emacs/elpaca/builds29.4/org-node/org-node.el’ newer than byte-compiled file; using older file

[progfolio]

Try something like this:

(elpaca-test
  :interactive t
  :early-init (setq elpaca-menu-functions nil)
  :init
  (defun +elpaca/build-if-new (e)
    (setf (elpaca<-build-steps e)
          (if-let ((default-directory (elpaca<-build-dir e))
                   (main (ignore-errors (elpaca--main-file e)))
                   (compiled (expand-file-name (concat (file-name-base main) ".elc")))
                   ((file-newer-than-file-p main compiled)))
              (progn (elpaca--signal e "Rebuilding due to source changes")
                     (cl-set-difference elpaca-build-steps
                                        '(elpaca--clone elpaca--configure-remotes elpaca--checkout-ref)))
            (elpaca--build-steps nil (file-exists-p (elpaca<-build-dir e))
                                 (file-exists-p (elpaca<-repo-dir e)))))
    (elpaca--continue-build e))
  (elpaca (doct :host github :repo "progfolio/doct" :build (+elpaca/build-if-new))))

Upon first run, that should install doct.
M-x restart-emacs followed by M-x elpaca-info doct should not show a rebuild.
Then edit doct.el and add something like:

;;;###autoload
(defun doct-testing () (interactive) (message "PASS"))

After an M-x restart-emacs you should see the rebuild in Elpaca's log for doct.
M-x doct-testing should message "PASS".
A third M-x restart-emacs should not rebuild doct.

[meedstrom]

Confirmed, that function works. Thanks! Would you like elpaca to have something like :build auto? Or you want to come up with a better DSL first?

[progfolio]

Confirmed, that function works. Thanks!

You're welcome.

Would you like elpaca to have something like :build auto? Or you want to come up with a better DSL first?

Let's wait on that for now.
Users who are interested can add a similar customization to above.
I'd rather get the design right first.