Elpakit is a bunch of tools for authoring Emacs ELPA packages.
- Elpakit makes package-archives for you
- Elpakit builds multi-file packages or single file packages
- Elpakit runs tests on packages in isolated Emacs processes
- Elpakit runs occur on package symbols for you
An example:
(elpakit
"/tmp/shoesoffsaas" ;; the directory to make the archive in
'("~/work/elnode-auth" ;; the list of package directories
"~/work/emacs-db"
"~/work/shoes-off"
"~/work/rcirc-ssh"))Elpakit will make a package archive in "/tmp/shoesoffsaas". It will have an archive-contents formatted correctly. It will have packages made from the listed directories but also any packages that those packages depend on will be downloaded from the rest of your package archives.
This is very early code and it's difficult to test a lot of it. It mostly works. You may find bugs. If you do please tell me.
To use elpakit your development tree must conform to a particular standard.
Packages are directories with either:
- a single elisp file in them which is the package file; a single-file package is constructed from these
- a single elisp file, which is the package file, and some form of README file; a single-file package is constructed from these. Essentially the README is ignored.
- multiple elisp files where all but one elisp file is of the form: test*.el or *test.el. The package is built from the one elisp file remaining. A test package may be built from the test files.
- a recipes directory containing a file named for the package (eg: "elnode", it can also be different from the dev dir, eg: "emacs-db" might have a "recipes/db" recipe file). The recipes file broadly follows the MELPA conventions. It mostly specifies the :files in the dev directory which belong to the package.
- There must be only one elisp file in the recipe for a recipe dev directory to be considered a single-file package.
- a recipes directory, just as above, but containing many elisp files; these are considered tar packages and are built as such.
The recipe format is based on the MELPA format but extended to include the extra meta-data required by tar packages.
The keys that are present in a recipe file are:
- name - the name of the package
- version - the string version of the package
- doc - a doc string
- requires - the standard package requires sexp
- files - a list of files that will be embedded in the package.
- test - how to test the package
The files component may include directories and the directories are then included in the package tarball.
The test is a sort of recursive recipe, more information below.
Here are some examples.
db is a single file package, but the repository directory includes a tests file and the README:
(db
:files ("db.el"))this is the simplest form of recipe. It should not be needed very often because there is enough intelligence in elpakit about what is in a repository directory to infer this recipe.
elpakit can help with testing if you tell it about your test files and dependancies, here's the recipe for web, an HTTP client:
(web
:files
("web.el")
:test
(:requires
((fakir "0.0.10")
(elnode "0.9"))
:files
("web-test.el")))This is still a single file recipe. The test file and it's requires are not needed for running the package, only testing it. Therefore they do not need to be packaged.
elnode is a tar package and has quite a lot of complexity:
(elnode
:version "0.9.9.6.1"
:doc "The Emacs webserver."
:requires
((web "0.1.4") ;; for rle
(creole "0.8.14") ;; for wiki
(db "0.0.1")
(kv "0.0.9"))
:files
("elnode.el"
"elnode-rle.el"
"elnode-wiki.el"
"default-wiki-index.creole"
"default-webserver-test.html"
"default-webserver-image.png"
"README.creole"
"COPYING")
:test
(:requires
((fakir "0.0.14"))
:files
("elnode-tests.el")))Elnode is like this because it needs to deliver the default wiki pages, these must be packaged so they can be installed at package deploy time.
elmarmalade is an example of packaging directories:
(marmalade-service
:version "2.0.9"
:doc "The Marmalade package store service."
:requires
((dash "1.1.0")
(s "1.6.0")
(s-buffer "0.0.4")
(elnode "0.9.9.6.11")
(htmlize "1.3.9"))
:files
("marmalade-archive.el"
"marmalade-service.el"
"marmalade-customs.el"
"marmalade-boot.el"
"front-page.html"
"login-page.html"
"upload-page.html"
"static/style.css") ;; the directory 'static' will be included
:test
(:requires
((fakir "0.1.1")
(s "1.4.0"))
:files
("marmalade-tests.el"
"elnode-0.9.9.6.9.tar")))Test packages can be defined in the recipe or automatically inferred through the presence of files matching the pattern test.*.el or .*test.el.
Test packages are always named package-name-test.
A useful feature of elpakit is being able to build the kit and then run tests defined for a package (note: only one of the packages in the kit right now).
You can do that like this:
(elpakit-test
'("~/work/emacs-db"
"~/work/emacs-db-pg"
"~/work/pg"
"~/work/emacs-kv")
'db-pg-tests
'db-pg)The output is stored in the buffer *elpakit*.
Elpakit constructs a temporary archive for the kit and then builds the packages to it and then runs an Emacs instance with your local package-archives plus the temporary one and then installs the package you've specified and then runs the tests you specified with an ERT selector.
elpakit-test can be run interactively. When used like that it uses the current directory as a guess about a package dir. This means you can only package and test one package right now. Any dependancies must be present in the repositories declared in your local package-archives variable.
You can also start an elpakit in a daemonized Emacs and optionally run tests.
(elpakit-start-server some-elpakit to-install)
will start a server, open the *elpakit-daemon* buffer and setup the to-install there.
To run tests automatically, add the test argument, for example:
(elpakit-start-server elnode-elpakit 'elnode-tests "elnode")
will install the elnode-tests package and then run tests for elnode.* in a server Emacs.
You can kill that server Emacs from the command line or from the *elpakit-daemon* buffer like this:
M-x elpakit-stop-server
Elpakit includes a management tool for processes it starts. Use:
M-x elpakit-list-processes
to make a process list of elpakit started processes. You can use this to show the log of an elpakit process (key L) or to kill a running elpakit process (key K) even if it's a daemon.
Keys available:
| F | elpakit-process-open-emacsd | Open the emacsd directory in dired |
| O | elpakit-process-open-emacsd | |
| f | elpakit-process-open-emacs-init | Open the init.el file |
| K | elpakit-process-kill | Kill the process or daemon |
| k | elpakit-process-kill | |
| L | elpakit-process-show-buffer | Show the process buffer |
Elpakit includes some support for refactoring. When you are dealing with projects with lots of files (which Elpakit makes quite easy) it's common to need to refactor.
Elpakit includes elpakit-multi-occur which can show you every reference to a symbol within your Elpakit project.
elpakit-isearch-hook-jack-in can be added to emacs-lisp-mode-hook to connect elpakit-multi-occur to isearch.
It might also be advisable to connect elpakit-multi-occur to a keybinding in the normal emacs-lisp-mode. This is left up to you but I suggest:
(define-key emacs-lisp-mode-map (kbd "M-o") 'elpakit-multi-occur)
I once hoped to merge elpakit with MELPA and Cask but it doesn't seem possible. So I build elpakit purely for my own ends these days.
The MELPA recipe standard has no:
- version
- doc
- requires
- or test sections
as far as I can see. These have been added to elpakit to make the task of building tar files possible without having to build the -pkg.el file.
MELPA doesn't even need the version since it pulls it from the GIT commit?
A side aim for elpakit is that it makes defining a package partly a function of the package's source control. Just like it normally is with other package systems (dpkg, rpm, pip, etc...).
MELPA specifies the recipes elsewhere than the repository source control. It's obvious why this has happened, they needed to support people MELPA packaging repositories they did not have commit access to.
I don't know how to solve this problem, maybe the MELPA guys could switch their code to autodetect repository provided recipes somehow?
Elpakit covers pretty much the same ground as Cask. The Cask program uses an external binary though while elpakit is just elisp. Elpakit also uses file based package archives quite a lot, while Cask seems to prefer HTTP. This requires an HTTP server which I don't think is a necessary step at all.
There are a bunch of useful things you can do if you're building apps that are collections of elpa packages.
elpakit has all the logic to build multi-file packages so it made sense to make that available with a command:
M-x elpakit-make-multi
will use the directory you are in as a package directory (if it can be established that it is a package) or ask you to specify a package directory. It will then build that package for you in a temorary directory. It will open the temporary directory with the built package in it. You can use that to upload to Marmalade or some such.
MELPA requires multi-file packages repositories to be packaged in a particular way. Namely that they should have the pkg.el file present in the repository.
elpakit will automatically build an orphan branch for you in your repository for use by MELPA.
Just set the customization variable elpakit-do-melpa-on-multi-file-package to true and then use elpakit-make-multi on a git repository based multi-file package and elpakit will make the branch for you. Elpakit won't push the branch so you get to review stuff before it goes back to your origin.
You can build an elpakit entirely inside your emacs. This basically means just evaling all the lisp it finds.
(elpakit-eval
'("~/work/elnode-auth"
"~/work/emacs-db"
"~/work/shoes-off"
"~/work/rcirc-ssh"))Sometimes you have to muck about with the ordering of the kit to make it work because there are dependancies (requires) that don't work unless you eval things in a particular order.
Elpakit doesn't care what the destination is as long as it looks like a directory to Emacs. That means you can use TRAMP:
(elpakit "/ssh:my-remote-host.example.com/myapp-elpa/"
'("~/work/elnode-auth"
"~/work/emacs-db"
"~/work/shoes-off"))This is useful for deploying packages to remote locations, for example "live" in an elnode app.
An alternative to the push to remote elpakit is building it locally and having the remote pull it. This is possible too, especially with a bit of elnode magic.
(elpakit "/tmp/my-archive"
'("~/work/elnode-auth"
"~/work/emacs-db"
"~/work/shoes-off"))
(elnode-make-webserver "/tmp/my-archive" :port 8007 :host "0.0.0.0")and on the remote use:
wget -r -np http://elnode-server:8007
to get the package archive built by elpakit.
Just collecting the list of package directories into a list means you can do lots of different things with elpakit:
(defconst shoes-off-elpakit
'("~/work/shoes-off"
"~/work/rcirc-ssh"
"~/work/emacs-db"
"~/work/esxml"
"~/work/elnode-auth"
"~/work/emacs-kv"
"~/work/shoes-off-aas/talkapp"))Then you can:
(elpakit "~/my-app-elpa" shoes-off-elpakit)
If you use a defconst then you can re-eval it more easily.
If you want to deploy to a remote host without pushing to an official repository you can still do that, just use tramp for the destination:
(elpakit "/ssh:nic@livehost.example.com:apps/app1/app-elpa" my-elpakit)
Presuming my-elpakit is an elpakit list. Elpakit uses a straight copy so tramp works fine.
Starting a server and running some tests in it:
(setq nic-server
(elpakit-start-server
shoes-off-train 'talkapp :test t))Server's can be started with extra lisp to initialize them, in this case they do not auto-require the install target.
So here's a quick snippet to start a server with an archive, copy some files in to the package location, then require the main thing:
(setq nic-server
(elpakit-start-server
shoes-off-train
'talkapp
:extra-lisp
'(progn
(shell-command
(format
"cp -r ~/work/teamchat/talkapp/*db*.elc %s"
(file-name-directory
(find-lisp-object-file-name
'talkapp-start
(symbol-function 'talkapp-start)))))
(require (quote talkapp)))))Note the clever hack to find the package location based on an autoload specified function.
Here's a more clever version of that wrapped up in an interactive, defun, very useful startup routine:
(defvar talkapp-elpakit-server nil)
;; Start the talkapp in an elpakit server
(defun talkapp-do-elpakit-server ()
(interactive)
(condition-case err
(progn
(server-eval-at (cdr talkapp-elpakit-server) '1)
(error
"talkapp-elpakit-server already running %s"
(cdr talkapp-elpakit-server)))
(error (if (string-match "^No such server" (cadr err))
(setq talkapp-elpakit-server
(elpakit-start-server
shoes-off-train
'talkapp
:extra-lisp
;; Copy in the db files from the working dir...
'(progn
(shell-command
(format
"cp -r ~/work/teamchat/talkapp/*db*.elc %s"
(file-name-directory
(find-lisp-object-file-name
'talkapp-start
(symbol-function 'talkapp-start)))))
(require (quote talkapp)))))
;; Else rethrow
(signal (car err) (cdr err))))))It only allows one at a time atm but that's still better than trying to do everything inside a single emacs.
Elpakit will also let you make a package from your currently installed ELPA packages. This is useful if you want to share a lot of dependencies with friends or colleagues.
M-x elpakit-package-list-buf
Makes a buffer with a list of your packages in it.
In the ELPA package list buffer:
| Key | What it does |
| k | kill the current package |
| M | make a new package file from the remaining package names |
The package file that it makes can then be checked into Git (to be made available with MELPA) or uploaded to marmalade-repo.
A package is either a single EmacsLisp file with a special header or a collection of EmacsLisp and possibly other files (like an info file or HTML files) in a tarball. A single file EmacsLisp package is called a single file package and a tar package is called a multi-file package. Elpakit deals with either single or multi file packages and can build a multi-file package for you (a non-trivial process) from a collection of source files.
A package dir is a directory in which the source files for a package are kept. Probably this is a checkout of a version control repository but need not be.
A recipe is a file that tells Elpakit how to put together a package. It is found in a package-dir. For single file packages this is almost never necessary. For multi-file packages it is necessary. The recipe is necessary for multi-file packages because it contains information about the package that has no other home and because Elpakit cannot guess what files in your tree you actually want to be present in your resulting package.
An archive is a place you can install packages from. Archives can either be http URLs or a local directory. Elpakit can make archives from collections of package dirs. To actually install from a particular archive you need to add the archive to the Emacs variable package-archives.
An elpakit is a list of package-dir that are going to be built into an archive. Once the archive is built we can do more with it, such as running tests on the packages we've built (which will be clean and not tainted by any local evaluation you have done in your local Emacs).
A local emacs is the Emacs you are writing code and running Elpakit in, as opposed to an Emacs instance you start to do testing or such.
- include any dir file and info files in multi-file packages
- add a walk through constructor process for recipe files?
- add a mode (other than lisp-mode) for editing recipe files?
- add the URL to the package in elpakit/make-pkg-lisp
- should go into define-package's extra-properties
- add upstream repository list to a recipe
- so that a package could specify where it's dependant repositorys can be found
- this is mainly so a package could be installed from a url and specify that depends come from marmalade (for example)