Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There is no good way to install and upgrade use-package #219

Closed
marcinant opened this issue Jun 4, 2015 · 25 comments
Closed

There is no good way to install and upgrade use-package #219

marcinant opened this issue Jun 4, 2015 · 25 comments

Comments

@marcinant
Copy link

Problem is that use-package is available as ELPA package and this is probably easiest way to install and upgrade.
Unfortunately use-package should be initialized before Emacs initializes ELPA packages.

So, when this macro is installed as ELPA package there is no way to (require 'use-package) because Emacs doesn't know where use-package is.

The only way I see is to add full path to bind-key and use-package as second argument to require.
It works however manual update is required every time ELPA package is upgraded.

Any other ideas to solve this problem?

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

I'm not sure that I follow.. When use-package is installed, package.el will find it and upgrade it after invoking list-packages

I have additionaly set up so that I don't need to package-initialize packages by saving the load path in a byte compiled load-path.el.

This is from the top of my init.el

(when load-file-name
  (load (expand-file-name
         "load-path" (file-name-directory load-file-name)) nil t))

(setq
 package-enable-at-startup nil
 package-archives
 '(("melpa-stable" . "http://stable.melpa.org/packages/")
   ("melpa" . "http://melpa.org/packages/")
   ("marmalade"   . "http://marmalade-repo.org/packages/")
   ("org"         . "http://orgmode.org/elpa/")
   ("gnu"         . "http://elpa.gnu.org/packages/")
   ("sc"   . "http://joseito.republika.pl/sunrise-commander/")))

(eval-when-compile
  (require 'package)
  (package-initialize t))
(unless (boundp 'package-pinned-packages)
  (setq package-pinned-packages ()))

(defun require-package (package &optional min-version no-refresh)
  "Install given PACKAGE, optionally requiring MIN-VERSION.
If NO-REFRESH is non-nil, the available package lists will not be
re-downloaded in order to locate PACKAGE."
  (if (package-installed-p package min-version)
      t
    (if (or (assoc package package-archive-contents) no-refresh)
        (package-install package)
      (progn
        (package-refresh-contents)
        (require-package package min-version t)))))

(eval-when-compile
  (require-package 'use-package)
  (require 'use-package))

This is the load-path.el located besides my init.el

In addition to byte compiling the effecive load-path load-path.el will recompile itself if one of the compiled paths are missing at start up which takes care of the most usual case when packages are upgraded and obsolete ones removed.

(I have not cared a great deal about style in this particular file...)

;;; load-path.el

(defconst user-emacs-directory
  (if (eq system-type 'ms-dos)
      ;; MS-DOS cannot have initial dot.
      "~/_emacs.d/"
    "~/.emacs.d/")
  "Directory beneath which additional per-user Emacs-specific files are placed.
Various programs in Emacs store information in this directory.
Note that this should end with a directory separator.
See also `locate-user-emacs-file'.")

(defconst user-data-directory
  (file-truename "~/.config/emacs-user-data"))
(defconst user-cache-directory
  (file-truename "~/.cache/emacs-user-cache"))
(defconst user-lisp-directory
  (expand-file-name "lisp" user-emacs-directory))

(defun load-path--take (n list)
  "Returns a new list of the first N items in LIST, or all items if there are fewer than N.
This is just a copy of the fully expanded macro from dash."
  (let (result)
    (let
        ((num n)
         (it 0))
      (while
          (< it num)
        (when list
          (setq result
                (cons
                 (car list)
                 result))
          (setq list
                (cdr list)))
        (setq it
              (1+ it))))
    (nreverse result)))


(defconst user-site-lisp-directory
  (expand-file-name "site-lisp/shared" user-emacs-directory))
(defconst user-themes-directory
  (expand-file-name "themes" user-emacs-directory))
(defconst user-notes-directory
  (file-truename "~/notes"))

;; These should always exist
(make-directory user-data-directory t)
(make-directory user-cache-directory t)

;; emacs23 compat
(if (boundp 'custom-theme-load-path)
    (add-to-list 'custom-theme-load-path user-themes-directory)
  (add-to-list 'load-path user-themes-directory))

(defun add-to-load-path (path &optional dir)
  (setq load-path
        (cons (expand-file-name path (or dir user-emacs-directory)) load-path)))

(defun load-path-load-path ()
  (let ((load-path load-path))
    ;; Add top-level lisp directories, in case they were not setup by the
    ;; environment.
    (require 'package)
    (package-initialize)
    (dolist (dir (nreverse
                  (list user-lisp-directory
                        user-site-lisp-directory)))
      (dolist (entry (nreverse (directory-files-and-attributes dir)))
        (and
         (cadr entry)
         (not (string= (car entry) ".."))
         (add-to-load-path (car entry) dir))))


    (mapc #'add-to-load-path
          (nreverse
           (list
            (expand-file-name "~/.config-private/emacs")
            (expand-file-name "~/.opt/extempore/extras")
            (expand-file-name "/usr/local/opt/extempore/extras")
            (concat user-site-lisp-directory "/emms/lisp")
            "/usr/local/share/emacs/site-lisp/"
            "/usr/local/share/emacs/site-lisp/mu4e/"
            "/opt/local/share/emacs/site-lisp/"
            "/usr/share/emacs/site-lisp/SuperCollider/"
            "/usr/share/emacs/site-lisp/supercollider/"
            "/var/lib/gems/1.9.1/gems/trogdoro-el4r-1.0.10/data/emacs/site-lisp/")))

    (let ((cl-p load-path))
      (while cl-p
        (setcar cl-p (file-name-as-directory
                      (expand-file-name (car cl-p))))
        (setq cl-p (cdr cl-p))))

    (when
        (or (not (boundp 'emacs-version))
           (string< emacs-version "24.3"))
      (add-to-load-path
       (expand-file-name "site-lisp/cl-lib" user-emacs-directory)))


    (delete-dups
     (delq nil (mapcar #'(lambda (x)
                         (if (file-directory-p x)
                             x
                           nil))
                     load-path)))))


(defmacro load-path-set-load-path ()
  `(progn
     (setq load-path ',(load-path-load-path))
     (let ((failed nil))
       (mapc #'(lambda (x)
                 (unless failed
                   (setq failed (not (file-directory-p x)))))
             load-path)
       (when failed
         (require 'bytecomp)
         (let ((byte-compile-verbose nil)
               (byte-compile-warnings nil)
               (use-package-verbose nil)
               (ad-redefinition-action 'accept))
           (byte-recompile-file "~/.emacs.d/load-path.el" t 0 t))))))

(load-path-set-load-path)

(eval-after-load "info"
  #'(progn
      (when (fboundp 'info-initialize)
        (info-initialize)
        (defun add-to-info-path (path &optional dir)
          (setq Info-directory-list
                (cons (expand-file-name path (or dir user-emacs-directory)) Info-directory-list)))
        (mapc #'add-to-info-path
              (nreverse
               (list
                (expand-file-name "~/.refdoc/info")))))))

(when (bound-and-true-p x-bitmap-file-path)
  (add-to-list 'x-bitmap-file-path
               (concat user-emacs-directory "/icons")))

(require 'cus-load nil t)

(provide 'load-path)

;;; load-path.el ends here

@xuchunyang
Copy link
Contributor

Unfortunately use-package should be initialized before Emacs initializes ELPA packages.

Why? I think there is no need to initialize use-package, of course you have to install use-package before using it, you can install use-package from ELPA like this way:

;; Setup package.el
(require 'package)
(setq package-enable-at-startup nil)
(add-to-list 'package-archives '("melpa" . "http://melpa.org/packages/"))
(package-initialize)

;; Bootstrap `use-package'
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))

;; Use use-package now

@marcinant
Copy link
Author

Running package-initialize on top of emacs is plain stupid.

You are right. If I add package-initialize before I try to require use-package then it's going to run properly.

However and idea behind package-initialize is to run this command AFTER .emacs or init.el and run all configuration on after-init-hook.

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

@marcinant My solution only requires/uses package.el inside eval-when-compile in init.el or when load-path.el is recompiled..

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

My emacs starts in about 1-1.1seconds on my 2009 i7 desktop computer (provided files are loaded into disk cache)..
Thats with 778 package dirs in elpa/ and a ~10kloc init.el with 518 (use-package forms and 389 uses of :ensure. (some of the use-package instances are probably :disabled but not many)

IIRC, the package-initialize otherwise needed to set up load paths of packages and their dependencies adds 0.3-0.5s to the init time which becomes way more noticable than closer to 1s.

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

I just also remembered my load-path.elonly recompiles itself if it already is byte compiled, It also does some non load path related things and I have not taken care of other variables like the theme path..
So my solarized-theme setup looks like this

(use-package solarized-theme
  :ensure t
  :if window-system
  :init
  (progn
    (setq solarized-use-less-bold t
          solarized-use-more-italic t
          solarized-emphasize-indicators nil
          solarized-distinct-fringe-background nil
          solarized-high-contrast-mode-line nil))
  :config
  (progn
    (load "solarized-theme-autoloads" nil t)
    (setq theme-dark 'solarized-dark
          theme-bright 'solarized-light)))

@npostavs
Copy link
Contributor

npostavs commented Jun 4, 2015

Running package-initialize on top of emacs is plain stupid.

?

Artur Malabarba wrote:

On the "Customizable modes..." thread I suggested we run
(package-initialize) sooner than the way it's currently done. Right
now, it's called after loading the init file. Which means any user who
tries to customize an installed package by pasting some code into his
init file will be confronted with errors.
This happens A LOT.

Stefan kindly explains why it can't just be done before loading the init file:

[...] the user may need/want to run some Elisp
code of his own choosing before package-initialize is called.
E.g. [...] set package-load-list and package-directory-list

But I'm asking that we try a little harder to find a better solution.
This package.el-induced "cannot find load file" error is the most
predominant issue I see people run into in the wild. Some people file
issues for this stuff on github (and waste developer time), [...]

Option 2) Instead of us manually telling users to add
`(package-initialize)' to their init-files, we have Emacs do that
automatically. [...]

@marcinant
Copy link
Author

Running package-initialize on top of emacs is plain stupid.

?

Just because startup.el contains this code:

       (package-initialize))

  (setq after-init-time (current-time))
  (run-hooks 'after-init-hook)

So, package-initialize is part of normal startup procedure and there is no reason to disturb this order and run package-initialize twice.

Artur Malabarba wrote:

On the "Customizable modes..." thread I suggested we run
(package-initialize) sooner than the way it's currently done. Right
now, it's called after loading the init file. Which means any user who
tries to customize an installed package by pasting some code into his
init file will be confronted with errors.
This happens A LOT.

It's their problem. They should use after-init-hook and that's it.

Some people file
issues for this stuff on github (and waste developer time), [...]

Developer should just provide documentation how to configure package on after-init-hook.

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

@marcinant it runs twice unless you set (setq package-enable-at-startup nil)

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

I used strace to guide my decision of removing package-initialize on normal emacs startup.. The combination of emacs synchronized io and lot's of file descriptor calls just makes it slow.

Since use-pakcage :commands generates autoloads there is little package.el does at start up except set the load path.

@marcinant
Copy link
Author

Hmm... is your .emacs available somewhere?

Anyway I decided to use this dirty piece of code:

(when package-enable-at-startup
  (eval-when-compile
    (let ((default-directory package-user-dir))
      (require 'bind-key (car (file-expand-wildcards "bind-key*/bind-key.elc" t)))
      (require 'use-package (car (file-expand-wildcards "use-package*/use-package.elc" t))))))

Now I need to switch my .emacs to use package and test it.
I don't care about load time that much as I use emacsclient and run emacs on startup just once.

@thomasf
Copy link
Contributor

thomasf commented Jun 4, 2015

yeah https://github.com/thomasf/dotfiles-thomasf-emacs I have one emacs per "workspace group" of desktops which more or less means one emacs per project.. I also have a default "home" emacs for all general emacsclient needs. Startup time is more important when working like that.

@marcinant
Copy link
Author

After further code analysis I have to admit that I don't understand the idea behind use-package.

Package.el is intended to load after user init file.
Unfortunately use-package doesn't want to work while packages are not initialized.

IMHO this is just plain wrong. Could someone (an author) enlighten me and tell why is it designed in this way?

@thomasf
Copy link
Contributor

thomasf commented Jun 5, 2015

It's not designed specifically for package.el.. John (the author) doesnt even use package.el. It's more or less up to the use-package user to do the integration part.

@marcinant
Copy link
Author

Ok. Now I can see what the problem was. All my packages are not in load-path until package.el is initialized. Sigh....

@marcinant
Copy link
Author

Well, you may close this bug. However problem is still alive.

Although use-package is available on ELPA it should not be installed as package.

@thomasf
Copy link
Contributor

thomasf commented Jun 10, 2015

I disagree that this is an use-package issue, after all it's elisp were talking about and it's as easy as invoking (package-initialize) when it's needed.. Also the emacs-devel discussion link posted by npostavs in this issue suggests that the default behavior might actually change into doing it before user init by default.
Having said that, the documentation could probably be updated to explain this situation.

@marcinant
Copy link
Author

It's undocumented. It should be mentioned in readme that one has to invoke (package-initialize) or provide load-path manually.

When someone will install this macro as ELPA package and initialize in the way described in readme then it's not going to work.

@thomasf
Copy link
Contributor

thomasf commented Jun 22, 2015

btw. emacs-25, ie. the master branch now does this to the top of the user init.el

;; Added by Package.el.  This must come before configurations of
;; installed packages.  Don't delete this line.  If you don't want it,
;; just comment it out by adding a semicolon to the start of the line.
;; You may delete these explanatory comments.
 (package-initialize)

@marcinant
Copy link
Author

Well it's just wierd. Default policy is: use init.el to set up config with after-init-hooks, then 'package-initialize' is issued automatically by 'startup.el' right after 'init.el' is loaded.

There is no good reason to run 'package-initialize' from 'init.el'.

@thomasf
Copy link
Contributor

thomasf commented Jun 22, 2015

Weird or not, the default policy seems to be changing.

I find it a bit less weird with (package-initialize) at the top than having to wrap most of init.el inside hook functions when the only reason is to wait for package.el. The new policy will probably be a lot less confusing for many Emacs newbies as well.

@errge
Copy link

errge commented Jul 26, 2017

I made a pull request to implement something similar to thomasf's setup, but a simpler way: #487

@raxod502
Copy link
Collaborator

raxod502 commented Mar 3, 2018

Default policy is: use init.el to set up config with after-init-hooks, then 'package-initialize' is issued automatically by 'startup.el' right after 'init.el' is loaded.

Sorry, but I disagree. There is simply no practical disadvantage to running package-initialize in the init-file, and it makes things so much simpler. Putting your entire configuration in after-init-hook? That's a needless layer of indirection.

In any case, Emacs now initializes the package system before loading the init file, thus rendering this entire debate superfluous. The feature will be released in Emacs 27.

@jwiegley
Copy link
Owner

jwiegley commented Mar 3, 2018 via email

@raxod502
Copy link
Collaborator

raxod502 commented Mar 3, 2018

this can be disabled, right?

In ~/.emacs.d/early-init.el, place:

(setq package-enable-at-startup nil)

xruins added a commit to xruins/dotfiles that referenced this issue Feb 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants