Packages installed using file fetcher are always marked as outdated

[d12frosted]

Description

Package installed using file fetcher are always marked as outdated when pressing 'update packages' button.

Reproduction guide 🪲

• Start Emacs.
• Press 'update packages' button.
• Notice that evil-unimpaired has new version (and probably some other packages).
• Ask Emacs to update them.
• Restart Emacs so packages are updated.
• Press update button again.

Observed behaviour: 👀 💔
Package evil-unimpaired has new version.

Expected behaviour: ❤️ 😄
Package evil-unimpaired wasn't changed. So it should not be marked as outdated.

System Info 💻

• OS: darwin
• Emacs: 25.1.50.1
• Spacemacs: 0.105.21
• Spacemacs branch: develop (rev. 7a44019)
• Graphic display: t
• Distribution: spacemacs
• Editing style: emacs
• Completion: ivy
• Layers:

(agda
 (auto-completion :disabled-for org)
 better-defaults colors dash erc
 (git :variables git-magit-status-fullscreen t)
 github
 (org :variables org-enable-github-support t)
 pandoc ranger restclient syntax-checking spacemacs-layouts ivy spell-checking shell version-control search-engine no-dots d12frosted-core d12frosted-org d12frosted-lua d12frosted-haskell d12frosted-lisp d12frosted-visual d12frosted-spellchecking emacs-lisp
 (haskell :variables haskell-completion-backend 'ghci)
 (html :variables web-mode-markup-indent-offset 2 web-mode-attr-indent-offset 2 web-mode-code-indent-offset 2 web-mode-css-indent-offset 2 web-mode-sql-indent-offset 2 css-indent-offset 2)
 (javascript :variables js-indent-level 2 js2-basic-offset 2)
 lua markdown shell-scripts yaml react)

[ynilu]

I can comfirm this and I got this error in the Compile-Log buffer
evil-unimpaired.el:32:1:Error: Cannot open load file: no such file or directory, dash

[bixuanzju]

This is annoying, as I constantly want to update packages...

[d12frosted]

Cannot open load file: no such file or directory, dash

This is strange, evil-unimpaired defined dash as dependency. I never seen this problem before.

This is annoying, as I constantly want to update packages...

Sorry about that. I still haven't looked into this problem. Will have some free time on weekends to investigate this issue.

[d12frosted]

It seems that it's an upstream issue with quelpa. Whenever I install package using file fetcher and then use quelpa-checkout to check if there is new version of that package, quelpa creates recipe.stamp in .cache/quelpa/build/evil-unimpaired which contains newer version. So quelpa-checkout returns version string which means that there is update (following it's own documentation string). Gonna reproduce it with clean Emacs before reporting.

[d12frosted]

I can't reproduce such behaviour in Emacs -Q. Snippet I used to test:

(package-initialize)
(if (require 'quelpa nil t)
    (quelpa-self-upgrade)
  (with-temp-buffer
    (url-insert-file-contents "https://raw.github.com/quelpa/quelpa/master/bootstrap.el")
    (eval-buffer)))

(setq
 pkg-name 'evil-unimpaired
 recipe `(,pkg-name
      :fetcher file
      :path "/Users/d12frosted/.spacemacs/layers/+spacemacs/spacemacs-evil/local/evil-unimpaired/evil-unimpaired.el"))

(quelpa recipe)

(quelpa-checkout recipe (expand-file-name
             (symbol-name pkg-name)
             quelpa-build-dir))

The final sexp always returns nil. Which is expected.

[bixuanzju]

@d12frosted Any update?

[dyxushuai]

@ynilu Same problem with me! Any suggestion?

[zzjjzzgggg]

Same problem here too, for days.

[NOBLES5E]

Same problem here for weeks.

[dyxushuai]

Uh...
I have to add it to dotspacemacs-excluded-packages

[d12frosted]

@bixuanzju No, there are no updates right now. If you are not using this package - the best you can do is to exclude it. But this issue is on my list (not top priority though). If someone wants to dig into it - don't hesitate. As you might already noticed - PRs are warmly welcome 😸

P. S. Sorry for long replay - was away from civilisation 😸

[korayal]

I added the dash package in spacemacs to dotspacemacs-configuration-layers but still having this exact same error

[d12frosted]

@korayal why dash? It's not installed using file fetcher. Try excluding evil-unimpaired.

[korayal]

@d12frosted well, I thought since evil-unimpaired required dash and since it's not installed automatically, I wanted to try out this if it would fix the not found error @ynilu mentioned in the second post.

Btw, dash may mean two different things, there is a dash which is a minimalist shell, and there is another dash which is about documentation. I don't know which is required here and why

[TheBB]

There are three dashes. The two you mention, and the Emacs package.

When evil-unimpaired requires dash, we're talking about dependencies between Emacs packages. (As you can see from the link, there are not many things that don't depend on dash.) It has nothing whatsoever to do with the documentation search tool dash (which is supported by the Emacs packages helm-dash and counsel-dash, and which you can obtain by using the dash layer (not the dash package)). It has even less to do with the dash shell which has nothing to do with Emacs.

[TheBB]

And to be precise, you cannot install the dash package by adding dash to the list of layers, that will install the dash layer.

[d12frosted]

There are three dashes. The two you mention, and the Emacs package.

@korayal Oh well. Since we are discussing 'Packages installed using file fetcher are always marked as outdated' topic, I am talking only about dash Emacs package. Which is required by evil-unimapried and which was the reason of startup errors before we started to use file fetcher for evil-unimpaired local package.

@TheBB thanks for the comprehensive explanations 💯

[d12frosted]

well, I thought since evil-unimpaired required dash and since it's not installed automatically, I wanted to try out this if it would fix the not found error

Hm. How does totally ignoring required package makes 'not found error' gone? 😸 But keep in mind that ignoring evil-unimpaired is needed only if you are using update packages button from spacemacs home buffer (or corresponding function). If you are ignoring it just to make the error about dash go away, you better update to latest develop branch (if you are on develop branch). Or well, just add evil-unimpaired to list of ignored packages 😸

[korayal]

@d12frosted it didn't make the error gone I thought the dash layer is related to the dash package, so added that to the list :) And yeah I'm always using the update-packages command every day and I am on develop branch. I'm using the evil-unimpaired keybindings a lot so I wanted to make sure nothing is wrong about that package.

@TheBB whould adding it to the dotspacemacs-additional-packages install that package?

[TheBB]

It would, but I find it hard to believe you don't already have it as a dependency of something else.

[d12frosted]

@korayal Ah I see 😸 Thanks for clarification.

Well, in case you use evil-unimpaired and update packages you'll have to wait for the fix for this issue (#6610). 😿

[andyleejordan]

I only recently started seeing this bug, and it happens when my Solarized theme is getting loaded. I did a Git bisect on the develop branch, and found:

5d6e9ab7892501079bba41afe0dcfb043d62be10 is the first bad commit
commit 5d6e9ab7892501079bba41afe0dcfb043d62be10
Author: ivanbrennan <ivan.brennan@gmail.com>
Date:   Fri Aug 12 08:47:27 2016 -0400

    optimize spacemacs//get-package-directory

    Saving a pkg-match regex to a temporary variable lets us avoid recomputing it on each
    iteration. I didn't benchmark the performance impact, but unless there's a downside to
    using `let*` vs `let`, it seems like a no-brainer.

    Passing pkg-match to `directory-files` reduces the returned files to those that match the
    regex rolling our own reduction.

:040000 040000 9795f52c42e9f41f13c760a4d6be1bc76cbd47a7 5802afca4915ead3fb67ed7eafc11d3993c9af81 M  core

The funny thing is that I didn't even have to clear package caches to find this; I get:

Warning (spacemacs): An error occurred while retrieving the theme, using default theme. (error: (file-error Cannot open load file No such file or directory dash))

only on that commit (and newer); but Spacemacs and Solarized load properly before it.

I'll see if I can find what's going on, but at least we have a place to start.

[andyleejordan]

Sure enough, simply reverting the commit on top of the tip of develop fixes my problem.

[d12frosted]

@andschwa I don't get the relation between your problem and this issue. As it's not about loading packages, but rather about packages being always marked as outdated when file fetcher is used to install them. From user perspective it's actually local fetcher, but under the hood we use file fetcher from quelpa.

[andyleejordan]

@d12frosted Yeah, I'm not sure either. The cause doesn't look the same. Google brought me here because @ynilu had the same error message, but in a different place. I probably have a different problem 😦

[robbyoconnor]

Couldn't quelpa use a hash of the file to determine if it changed?

[andyleejordan]

@d12frosted I do really need to a file a different but. But I've reproduced the error:

Warning (spacemacs): An error occurred while retrieving the theme, using default theme. (error: (file-error Cannot open load file No such file or directory dash))

On everything I use: Ubuntu 14.04 with Emacs built from source (emacs-25 branch), Ubuntu 16.04 using emacs-snapshot, CentOS 7 (GNU Emacs 25.0.95.1), and Emacs from home brew on OS X.

And reverting 5d6e9ab fixes it every time.

[ynilu]

Hi everyone,
After a long dig, I finally found the cause of this issue. I made a pull request #7220 for this issue. Please help me to verify the changes. I am still a Elisp newbie 😸

[d12frosted]

Thanks to @ynilu the issue is fixed in develop branch. 💯 ❤️

[TheBB]

Should be fixed in master.