Invalid function: ido-ubiquitous-with-override

[skeeto]

It appears something is still broken with the autoloads. After installing ido-ubiquitous from MELPA I get Invalid function: ido-ubiquitous-with-override whenever I try to rename a file with R in a dired buffer, among a few other situations. After this error, my whole Emacs session slowly unravels -- font locking breaks, minibuffer becomes unusable -- and I need to restart Emacs.

Here's how to reproduce

mkdir /tmp/$$
HOME=/tmp/$$ emacs -Q -l test.el .

Here's test.el:

;;; test.el
(require 'package)

(add-to-list 'package-archives
             '("melpa" . "http://melpa.milkbox.net/packages/") t)
(package-initialize)
(package-refresh-contents)
(unless (package-installed-p 'ido-ubiquitous)
  (package-install 'ido-ubiquitous))

(require 'ido-ubiquitous)

(ido-mode 1)
(ido-ubiquitous-mode)

After Emacs starts, type R and you'll get the error message.

[noinil]

Getting the same error!
Seems to be related to the ido file behavior?
Everything is OK with ido buffer behavior, but when I tried to C-x C-f to open a file in the `Ibuffer' buffer, I got the error "Invalid function: ido-ubiquitous-with-override"

btw, I got this ERROR at Emacs startup:
ido-ubiquitous-2.0.4/ido-ubiquitous.elc:Error: Symbol's value as variable is void: cmd

[DarwinAwardWinner]

Can you start emacs with -f toggle-debug-on-error so that it will give you backtraces for all errors, and then post the backtraces here? That will help me track down the sources of these errors. Unfortunately, the errors seem to be caused by subtle order-of-execution bugs resulting from autoloading and macro expansion; tricky stuff to debug.

[skeeto]

Debugger entered--Lisp error: (invalid-function ido-ubiquitous-with-override)
  ido-ubiquitous-with-override(disable ("ChangeLog" "README.md" "test.el" ".git/" "../" "ido-ubiquitous.el" "./"))
  read-file-name-internal("" file-exists-p t)
  all-completions("" read-file-name-internal file-exists-p)
  completing-read-ido("Rename ChangeLog to: " read-file-name-internal file-exists-p nil "/tmp/ido-ubiquitous/" file-name-history "/tmp/ido-ubiquitous/ChangeLog" nil)
  completing-read("Rename ChangeLog to: " read-file-name-internal file-exists-p nil "/tmp/ido-ubiquitous/" file-name-history "/tmp/ido-ubiquitous/ChangeLog")
  read-file-name-default("Rename ChangeLog to: " "/tmp/ido-ubiquitous/" "/tmp/ido-ubiquitous/ChangeLog" nil nil nil)
  ad-Orig-read-file-name("Rename ChangeLog to: " "/tmp/ido-ubiquitous/" "/tmp/ido-ubiquitous/ChangeLog" nil nil nil)
  read-file-name("Rename ChangeLog to: " "/tmp/ido-ubiquitous/" "/tmp/ido-ubiquitous/ChangeLog")
  apply(read-file-name ("Rename ChangeLog to: " "/tmp/ido-ubiquitous/" "/tmp/ido-ubiquitous/ChangeLog"))
  dired-mark-pop-up(nil move ("ChangeLog") read-file-name "Rename ChangeLog to: " "/tmp/ido-ubiquitous/" "/tmp/ido-ubiquitous/ChangeLog")
  dired-mark-read-file-name("Rename %s to: " "/tmp/ido-ubiquitous/" move nil ("ChangeLog") "/tmp/ido-ubiquitous/ChangeLog")
  dired-do-create-files(move dired-rename-file "Move" nil t "Rename")
  dired-do-rename(nil)
  ad-Orig-call-interactively(dired-do-rename nil nil)
  call-interactively(dired-do-rename nil nil)

[DarwinAwardWinner]

@skeeto, I see you've installed through MELPA. Can you tell me which version of ido-ubiquitous you are on? In ido-ubiquitous.el, near the top, there should be a version number. The latest is 2.0.4, which I believe fixes this bug.

[skeeto]

The backtrace was 2.0.2 from MELPA. The reason I was reporting the issue
with the MELPA version was because it has something to do with the way
MELPA is generating autoloads.

I just built a 2.0.4 package locally with MELPA ("make recipes/ido-ubiquitous")
and I'm still getting the exact same issue on 0cb5409 (v2.0.4).

[DarwinAwardWinner]

You're still getting the "(invalid-function ido-ubiquitous-with-override)" with 2.0.4? Same backtrace? Did you do it in a clean emacs, including clearing the elpa folder and reinstalling?

[skeeto]

Ryan Thompson notifications@github.com writes:

Did you do it in a clean emacs, including clearing the elpa folder and
reinstalling?

Yup. Here's a full terminal session setting everything up from scratch.
You should be able to repeat these same steps and get the error. Hit R
(dired-do-rename) after Emacs starts and it produces the exact same
backtrace as before. Note that it must actually be invoked by the
keystroke, not through M-x.

$ emacs --version
GNU Emacs 24.3.1
Copyright (C) 2013 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.
$ mkdir /tmp/$$
$ cd /tmp/$$
$ pwd
/tmp/12396
$ ls -a # an empty directory
. ..
$ git clone https://github.com/milkypostman/melpa.git
Cloning into 'melpa'...
remote: Counting objects: 8903, done.
remote: Compressing objects: 100% (5102/5102), done.
remote: Total 8903 (delta 4442), reused 7962 (delta 3525)
Receiving objects: 100% (8903/8903), 1.21 MiB | 1.94 MiB/s, done.
Resolving deltas: 100% (4442/4442), done.
Checking connectivity... done
$ cd melpa/
$ make recipes/ido-ubiquitous
• Building recipe ido-ubiquitous ...
emacs --no-site-file --batch -l package-build.el --eval "(package-build-archive 'ido-ubiquitous)"
Loading /tmp/12396/melpa/json-fix.el (source)...

;;; ido-ubiquitous

Fetcher: github
Source: DarwinAwardWinner/ido-ubiquitous

Cloning git://github.com/DarwinAwardWinner/ido-ubiquitous.git to /tmp/12396/melpa/working/ido-ubiquitous/
Loading vc-git...
Saving file /tmp/12396/melpa/packages/ido-ubiquitous-20130904.1515.el...
Wrote /tmp/12396/melpa/packages/ido-ubiquitous-20130904.1515.el
Wrote /tmp/12396/melpa/packages/ido-ubiquitous-readme.txt
Wrote /tmp/12396/melpa/packages/archive-contents
Built in 0.043s, finished at Thu Sep 5 00:42:26 2013
✓ Wrote 28K -rw-r--r-- 1 wellons wellons 26K Sep 5 00:42 ./packages/ido-ubiquitous-20130904.1515.el
4.0K -rw-r--r-- 1 wellons wellons 343 Sep 5 00:42 ./packages/ido-ubiquitous-readme.txt

$ ls packages/
archive-contents ido-ubiquitous-20130904.1515.el ido-ubiquitous-readme.txt
$ grep Version packages/ido-ubiquitous-20130904.1515.el
;; Version: 20130904.1515
;; X-Original-Version: 2.0.4
$ HOME=$(pwd) emacs -Q -f toggle-debug-on-error --eval '(package-install-file "packages/ido-ubiquitous-20130904.1515.el")' -f ido-mode -f ido-ubiquitous-mode .

[DarwinAwardWinner]

What if I just remove all autoload markings from ido-ubiquitous?

[DarwinAwardWinner]

I'm sorry, I just can't figure out why it doesn't work when installed through MELPA. I think maybe MELPA should stick to the emacs23 branch of ido-ubiquitous until I have time to sort this out (which I won't in the next week, unfortunately).

If anyone knows why there is a difference when installing and loading through MELPA, please tell me.

[swsnr]

@DarwinAwardWinner I am not sure that autoloads are the culprit here (imho there is some kind of a macro expansion issue in the byte compiler), but I am not sure either, whether all the autoloads are really needed.

I would not place autoloads on anything else, especially not on defcustoms and defconsts. None of these are ever used before ido-ubiquitous-mode is actually enabled. Imho only the autoload on ido-ubiquitous-mode is really required, and every other should be removed.

[noinil]

@DarwinAwardWinner
I'm not quite sure whether it's problem with MELPA. I git clone the 2.0.4 version from here, and load the ido-ubiquitous.el into Emacs, and use package-install-from-buffer to install, and still get the same errors. :(

Now I have to use version 1.7 back again and everything's OK at last.

[DarwinAwardWinner]

Well, perhaps it isn't MELPA, but package.el doing something odd. What if you just load the file directly without doing any "install"?

[noinil]

@DarwinAwardWinner
Yes you are right.
Directly loading the file works fine with ibuffer and dired functions.

[DarwinAwardWinner]

Ok, I think I have fixed this in 2.0.5.

[noinil]

Thanks for fixing the bug!

I tried version 2.0.5 and got the ERROR:
ido-ubiquitous-2.0.5/ido-ubiquitous.elc:Warning: reference to free variable
ido-cur-item' ido-ubiquitous-2.0.5/ido-ubiquitous.elc:Warning: reference to free variable ido-default-item'
ido-ubiquitous-2.0.5/ido-ubiquitous.elc:Warning: reference to free variable
`ido-cur-list'
ido-ubiquitous-2.0.5/ido-ubiquitous.elc:Error: Symbol's value as variable is void: cmd
Ido-ubiquitous mode disabled

What is the 'cmd' variable?

[skeeto]

I've got it worked out without having to touch MELPA, and, unfortunately, it's still not fixed in 2.0.5. I also just discovered your use of cmd in the defadvice, which you corrected in 7ccd8c9. However, that (ad-get-arg 0) just expands to exactly the same thing, relying on dynamic binding to show the argument through. To see this, run (ad-get-advice-info 'command-execute) and look at the expanded advice code.

What seems to be happening is it's throwing an error on this cmd issue while loading, leaving critical macros and functions undefined, specifically ido-ubiquitous-with-override . I'm pretty sure this is actually a bug in advice (one of many!), probably introduced by lexical scoping in Emacs 24, so you may have to take a different approach. (How did this work before?)

Here's how I'm testing it now right in the repository:

rm -rf /tmp/$$   # empty .emacs.d and elpa
mkdir /tmp/$$
HOME=/tmp/$$ emacs -Q -f toggle-debug-on-error \
                   --eval '(package-install-file "ido-ubiquitous.el")' \
                   -f ido-mode -f ido-ubiquitous-mode

After running ido-ubiquitous-mode it will fail to autoload.

A really dirty hack that fixes even v2.0.4 is to simply add this before defining the advice.

(defvar cmd nil)

Then it doesn't fail to load (the variable exists), all the functions get defined as needed. Everything works.

[DarwinAwardWinner]

Why doesn't Emacs stop and show a backtrace when it encounters the void variable error? Anyway, would it be better to defvar the cmd variable or to access it via (bound-and-true-p cmd)?

[swsnr]

bound-and-true-p. you really don't want to make the generic name cmd a
global special variable.
Am 05.09.2013 19:22 schrieb "Ryan Thompson" <>:

Why doesn't Emacs stop and show a backtrace when it encounters the void
variable error? Anyway, would it be better to defvar the cmd variable or to
access it via (bound-and-true-p cmd)?

—
Reply to this email directly or view it on GitHub.

[DarwinAwardWinner]

Ok, I added the bound-and-true-p in 2.0.7 (specifically 0f3bb89).

[skeeto]

Thanks! 0f3bb89 seems to fix it for me. I'll close this issue once 2.0.7 makes it into MELPA later today and works properly (unless you want to close it yourself now).

[DarwinAwardWinner]

You close it when MELPA pushes the update.