Spell-Fu marks every word as incorrect

[ngharrison]

Main Problem
When I enable Spell-Fu (spell-fu-mode using SPC t s), every word in a buffer is underlined in red.
When jumping to the next spelling error (+spell/next-error using ] s), it jumps to every word.
If the word is checked (+spell/correct using z =), it does correctly tell if it is spelled correctly or gives suggestions if incorrect.

I'm on Windows 10 and just recently updated to Emacs 27.1 and the latest release of Doom. The problem didn't occur before that.

Examples
With Spell-Fu enabled:

Type z = with point on "sentence", and it prints out:

Type z = with point on "sentance", and it prints out:

System information:

((system
  (type . windows-nt)
  (config . "x86_64-w64-mingw32")
  (shell . "C:/Program Files/emacs-27.1-x86_64/libexec/emacs/27.1/x86_64-w64-mingw32/cmdproxy.exe")
  (uname . "n/a")
  (path "~/Anaconda3/condabin" "C:/Strawberry/c/bin" "C:/Strawberry/perl/site/bin" "C:/Strawberry/perl/bin" "C:/Program Files/ImageMagick-7.0.7-Q16" "~/Anaconda3" "~/Anaconda3/Scripts" "C:/ProgramData/Oracle/Java/javapath" "C:/Program Files (x86)/Intel/iCLS Client/" "C:/Program Files/Intel/iCLS Client/" "C:/WINDOWS/system32" "C:/WINDOWS" "C:/WINDOWS/System32/Wbem" "C:/WINDOWS/System32/WindowsPowerShell/v1.0/" "C:/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL" "C:/Program Files/Intel/Intel(R) Management Engine Components/DAL" "C:/Program Files (x86)/Intel/Intel(R) Management Engine Components/IPT" "C:/Program Files/Intel/Intel(R) Management Engine Components/IPT" "C:/Program Files/Intel/WiFi/bin/" "C:/Program Files/Common Files/Intel/WirelessCommon/" "C:/Program Files/nodejs/" "C:/Program Files/Git/cmd/" "~/AppData/Local/Android/sdk/platform-tools" "C:/Program Files/Java/jdk1.8.0_131/bin" "C:/Program Files/MATLAB/R2019a/runtime/win64" "C:/Program Files/MATLAB/R2019a/bin" "C:/Program Files/PuTTY/" "C:/Program Files (x86)/PDFtk/bin/" "C:/Program Files/Sublime Text 3" "C:/WINDOWS/System32/OpenSSH/" "C:/Program Files (x86)/Wolfram Research/WolframScript/" "C:/Program Files/CMake/bin" "C:/WINDOWS/system32" "C:/WINDOWS" "C:/WINDOWS/System32/Wbem" "C:/WINDOWS/System32/WindowsPowerShell/v1.0/" "C:/WINDOWS/System32/OpenSSH/" "C:/Program Files/MiKTeX 2.9/miktex/bin/x64/" "C:/Program Files/SumatraPDF" "C:/Program Files/Julia-1.4.0/bin" "C:/Program Files/Neovim/bin" "C:/Program Files/Pandoc/" "C:/Program Files/emacs-27.1-x86_64/bin" "C:/Installers/libxml2-2.9.3-win32-x86_64/bin" "C:/Program Files/ripgrep/ripgrep-12.1.1-x86_64-pc-windows-msvc" "~/.emacs.d/bin" "C:/Program Files/fd/fd-v8.1.1-x86_64-pc-windows-msvc" "C:/Program Files (x86)/sqlite" "C:/Ruby26-x64/msys64/mingw64/bin" "C:/Ruby26-x64/msys64/usr/bin" "C:/Ruby26-x64/bin" "~/AppData/Local/Microsoft/WindowsApps" "~/AppData/Roaming/npm" "~/AppData/Local/Box/Box Edit/" "C:/Program Files (x86)/Oni" "~/AppData/Local/Programs/Microsoft VS Code/bin" "c:/Program Files/emacs-27.1-x86_64/libexec/emacs/27.1/x86_64-w64-mingw32"))
 (emacs
  (dir . "~/.emacs.d/")
  (version . "27.1")
  (build . "Aug 21, 2020")
  (buildopts . "--without-dbus --host=x86_64-w64-mingw32 --without-compress-install 'CFLAGS=-O2 -static'")
  (features . "XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY W32NOTIFY ACL GNUTLS LIBXML2 HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS JSON PDUMPER LCMS2 GMP")
  (traits batch server-running envvar-file))
 (doom
  (dir . "~/.doom.d/")
  (version . "2.0.9")
  (build . "HEAD -> develop 1456108d5 2020-09-10 18:03:32 -0400")
  (elc-files . 0)
  (modules :completion company (ivy +fuzzy +prescient +icons) :ui doom hl-todo hydra modeline ophints (popup +defaults) treemacs vc-gutter vi-tilde-fringe (window-select +numbers) workspaces zen :editor (evil +everywhere) file-templates fold multiple-cursors snippets word-wrap :emacs (dired +icons) electric (ibuffer +icons) undo vc :term eshell :checkers syntax (spell +aspell) :tools ein (eval +overlay) (lookup +dictionary) magit pdf :lang data emacs-lisp (latex +latexmk +cdlatex) markdown (org +jupyter +gnuplot +roam) (python +conda) sh yaml :config (default +bindings +smartparens))
  (packages "n/a")
  (unpin "n/a")
  (elpa "n/a")))

[gatlee]

Do you have aspell or flyspell enabled?

[ngharrison]

I have aspell enabled and installed. The line in my init.el file looks like (spell +aspell).

[gatlee]

Yeah I'm having the same issue

[askurihin]

spell-fu caches words in ~/.emacs.d/.local/etc/spell-fu
I had the same issue, but it got fixed, after I deleted ~/.emacs.d/.local/etc/spell-fu/*

[gatlee]

Perfect, that fixes it for me. Thanks!

[ngharrison]

I deleted the files in ~/.emacs.d/.local/etc/spell-fu, but the problem remained. It had the same behavior with the files gone before I restarted emacs as well as after I restarted and the files were re-created. As a note, the two files I found in the directory were words_en.el.data, and words_en.txt.

Any ideas as to how I could debug this?

[askurihin]

Usually, if deleting those files does not help, you can look inside the files and see errors, that
can give you a clue how to proceed.

[ngharrison]

Sounds smart. Since I don't know what these files are supposed to contain, could you tell me what is in yours? The first few lines of mine are:

~/.emacs.d/.local/etc/spell-fu/words_en.txt

A
a
A's
AA
AA's
AAA

~/.emacs.d/.local/etc/spell-fu/words_en.el.data

(("version" . "0.1"))#s(hash-table size 127250 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("a
" t "a's
" t "aa
" t "aa's
" t "aaa
" t "aachen

[gatlee]

Looks like the same as mine but with
words_en_AU.txt and words_en_AU.el.data

[ngharrison]

I've done some more testing and have found the following:

• I get the same incorrect behavior on emacs 26.3, which I was using up until a few weeks ago
• I reverted back to a doom version from the end of July and got the correct behavior that I had seen before. I see now this is because at that point flyspell was being used. Spell-fu was introduced Aug 21 on commit ff9c1ac.
• Spell-fu with aspell works on an instance inside wsl

I should also note that I tried each of these on a base installation of doom without any private configurations besides addding (spell +aspell) to init.el. So the problem remains and, as far as I can tell, is something to do with spell-fu on windows or how doom handles it.

[ngharrison]

I finally discovered it was the carriage returns placed in the words_en.el.data file on Windows. Manually deleting all of them from the file brought back normal behavior.

[hlissner]

So the issue is that spell-fu generates the word list in the wrong encoding? (i.e. wrong newlines)

Should this be reported upstream (to spell-fu)?

[ngharrison]

That sounds like the right action to me.

A few other notes about functionality if anyone cares:

• Words in ALL CAPS are always taken as correct.
• There are some instances where spell-fu marks a word as incorrect, but aspell says it is correct. One case I saw was "rightarrow".

I guess these are probably intended features or side-effects of spell-fu functionality and should be referred to the upstream repository as well.

[shombando]

@ngharrison thanks for documenting your work around. Did you report this bug upstream? I'd like to follow it but can't find it.

@hlissner you're right the line encoding for words_en_US.txt is generated as CR LF but the parsed elisp array stored in words_en_US.el.data only removes LF so the array is broken up with CRs. Ha! So is the generator using the wrong encoding or the parser? 🐔/🥚➡️🐣 Schrodinger's chicken!

[ngharrison]

@shombando Thanks for the reminder to file it upstream. I guess you tend to forget about things when they start working again. It is now up: https://gitlab.com/ideasman42/emacs-spell-fu/-/issues/13

[jcguu95]

This might related to the issue here. I also got most words labeled wrong. Here is the post and solution that works for me.

OP

Anyone using aspell to check spelling? I have both aspell and aspell-en installed in my arch machine. I also have :checker (spell +aspell) enabled in doom, and have doom sync and relaunched emacs. However, spell-fu still marks most words incorrect.

The directory of spell-fu points to ~/.emacs.d/.local/etc/spell-fu/words_english.txt. In that file, it says "Error: The file "/usr/lib/aspell-0.60/english" can not be opened for reading." I ls | grep english that dir, and only see

english.alias
english-variant_0.alias
english-variant_1.alias
english-variant_2.alias
english-w_accents.alias
english-wo_accents.alias

Certainly spell-fu is looking for the wrong stuff. How should I fix this within doomemacs?

(Related: #4009)

Solution

Related packages: spell-fu, aspell, and ispell.

On arch, I installed packages aspell and aspell-en using pacman.
On doomemacs, I enabled the module (:checkers (spell +aspell)).

Use M-x spell-fu-word-add to add new words to personal
dictionary.

(setq ispell-program-name "aspell")
(setq ispell-extra-args '("--sug-mode=ultra" "--lang=en_US"))
(setq spell-fu-directory "~/+STORE/dictionary") ;; Please create this directory manually.
(setq ispell-personal-dictionary "~/+STORE/dictionary/.pws")