The build process is so slow, it's already been a day and it's still compiling.

[gorakhargosh]

I'm building Remacs on a relatively new machine with ample RAM,
but this thing is taking forever to finish building. What is it trying to do?

git:(master)  [130]fish> make CARGO_FLAGS="-vv" RUSTFLAGS="-Zunstable-options --cfg MARKER_DEBUG" -j 16
make -C lib all
make info-real info-dir
make[1]: Entering directory '/home/yesudeep/code/remacs/lib'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/yesudeep/code/remacs/lib'
make -C lib-src all
make[1]: Entering directory '/home/yesudeep/code/remacs'
make -C doc/lispref info
make -C doc/lispintro info
make -C doc/emacs info
make -C doc/misc info
make[1]: Nothing to be done for 'info-dir'.
make[1]: Entering directory '/home/yesudeep/code/remacs/lib-src'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/yesudeep/code/remacs/lib-src'
make[2]: Entering directory '/home/yesudeep/code/remacs/doc/lispintro'
make[2]: Nothing to be done for 'info'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/doc/lispintro'
make -C src VCSWITNESS='$(srcdir)/../.git/logs/HEAD' all
make[2]: Entering directory '/home/yesudeep/code/remacs/doc/misc'
make[2]: Nothing to be done for 'info'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/doc/misc'
make[2]: Entering directory '/home/yesudeep/code/remacs/doc/lispref'
make[2]: Nothing to be done for 'info'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/doc/lispref'
make[2]: Entering directory '/home/yesudeep/code/remacs/doc/emacs'
make[2]: Nothing to be done for 'info'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/doc/emacs'
make[1]: Leaving directory '/home/yesudeep/code/remacs'
make[1]: Entering directory '/home/yesudeep/code/remacs/src'
RUSTFLAGS="-Zunstable-options --cfg MARKER_DEBUG" \
EMACS_CFLAGS="-Demacs  -I. -I. -I../lib -I../lib   -pthread -isystem /usr/include/gtk-3.0 -isystem /usr/include/at-spi2-atk/2.0 -isystem /usr/include/at-spi-2.0 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include -isystem /usr/include/gtk-3.0 -isystem /usr/include/gio-unix-2.0 -isystem /usr/include/cairo -isystem /usr/include/libdrm -isystem /usr/include/pango-1.0 -isystem /usr/include/harfbuzz -isystem /usr/include/pango-1.0 -isystem /usr/include/fribidi -isystem /usr/include/atk-1.0 -isystem /usr/include/cairo -isystem /usr/include/pixman-1 -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/gdk-pixbuf-2.0 -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /usr/include/libpng16     -isystem /usr/include/libpng16 -isystem /usr/include/libxml2 -isystem /usr/include/dbus-1.0 -isystem /usr/lib/x86_64-linux-gnu/dbus-1.0/include       -pthread -isystem /usr/include/libmount -isystem /usr/include/blkid -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/glib-2.0 -isystem /usr/lib/x86_64-linux-gnu/glib-2.0/include -isystem /usr/include/freetype2 -isystem /usr/include/libpng16 -isystem /usr/include/uuid -isystem /usr/include/freetype2 -isystem /usr/include/libpng16   -MMD -MF deps/../rust_src/target/debug/libremacs_lib.d -MP   -isystem /usr/include/p11-kit-1    -pthread" \
SRC_HASH="26d252d6bbef8e816d9d257989ec6c908a24d399979608b5c7b9fb8c6b98df26" \
cargo build -vv --manifest-path ../rust_src/Cargo.toml
mkdir -p ../rust_src/target/
make -C ../admin/charsets all
make -C ../admin/unidata charscript.el
make[2]: Entering directory '/home/yesudeep/code/remacs/admin/unidata'
echo "-vv -Zunstable-options --cfg MARKER_DEBUG" > ../rust_src/target/.rustflags.tmp
make -C ../admin/charsets cp51932.el
make -C ../admin/charsets eucjp-ms.el
diff -q ../rust_src/target/.rustflags ../rust_src/target/.rustflags.tmp || cp ../rust_src/target/.rustflags.tmp ../rust_src/target/.rustflags
make[2]: Entering directory '/home/yesudeep/code/remacs/admin/charsets'
make[2]: Nothing to be done for 'cp51932.el'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/admin/charsets'
make[2]: Entering directory '/home/yesudeep/code/remacs/admin/charsets'
make[2]: Nothing to be done for 'eucjp-ms.el'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/admin/charsets'
rm -f ../rust_src/target/.rustflags.tmp
make[2]: Nothing to be done for 'charscript.el'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/admin/unidata'
make[2]: Entering directory '/home/yesudeep/code/remacs/admin/charsets'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/admin/charsets'
       Fresh semver-parser v0.7.0
       Fresh unicode-xid v0.0.4
       Fresh quote v0.3.15
       Fresh build_const v0.2.1
       Fresh autocfg v0.1.2
       Fresh lazy_static v1.2.0
       Fresh unicode-xid v0.1.0
       Fresh rand_core v0.4.0
       Fresh ident_case v1.0.0
       Fresh cfg-if v0.1.6
       Fresh cc v1.0.28
       Fresh byte-tools v0.3.1
       Fresh ucd-util v0.1.3
       Fresh adler32 v1.0.3
       Fresh utf8-ranges v1.0.2
       Fresh fake-simd v0.1.2
       Fresh bytesize v0.1.3
       Fresh opaque-debug v0.2.2
       Fresh safemem v0.3.0
       Fresh either v1.5.1
       Fresh field-offset v0.1.1
       Fresh sha1 v0.6.0
       Fresh md5 v0.6.1
       Fresh semver v0.9.0
       Fresh synom v0.11.3
       Fresh thread_local v0.3.6
       Fresh rand_core v0.3.1
       Fresh rand_jitter v0.1.2
       Fresh regex-syntax v0.5.6
       Fresh regex-syntax v0.6.5
       Fresh miniz_oxide v0.2.1
       Fresh block-padding v0.1.3
       Fresh line-wrap v0.1.1
       Fresh itertools v0.8.0
       Fresh rustc_version v0.2.3
       Fresh libc v0.2.48
       Fresh syn v0.11.11
       Fresh num-traits v0.2.6
       Fresh typenum v1.10.0
       Fresh proc-macro2 v0.4.29
       Fresh rand_isaac v0.1.1
       Fresh rand_xorshift v0.1.1
       Fresh rand_hc v0.1.0
       Fresh byteorder v1.3.1
       Fresh crc32fast v1.1.2
       Fresh darling_core v0.2.2
       Fresh memchr v2.1.3
       Fresh time v0.1.42
       Fresh memchr v1.0.2
       Fresh errno v0.2.4
       Fresh rand_os v0.1.2
       Fresh crc v1.8.1
       Fresh quote v0.6.12
       Fresh rand_chacha v0.1.1
       Fresh num-integer v0.1.39
       Fresh generic-array v0.12.0
       Fresh darling_macro v0.2.2
       Fresh base64 v0.10.1
       Fresh aho-corasick v0.6.9
       Fresh nom v3.2.1
       Fresh rand_pcg v0.1.1
       Fresh darling v0.2.2
       Fresh chrono v0.4.6
       Fresh syn v0.15.33
       Fresh block-buffer v0.7.2
       Fresh regex v0.2.11
       Fresh digest v0.8.0
       Fresh miniz_oxide_c_api v0.2.1
       Fresh regex v1.1.0
       Fresh rand v0.6.5
       Fresh remacs-util v0.1.0 (/home/yesudeep/code/remacs/rust_src/remacs-util)
       Fresh systemstat v0.1.4
       Fresh flate2 v1.0.6
       Fresh sha2 v0.8.0
       Fresh remacs-lib v0.1.0 (/home/yesudeep/code/remacs/rust_src/remacs-lib)
       Fresh remacs-macros v0.1.0 (/home/yesudeep/code/remacs/rust_src/remacs-macros)
       Fresh remacs v0.1.0 (/home/yesudeep/code/remacs/rust_src)
    Finished dev [unoptimized + debuginfo] target(s) in 0.04s
  CCLD     temacs
gcc: warning: switch ‘-Wchkp’ is no longer supported
/usr/bin/mkdir -p ../etc
make -C ../lisp update-subdirs
make[2]: Entering directory '/home/yesudeep/code/remacs/lisp'
make[2]: Leaving directory '/home/yesudeep/code/remacs/lisp'
./temacs --batch  --load loadup bootstrap
Loading loadup.el (source)...
Using load-path (/home/yesudeep/code/remacs/lisp /home/yesudeep/code/remacs/lisp/emacs-lisp /home/yesudeep/code/remacs/lisp/progmodes /home/yesudeep/code/remacs/lisp/language /home/yesudeep/code/remacs/lisp/international /home/yesudeep/code/remacs/lisp/textmodes /home/yesudeep/code/remacs/lisp/vc)
Loading emacs-lisp/byte-run...
Loading emacs-lisp/backquote...
Loading subr...
Loading version...
Loading widget...
Loading custom...
Loading emacs-lisp/map-ynp...
Loading international/mule...
Loading international/mule-conf...
Loading env...
Loading format...
Loading bindings...
Loading window...
Loading files...
Loading emacs-lisp/macroexp...
Loading cus-face...
Loading faces...
Loading button...
Loading /home/yesudeep/code/remacs/lisp/ldefs-boot.el (source)...
Loading emacs-lisp/nadvice...
Loading emacs-lisp/cl-preloaded...
Loading minibuffer...
Loading obarray...
Loading abbrev...
Loading simple...
Loading help...
Loading jka-cmpr-hook...
Loading epa-hook...
Loading international/mule-cmds...
Loading case-table...
Loading /home/yesudeep/code/remacs/lisp/international/charprop.el (source)...
Loading international/characters...
Loading composite...
Loading language/chinese...
Loading language/cyrillic...
Loading language/indian...
Loading language/sinhala...
Loading language/english...
Loading language/ethiopic...
Loading language/european...
Loading language/czech...
Loading language/slovak...
Loading language/romanian...
Loading language/greek...
Loading language/hebrew...
Loading international/cp51932...
Loading international/eucjp-ms...
Loading language/japanese...
Loading language/korean...
Loading language/lao...
Loading language/tai-viet...
Loading language/thai...
Loading language/tibetan...
Loading language/vietnamese...
Loading language/misc-lang...
Loading language/utf-8-lang...
Loading language/georgian...
Loading language/khmer...
Loading language/burmese...
Loading language/cham...
Loading indent...
Loading emacs-lisp/cl-generic...
Loading frame...
Loading startup...
Loading term/tty-colors...
Loading font-core...
Loading facemenu...
Loading emacs-lisp/syntax...
Loading font-lock...
Loading jit-lock...
Loading mouse...
Loading scroll-bar...
Loading select...
Loading emacs-lisp/timer...
Loading isearch...
Loading rfn-eshadow...
Loading menu-bar...
Loading emacs-lisp/lisp...
Loading textmodes/page...
Loading register...
Loading textmodes/paragraphs...
Loading progmodes/prog-mode...
Loading emacs-lisp/lisp-mode...
Loading progmodes/elisp-mode...
Loading textmodes/text-mode...
Loading textmodes/fill...
Loading newcomment...
Loading replace...
Loading emacs-lisp/tabulated-list...
Loading buff-menu...
Loading fringe...
Loading emacs-lisp/regexp-opt...
Loading image...
Loading international/fontset...
Loading dnd...
Loading tool-bar...
Loading dynamic-setting...
Loading x-dnd...
Loading term/common-win...
Loading term/x-win...
Loading mwheel...
Loading emacs-lisp/float-sup...
Loading vc/vc-hooks...
Loading vc/ediff-hook...
Loading uniquify...
Loading electric...
Loading emacs-lisp/eldoc...
Loading cus-start...
Loading tooltip...
Loading /home/yesudeep/code/remacs/lisp/leim/leim-list.el (source)...
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name remacs
11687872 of 33554432 static heap bytes used
97343 pure bytes used
mv -f remacs bootstrap-emacs
make -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
make[2]: Entering directory '/home/yesudeep/code/remacs/lisp'
make[2]: Nothing to be done for 'compile-first'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/lisp'
make -C ../lisp autoloads EMACS="../src/bootstrap-emacs"
make -C ../admin/unidata all EMACS="../../src/bootstrap-emacs"
make[2]: Entering directory '/home/yesudeep/code/remacs/admin/unidata'
make[2]: Entering directory '/home/yesudeep/code/remacs/lisp'
make -C ../leim all EMACS="../src/bootstrap-emacs"
make -C ../admin/grammars all EMACS="../../src/bootstrap-emacs"
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/yesudeep/code/remacs/admin/unidata'
make[3]: Entering directory '/home/yesudeep/code/remacs/admin/grammars'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/home/yesudeep/code/remacs/admin/grammars'
make[3]: Entering directory '/home/yesudeep/code/remacs/leim'
  GEN      ../lisp/leim/ja-dic/ja-dic.el
Reading file "/home/yesudeep/code/remacs/leim/SKK-DIC/SKK-JISYO.L" ...
Processing OKURI-ARI entries ...
Processing POSTFIX entries ...
Processing PREFIX entries ...
Collecting OKURI-NASI entries...
Collecting OKURI-NASI entries...26%

[shaleh]

I have never seen it take a day. The initial compile can take a while though. it definitely helps to pass in something like -j 4 so it runs in parallel. You will see a massive improvement. I have seen it go from over an hour to like 15 minutes that way.

(Edited, I just scrolled and saw the j16 at the end).

I suspect 16 is not going to work. Try something more tame like j4. I have seen build failures whenever I go over j8.

[gorakhargosh]

Hi. I'm not so sure about the problem. Even with -j4 it's taking quite a while.
I built GNU Emacs from source and that takes a lot less time to build than remacs.

This is taking a lot of time:

Converting cangjie-table.cns to tsang-cns.el...
Converting ziranma.cin to ZIRANMA.el...
Converting /home/yesudeep/code/remacs/leim/CXTERM-DIC/ARRAY30.tit to quail-package...
Decoding with coding system cn-big5...
Converting cangjie-table.b5 to tsang-b5.el...
Processing header part...
Formatting translation rules...
Converting cangjie-table.b5 to tsang-b5.el...done
Converting cangjie-table.b5 to quick-b5.el...
Converting cangjie-table.cns to tsang-cns.el...done
Converting cangjie-table.cns to quick-cns.el...
Converting cangjie-table.cns to quick-cns.el...done
  GEN      ../lisp/leim/ja-dic/ja-dic.el
Reading file "/home/yesudeep/code/remacs/leim/SKK-DIC/SKK-JISYO.L" ...
Converting cangjie-table.b5 to quick-b5.el...done

This is the part that is also extremely slow and takes forever:

Collecting OKURI-NASI entries...
Collecting OKURI-NASI entries...26%

[db48x]

The OKURI-NASI stuff is part of the process of building a Japanese dictionary. I'm surprised that it's slow, but I'm not surprised that -j4 doesn't help; this step isn't parallelizable. By the time this runs, at least the first stage Remacs binary has already been built; the Emacs Lisp code that builds this dictionary is running inside Remacs. It may well indicate a problem on your system, but I think it's more likely to indicate a problem with Remacs.

You can try to figure out what's going on by cancelling the build (just hit control-c), and running this command:

src/remacs -batch -l ja-dic-cnv -f batch-skkdic-convert -dir "list/leim/ja-dic" "leim/SKK-DIC/SKK-JISYO.L"

This command took 4 minutes to run on my machine. If you've got a way to profile it, use it.

[gorakhargosh]

Running just `make -j4' works pretty fast. Please feel free to close the bug. :)

[db48x]

No, the bug is still there. list/leim/ja-dic/ja-dic.el is not deleted by a make clean, and so if you cancel the build before it finishes that file, even running make clean will never cause it to be rebuilt. It'll only be rebuilt if it's deleted, or if the input file leim/SKK-DIC/SKK-JISYO.L is modified.

Because this is building a data file for a Japanese input method, I doubt that the incomplete file will ever cause you any trouble. Still, the speed problem should still be investigated and fixed.

[shaleh]

It would be nice if j8 and j16 worked correctly. I have seen them fail often enough I reverted by to j4.

[shaleh]

Should we close this and open a specific ticket for the ja-dic.el issue? Or should we rename/update this ticket?