Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lagging with ivy-mode #1571

Closed
seagle0128 opened this issue May 14, 2018 · 20 comments

Comments

@seagle0128
Copy link
Contributor

commented May 14, 2018

I am using GNU Emacs 25.3 on terminal of Ubuntu 16.04. After a few days, the emacs is lagging when do some operations with ivy, swiper or counsel. If I restart the Emacs daemon, the issue disappeared and occurs again after a few days. My config is: https://github.com/seagle0128/.emacs.d.

I did the profiling and attach the screenshot here. It seems ivy--add-face caused high CPU.

emacs_hang

@basil-conto

This comment has been minimized.

Copy link
Collaborator

commented May 14, 2018

After a few days, the emacs is lagging when do some operations with ivy, swiper or counsel.

Only these packages?

If I restart the Emacs daemon, the issue disappeared and occurs again after a few days.

I wonder if it's an issue with increasing non-garbage-collectible memory usage, more strings, symbols, etc. I'm not sure, though.

@abo-abo

This comment has been minimized.

Copy link
Owner

commented May 14, 2018

I wonder if it's an issue with increasing non-garbage-collectible memory usage, more strings, symbols, etc. I'm not sure, though

Me too. I wonder how can Ivy produce strings that aren't garbage-collectible. There's no global var that keeps references to the strings, and no circular references.

@seagle0128

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2018

I didn't reproduced with emacs -Q with ivy. Actually my college and I encountered this issue with my config https://github.com/seagle0128/.emacs.d. The issue only occurs after a few days, and was observed in daemon mode. Not sure for GUI mode. In my experience, the issue disappeared once I disabled ivy-mode and counsel-mode. Occurs again if enable. Another workaround is restarting Emacs daemon.

@basil-conto

This comment has been minimized.

Copy link
Collaborator

commented May 15, 2018

The issue only occurs after a few days, and was observed in daemon mode. Not sure for GUI mode.

Just to be clear, when you say "daemon mode", are you referring to a daemon Emacs session, to Emacs running in the TTY, or both? Because daemon and GUI are not mutually exclusive.

@seagle0128

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2018

@basil-conto I am running Emacs in terminal via SSH, with server-start in init.el. Generally emacsclient is used to open the files. Hope it's clear.

@c0001

This comment has been minimized.

Copy link

commented Jun 15, 2018

I found this problem too, also be referring to ivy--add-face, and set the large garbage-collection counts.

I follow the advice by :

I wonder if it's an issue with increasing non-garbage-collectible memory usage, more strings, symbols, etc. I'm not sure, though

And waiting for testing the truth whether about this refer.

emacs-version: 26.1
operatioin-system: win10-18030

Usage-like: GUI emacs

@basil-conto

This comment has been minimized.

Copy link
Collaborator

commented Jun 15, 2018

I wonder how can Ivy produce strings that aren't garbage-collectible. There's no global var that keeps references to the strings, and no circular references.

I was mostly speculating, but possibilities include ivy-last, increasingly large obarray and history lists, and memory leaks, no?

@c0001

This comment has been minimized.

Copy link

commented Jun 16, 2018

After reducing the garbage-collection without any effects for solving this problem.

Just refer to the main topic question of this issue, I think the core direction was not whether is daemon or not, because it was found on daily using in GUI side without tramp and any other server reference, just daily usage problem.

So this may the bug by the underlying implementation mechanism of ivy.

I was mostly speculating, but possibilities include ivy-last, increasingly large obarray and history lists, and memory leaks, no?

Above studying may the cause which I thought too.

@abo-abo

This comment has been minimized.

Copy link
Owner

commented Jun 18, 2018

I was mostly speculating, but possibilities include

ivy-last

This gets overwritten by the new ivy-last on each completion. The last one is garbage-collected.

increasingly large obarray

Might be worth looking into. I have (length obarray) at 15121 now.

history lists

Probably not an issue.

and memory leaks

If there's a memory leak, it's an Emacs issue that we should report. As far as I understand, there's nothing Elisp can do to cause a memory leak.

@basil-conto

This comment has been minimized.

Copy link
Collaborator

commented Jun 18, 2018

This gets overwritten by the new ivy-last on each completion. The last one is garbage-collected.

Sure, but each ivy-last value is still able to hog a decent amount of memory, which is what I was listing (though I didn't make this clear in my previous comment).

I have (length obarray) at 15121 now.

But AIUI each obarray slot can hold multiple interned symbols, so is the overall length of the obarray vector representative enough?

As far as I understand, there's nothing Elisp can do to cause a memory leak.

This is also my understanding, though Elisp can still be written in a leak-y way, e.g. by leaving lots of live markers lying around.

Anyway, the point is that we can't do much but speculate without more CPU/memory profiling of Ivy and Emacs itself.

@c0001

This comment has been minimized.

Copy link

commented Jun 26, 2018

Hi maintainer:

Sorry for the long time testing and nothing to feedback!

In the begining, what I thought was the binary-code version cross different emacs-version using caused this problem and with the mechanism of ivy also.

The experience for this debugging of above was takes for about 5 days, of course through limitting the adapted binary-code file for each emacs-version independently but problem occur again tonight. It almostly destroyed the native core stable emotion in my-mind, and then with the attempt for checking the raw-code of ivy, I found follow suspicious code-block:

(defun ivy--add-face (str face)
  "Propertize STR with FACE.
`font-lock-append-text-property' is used, since it's better than
`propertize' or `add-face-text-property' in this case."
  (require 'colir) ;;=============>***NOTICE THIS ONE****
  (condition-case nil
      (progn
        (colir-blend-face-background 0 (length str) face str)
        (let ((foreground (face-foreground face)))
          (when foreground
            (add-face-text-property
             0 (length str)
             `(:foreground ,foreground)
             nil
             str))))
    (error
     (ignore-errors
       (font-lock-append-text-property 0 (length str) 'face face str))))
  str)

With the speculation for this (require 'colir), I adding 'adviced' to ivy--add-face with 'override' optioin for commented this line in the current session (which have the lagging performance now, not to restart emacs), everything has gone!!! GOD!!!!!! TKS.

So the main hole of this issue may be this one , why use 'require' in the frequency-used funcition? I don't understand (I'm the newbie for emacs-lisp).

If this was the trigger , hope for fix it in the next gen or not : could you anwser me that why be that?

Thanks a lot for creating ivy --- the best completion tool in emacs for me 😸 💯 .

@abo-abo

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2018

why use 'require' in the frequency-used function?

Indeed, calls to require aren't free. And it does call Fcons. I'll move the require outside.

Thanks for your work on investigating the source of the bug.

abo-abo added a commit that referenced this issue Jun 26, 2018
ivy.el (ivy--add-face): Move require colir to top-level
`ivy--add-face' is called frequently, and calls to `require' aren't
free.

Moreover, this bit in the source code of `require' looks troublesome:

    tem = Fcons (Qrequire, feature);

Re #1571
@abo-abo

This comment has been minimized.

Copy link
Owner

commented Jun 26, 2018

@c0001 The require is now moved outside, you can remove your advice. Please test for a few days and let us know if the issue is gone.

@c0001

This comment has been minimized.

Copy link

commented Jun 27, 2018

@abo-abo I'd cloned swiper repo for further testing this minor improvement.

According to the cycle of previous testing experience, it may used 5 days with none-closed emacs session for this testing.

Thanks for replying and I will submit the newest report here after testing again.

@c0001

This comment has been minimized.

Copy link

commented Jun 30, 2018

@abo-abo I'd test fiew days and the bug haven't occurred again til now.

@basil-conto

This comment has been minimized.

Copy link
Collaborator

commented Jul 1, 2018

If the problem really does lie with repeated calls to require, I'd M-xreport-emacs-bugRET.

@c0001

This comment has been minimized.

Copy link

commented Jul 1, 2018

If the problem really does lie with repeated calls to require, I'd M-x report-emacs-bug RET.

Did it mean that calling require repeated will not cause this in the original design concept ? If indeed for this aspect, the bug was long time remaining and without paid attention by Emacs developer organization , reporting this to them was necessary.

Thanks for helping to deal with this headache like problem for me 😺 .

@basil-conto

This comment has been minimized.

Copy link
Collaborator

commented Jul 1, 2018

Did it mean that calling require repeated will not cause this in the original design concept ?

I'm not advocating calling require repeatedly and in a tight loop, but ideally nothing should lead to such lagging / memory consumption, particularly not something as basic as require.

@abo-abo

This comment has been minimized.

Copy link
Owner

commented Jul 3, 2018

@basil-conto Agreed about the memory leak. But still, the code of require has to have this code:

tem = Fmemq (feature, Vfeatures);

And I have this on my system:

(length features)
;; => 667

We don't want to call memq on a large list in a loop. And if require would be implemented via hash table lookup (which is faster than lists for collections larger than 100 items), we would still probably want to avoid that in a loop.

@abo-abo

This comment has been minimized.

Copy link
Owner

commented Sep 6, 2018

It seems like this is fixed now, so I'll close it. If it's still an issue, please create a new ticket referencing this one.

@abo-abo abo-abo closed this Sep 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.