Index view refreshes and loses state/position

[l-mb]

Hi, thanks for the best email client ever! Since a few weeks to months - I'm sorry I don't know precisely when this started -, I noticed the following behaviour:

Actual Behaviour

I tend to view my folders in a limited and threaded view (showing unreads), such as ~d <3d ( ~N | ~F | ~O )

Without switching folders, neomutt will notice that new mail has arrived (check_new) and refresh the index view. The message(s) I've just read will now be filtered, meaning I'm losing my position in the thread/index.

I'm working around this by setting check_new to false.

Expected Behaviour

I'd like newly found mails to be merged into the current index view instead.

How often does this happen?

• Always

NeoMutt Version

NeoMutt 20200320
Copyright (C) 1996-2016 Michael R. Elkins and others.
NeoMutt comes with ABSOLUTELY NO WARRANTY; for details type 'neomutt -vv'.
NeoMutt is free software, and you are welcome to redistribute it
under certain conditions; type 'neomutt -vv' for details.

System: Linux 5.5.11-1-default (x86_64)
ncurses: ncurses 6.2.20200308 (compiled with 6.2.20200308)
libidn: 1.35 (compiled with 1.35)
GPGME: 1.13.1
GnuTLS: 3.6.12
libnotmuch: 5.2.0
hcache backends: kyotocabinet, lmdb

Configure options: --prefix=/usr --docdir=/usr/share/doc/packages/neomutt --with-mailpath=/var/mail --kyotocabinet --lua --lmdb --gnutls --gpgme --notmuch --sasl --gss --idn --mixmaster

Compilation CFLAGS: -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -std=c99 -D_ALL_SOURCE=1 -D_GNU_SOURCE=1 -D__EXTENSIONS__ -I/usr/include -I/usr/include/lua5.3 -DNCURSES_WIDECHAR

Default options:
  +attach_headers_color +compose_to_sender +compress +cond_date +debug 
  +encrypt_to_self +forgotten_attachments +forwref +ifdef +imap +index_color 
  +initials +limit_current_thread +multiple_fcc +nested_if +new_mail +nntp +pop 
  +progress +quasi_delete +regcomp +reply_with_xorig +sensible_browser +sidebar 
  +skip_quoted +smtp +status_color +timeout +tls_sni +trash 

Compile options:
  -autocrypt +bkgdset +color +curs_set +fcntl -flock -fmemopen +futimens 
  +getaddrinfo +gnutls +gpgme +gss +hcache -homespool +idn +inotify 
  -locales_hack +lua -lz4 +meta +mixmaster +nls +notmuch -openssl +pgp +sasl 
  +smime -sqlite +start_color +sun_attachment +typeahead -zlib -zstd 
MAILPATH="/var/mail"
MIXMASTER="mixmaster"
PKGDATADIR="/usr/share/neomutt"
SENDMAIL="/usr/sbin/sendmail"
SYSCONFDIR="/etc"

To learn more about NeoMutt, visit: https://neomutt.org
If you find a bug in NeoMutt, please raise an issue at:
    https://github.com/neomutt/neomutt/issues
or send an email to: <neomutt-devel@neomutt.org>

Extra Info

• Operating System and its version
  openSUSE Tumbleweed

• Were you using multiple copies of NeoMutt at once?
  No

• Were you using 'screen' or 'tmux'?
  No

• Is your email local (maildir) or remote (IMAP)?
  Local, maildir.

[scottkosty]

Do you sync when you have NeoMutt open using something like offlineimap?

[bjparent]

I noticed the same problem. I use imap to google. I noticed it first in
"NeoMutt 20200501-56-58e35d".
The prior version I was using without that problem was 20180716.

[michael-j-oconnor]

I noticed this behavior after a recent neomutt update, as well. I'm using IMAP to O365. I think I want the index to point me at the next message in the index, relative to the last message that I just operated on. It's not doing that consistently, and driving me nuts!

[flatcap]

First of all, sorry everyone. This is annoying.

But... there's a new fix for this 231ac26 (committed a few weeks ago).
Thanks @gahr ❤️

The next release won't be for a few weeks, at least, but the fix will be in it.

---

Technical bit

The design of the Mutt code that we inherited is a complete mess.
We're trying to fix this, but it's causing these side-effects.

The jumping cursor is a struggle between the frontend (GUI) and the backend (Mailbox).
When new mail arrives, for example, the backend sorts the Emails, adds the new ones, then tries to sort them back into the user's sort order.

The frontend, used to keep the index number of the 'current' email.
Now we track the Message-Id too.

This should make the cursor-tracking more reliable
(until we can fix the underlying problem).

[roccoblues]

Hi, I'm currently cleaning up some older issues here. Can you still reproduce this with the latest version? Thanks!

[rbmarliere]

I can confirm this is still an issue.

NeoMutt 20231103-135-f9bbd8
Copyright (C) 1996-2022 Michael R. Elkins and others.
NeoMutt comes with ABSOLUTELY NO WARRANTY; for details type 'neomutt -vv'.
NeoMutt is free software, and you are welcome to redistribute it
under certain conditions; type 'neomutt -vv' for details.

System: Linux 6.7.0-rc6-ktest-next-20231222+ (x86_64)
ncurses: ncurses 6.4.20231209 (compiled with 6.4.20231209)
libidn2: 2.3.4 (compiled with 2.3.4)
GPGME: 1.18.0
GnuTLS: 3.8.2
libnotmuch: 5.6.0
storage: tokyocabinet

Configure options: --gsasl --mandir=/usr/share/man --libexecdir=/usr/libexec --with-mailpath=/var/mail --gpgme --lua --notmuch --gnutls --gss --mixmaster --tokyocabinet --sqlite --autocrypt --locales-fix

Compilation CFLAGS: -fno-delete-null-pointer-checks -D_ALL_SOURCE=1 -D_GNU_SOURCE=1 -D__EXTENSIONS__ -D_XOPEN_SOURCE_EXTENDED -I/usr/include/lua5.4 -I/usr/include -I/usr/include -DNCURSES_WIDECHAR -I/usr/include -I/usr/include/p11-kit-1 -isystem /usr/include/mit-krb5 -O2

Compile options:
+autocrypt +fcntl -flock -fmemopen +futimens +getaddrinfo +gnutls +gpgme
+gsasl +gss +hcache -homespool +idn +inotify +locales_hack +lua +mixmaster
+nls +notmuch -openssl +pgp +regex -sasl +smime +sqlite +truecolor

MAILPATH="/var/mail"
MIXMASTER="mixmaster"
PKGDATADIR="/usr/share/neomutt"
SENDMAIL="/usr/sbin/sendmail"
SYSCONFDIR="/etc"

To learn more about NeoMutt, visit: https://neomutt.org
If you find a bug in NeoMutt, please raise an issue at:
https://github.com/neomutt/neomutt/issues
or send an email to: neomutt-devel@neomutt.org

[roccoblues]

I can confirm this is still an issue.

Damn. 😞 Thanks for the feedback!