run with lower priority (renice, ionice, nocache, etc)

[anarcat]

I run offlineimap in a terminal window, because I haven't found a good way (or the desire) to store the password on disk safely yet. I prefer to start it once and see it do its thing, because sometimes it fails in weird ways, and I like the visual feedback the blinkenlights interface gives me anyways.

So I start offlineimap by hand.

I have developped, over the years, a surprisingly long commandline to start offlineimap with lower priority. This is primarly due to the fact that I have a huge amount of mail I sync with it. To quote another software:

   Welcome to notmuch. You have 200 970 messages.

This takes about 10GB of disk space. Offlineimap is surprisingly fast at syncing all that data, probably because of special IMAP features, but it still is a major disk and CPU usage on my machine.

This is the commandline I am using:

nice ionice -c3 nocache offlineimap

What this does:

• nice lowers the priority of the process in the run queue, implemented using the simple nice(2) systemcall and available in Python/UNIX as os.nice
• ionice lowers the disk priority of the process and ensures that it doesn't trash the disk while something else is going on. -c3 is the "idle" priority. this is implemented using the ioprio_set syscall, specific to Linux, and available in Python 3.3 as os.setpriority or as psutil in previous versions
• nocache is similar, but ensures that we don't swap out valuable stuff from the page cache when running offlineimap. this is implemented using the posix_fadvise syscall and is available in Python 3.3 as os.posix_fadvise or as a third-party library

It would be nice if those system calls would be used (optionnally) lower the priority of offlineimap. Would a patch doing that be considered?

Thanks!

[nicolas33]

It would be nice if those system calls would be used (optionnally) lower the priority of offlineimap. Would a patch doing that be considered?

I tend to disagree. The more we can keep things out of offlineimap the cleaner the code remains. Also, the configuration file is quite long already.

Why not use a shell script or a shell alias to do the job? For my day to day use, I have a script shell called "emails" doing this work. This supports command line options for the accounts, tmux, background start on network connectivity, etc.

Notice that I regurlarly call the users to share their own scripts to help others so they don't have to start from scratch. We have a "./contrib" directory for this purpose in the official repository. Sadly, few users are contributing in this area and prefer to expose their scripts in their own blog. If "new" users don't spend time on searching the web, they don't know such scripts might already exist somewhere. We are fine with scripts with a description of the purpose and a reference to personal blog post.

On top of that we encourage users to write blog post about offlineimap in the official blog. Some did and I find they are very great contributions, if not the best posts. Users are usually good at talking about the tools they use. I'm pretty sure that once someone will insert a script, others will follow the move.

I've added a simple README in "./contrib" about that. I hope this will help.

[anarcat]

understood. my idea of a config setting was a single "nice" flag or something similar. but I understand where you are coming from and appreciate the effort you made in explaining it!

i am not sure what I would contribute, as a script - given it's a fairly simple thing, i wouldn't even know how to call it.

in all honesty, i should also mention that others pointed towards using systemd units to accomplish similar purposes, something that shouldn't be neglected as well. it seems there are no such settings right now in the systemd .service files provided in contrib, and i am not sure how those priorities can be set, but it could be the way forward here...