Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

question concerning timers in lain #114

Closed
ff2000 opened this issue Jul 7, 2015 · 18 comments
Closed

question concerning timers in lain #114

ff2000 opened this issue Jul 7, 2015 · 18 comments

Comments

@ff2000
Copy link

ff2000 commented Jul 7, 2015

helpers.newtimer seems to not use stored timers efficiently. If I add more widgets of the same type (e.g. there are several batteries or sound cards or network devices etc) it creates a new timer and overwrites the previous one in the table (as I undertand LUA - which might be wrong). vicious indexes the timer table with the actual timeout which really seems to greatly affect wakeups and general CPU usage (0-1% in contrast to 2-3% with lain, both running a comparable set of widgets)
Furthermore vicious seems to efficiently cache formatted strings (don't get that part of the code entirely ATM, will have to stare at it some more time ;))

I think lain could take a similar approach to maximize performance/energy efficiency.

lcpz pushed a commit that referenced this issue Jul 9, 2015
lcpz pushed a commit that referenced this issue Jul 9, 2015
@lcpz
Copy link
Owner

lcpz commented Jul 9, 2015

You are right on bat, alsa and alsabar, but wrong on net: timer here is set with iface, which is unique.

I updated the following widgets in order to use unique ids for timer:

  • alsa
  • alsabar
  • bat
  • tpbat
  • fs
  • weather

but not for

  • cpu
  • mem
  • mpd
  • sysload
  • temp

because they're always instantiated once at most.

Test 6457c4d and tell me.

@ff2000
Copy link
Author

ff2000 commented Jul 12, 2015

Thx for the commits! I am away for a week and will test it when I'm back.
Though still this doesn't share timers like vicious: Setting the timeout for CPU, MEM, NET to 1 sec. will use only 1 timer not 3 like with lain. That way the awesome process wakes up only once. Don't you like that approach?

@lcpz
Copy link
Owner

lcpz commented Jul 12, 2015

So, not just unique ids, but also the same timeout for one-instance-only widgets, right?

Ok, but keep in mind that a user can change timeouts arbitrarily, so the general complexity is not affected.

Can you tell me how do you precisely test this?

@lcpz
Copy link
Owner

lcpz commented Jul 12, 2015

Plus, please note that imap, mpd and weather are asynchronous, so if you're using them too, of course your config is a little more cpu demanding.

@lcpz
Copy link
Owner

lcpz commented Jul 27, 2015

@ff2000 Can you update me on this? Also, tell me how do you do your tests.

@ff2000
Copy link
Author

ff2000 commented Jul 27, 2015

The comparison between vicious and lain probably was not that great. As I said I took a similar set of widgets and the same timeouts and compared the resulting CPU usage. Of course the widgets (and the logic behind them) is different. I would need to write my own widgets and adopt them to both APIs.
But I can at least get a vicious-like implementation in lain if I simply change lain/helpers.lua:

diff --git a/helpers.lua b/helpers.lua
index dbee617..9e0c27f 100644
--- a/helpers.lua
+++ b/helpers.lua
@@ -75,7 +75,8 @@ end

 helpers.timer_table = {}

-function helpers.newtimer(name, timeout, fun, nostart)
+function helpers.newtimer(_name, timeout, fun, nostart)
+    local name = timeout
     helpers.timer_table[name] = capi.timer({ timeout = timeout })
     helpers.timer_table[name]:connect_signal("timeout", fun)
     helpers.timer_table[name]:start()

(and a change in rc.lua.holo to use the same timeout for all the widgets)
Doing this gave me a drop of 1-2% in CPU-usage when I tried the last time. Now (within Xephyr and the most recent awesome-copycats) I do not see this anymore :( Probably because asyncshell fires errors about asyncshell.deliver not being able to index a nil value (due to implementation: It pipes the command to awesome-client, the call will end up in my actual desktop instance of awesome which currently does not use asyncshell)...

@lcpz
Copy link
Owner

lcpz commented Jul 27, 2015

I don't get it. Why name timer_table indexes after timeout? Doesn't this overwrite previous table entries?

@ff2000
Copy link
Author

ff2000 commented Jul 27, 2015

Damn, sorry :/ Was in a hurry and posted a change I do not actually run. (Fresh git checkout vs. copy in ~/.config/awesome).

Here is the fixed up setup:
~/.xinitrc

mv ~/.xsession-errors ~/.xsession-errors.old

xrdb -merge ~/.Xresources
xmodmap ~/.xmodmap

compton --config ~/.config/compton.conf 2>> ~/.xsession-errors &

export AWCFG_HOME="/home/franz/src/awesome-configs/copycat-killer"
export LUA_PATH="$AWCFG_HOME/?/init.lua;$AWCFG_HOME/?.lua;;"

exec dbus-launch awesome -c /home/franz/src/awesome-configs/copycat-killer/rc_wrap.lua 2>> ~/.xsession-errors

rc_wrap.lua

local awesome = require("awesome")
local awful = require("awful")
local lfs = require("lfs")
orig_getdir = awful.util.getdir

awful.util.getdir = function(d)
  print(d)
  print(awesome.conffile)
  print(debug.getinfo(1).source)
  print(lfs.currentdir())
  if d == "config" then
    -- return debug.getinfo(1).source:match("(.*/)")
    local path = ""
    if awesome.conffile:sub(1,1) == '/' then
      path = awesome.conffile
    else
      path = lfs.currentdir() .. "/" .. awesome.conffile
    end
    return path:match("(.*/)")
  else
    return orig_getdir(d)
  end
end


dofile(awful.util.getdir("config") .. "/rc.lua.holo.mine")

The actual change in helpers.lua:

diff --git a/helpers.lua b/helpers.lua
index dbee617..d3f1db5 100644
--- a/helpers.lua
+++ b/helpers.lua
@@ -75,8 +75,11 @@ end

 helpers.timer_table = {}

-function helpers.newtimer(name, timeout, fun, nostart)
-    helpers.timer_table[name] = capi.timer({ timeout = timeout })
+function helpers.newtimer(_name, timeout, fun, nostart)
+    local name = timeout
+    if not helpers.timer_table[name] then
+        helpers.timer_table[name] = capi.timer({ timeout = timeout })
+    end
     helpers.timer_table[name]:connect_signal("timeout", fun)
     helpers.timer_table[name]:start()
     if not nostart then

And the diff to the original rc.lua.holo

diff --git a/rc.lua.holo b/rc.lua.holo.mine
index 534dc89..bd35f92 100644
--- a/rc.lua.holo
+++ b/rc.lua.holo.mine
@@ -55,7 +55,7 @@ run_once("unclutter -root")
 -- {{{ Variable definitions

 -- beautiful init
-beautiful.init(os.getenv("HOME") .. "/.config/awesome/themes/holo/theme.lua")
+beautiful.init(awful.util.getdir("config") .. "/themes/holo/theme.lua")

 -- common
 modkey     = "Mod4"
@@ -106,6 +106,7 @@ mylauncher = awful.widget.launcher({ image = beautiful.awesome_icon,
 -- }}}

 -- {{{ Wibox
+local ltm = 1
 markup = lain.util.markup
 blue   = "#80CCE6"
 space3 = markup.font("Tamsyn 3", " ")
@@ -165,6 +166,7 @@ play_pause_icon = wibox.widget.imagebox()
 play_pause_icon:set_image(beautiful.play)

 mpdwidget = lain.widgets.mpd({
+    timeout = ltm,
     settings = function ()
         if mpd_now.state == "play" then
             mpd_now.artist = mpd_now.artist:upper():gsub("&.-;", string.lower)
@@ -219,6 +221,7 @@ end)))

 -- Battery
 batwidget = lain.widgets.bat({
+    timeout = ltm,
     settings = function()
         bat_header = " Bat "
         bat_p      = bat_now.perc .. " "
@@ -234,6 +237,7 @@ batwidget = lain.widgets.bat({

 -- ALSA volume bar
 myvolumebar = lain.widgets.alsabar({
+    timeout = ltm,
     card   = "0",
     ticks  = true,
     width  = 80,
@@ -258,6 +262,7 @@ volumewidget:set_bgimage(beautiful.widget_bg)

 -- CPU
 cpu_widget = lain.widgets.cpu({
+    timeout = ltm,
     settings = function()
         widget:set_markup(space3 .. "CPU " .. cpu_now.usage
                           .. "%" .. markup.font("Tamsyn 5", " "))
@@ -275,6 +280,7 @@ netdown_icon:set_image(beautiful.net_down)
 netup_icon = wibox.widget.imagebox()
 netup_icon:set_image(beautiful.net_up)
 netwidget = lain.widgets.net({
+    timeout = ltm,
     settings = function()
         widget:set_markup(markup.font("Tamsyn 1", " ") .. net_now.received .. " - "
                           .. net_now.sent .. space2)

And with this setup I get an idle cpu usage (read from the cpu widget) from constant 3%. With the change to helpers.lua reverted it gets up to 4%.
Do you have a better tool to measure performance?

But I also see the issue with timeout changes. Might be interesting to change CPU/Disc-intensive widgets to bigger timeouts when the laptop is on battery. So it could really be your approach is better...

@everslick
Copy link
Contributor

I can confirm that the timer sharing patch does indeed improve performance. I can see it in top as well as with the following measurement program:

//Last modified: 18/11/12 19:13:35(CET) by Fabian Holler
#include <stdlib.h> 
#include <sys/types.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>

struct pstat {
    long unsigned int utime_ticks;
    long int cutime_ticks;
    long unsigned int stime_ticks;
    long int cstime_ticks;
    long unsigned int vsize; // virtual memory size in bytes
    long unsigned int rss; //Resident  Set  Size in bytes

    long unsigned int cpu_total_time;
};

/*
 * read /proc data into the passed struct pstat
 * returns 0 on success, -1 on error
*/
int get_usage(const pid_t pid, struct pstat* result) {
    //convert  pid to string
    char pid_s[20];
    snprintf(pid_s, sizeof(pid_s), "%d", pid);
    char stat_filepath[30] = "/proc/"; strncat(stat_filepath, pid_s,
            sizeof(stat_filepath) - strlen(stat_filepath) -1);
    strncat(stat_filepath, "/stat", sizeof(stat_filepath) -
            strlen(stat_filepath) -1);

    FILE *fpstat = fopen(stat_filepath, "r");
    if (fpstat == NULL) {
        perror("FOPEN ERROR ");
        return -1;
    }
    FILE *fstat = fopen("/proc/stat", "r");
    if (fstat == NULL) {
        perror("FOPEN ERROR ");
        fclose(fstat);
        return -1;
    }

    //read values from /proc/pid/stat
    bzero(result, sizeof(struct pstat));
    long int rss;
    if (fscanf(fpstat, "%*d %*s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %lu"
                "%lu %ld %ld %*d %*d %*d %*d %*u %lu %ld",
                &result->utime_ticks, &result->stime_ticks,
                &result->cutime_ticks, &result->cstime_ticks, &result->vsize,
                &rss) == EOF) {
        fclose(fpstat);
        return -1;
    }
    fclose(fpstat);
    result->rss = rss * getpagesize();

    //read+calc cpu total time from /proc/stat
    long unsigned int cpu_time[10];
    bzero(cpu_time, sizeof(cpu_time));
    if (fscanf(fstat, "%*s %lu %lu %lu %lu %lu %lu %lu %lu %lu %lu",
                &cpu_time[0], &cpu_time[1], &cpu_time[2], &cpu_time[3],
                &cpu_time[4], &cpu_time[5], &cpu_time[6], &cpu_time[7],
                &cpu_time[8], &cpu_time[9]) == EOF) {
        fclose(fstat);
        return -1;
    }

    fclose(fstat);

    for(int i=0; i < 10;i++)
        result->cpu_total_time += cpu_time[i];

    return 0;
}

/*
* calculates the elapsed CPU usage between 2 measuring points. in percent
*/
void calc_cpu_usage_pct(const struct pstat* cur_usage,
                        const struct pstat* last_usage,
                        double* ucpu_usage, double* scpu_usage)
{
    const long unsigned int total_time_diff = cur_usage->cpu_total_time -
                                              last_usage->cpu_total_time;

    *ucpu_usage = 100 * (((cur_usage->utime_ticks + cur_usage->cutime_ticks)
                    - (last_usage->utime_ticks + last_usage->cutime_ticks))
                    / (double) total_time_diff);

    *scpu_usage = 100 * ((((cur_usage->stime_ticks + cur_usage->cstime_ticks)
                    - (last_usage->stime_ticks + last_usage->cstime_ticks))) /
                    (double) total_time_diff);
}

/*
* calculates the elapsed CPU usage between 2 measuring points in ticks
*/
void calc_cpu_usage(const struct pstat* cur_usage,
                    const struct pstat* last_usage,
                    long unsigned int* ucpu_usage,
                    long unsigned int* scpu_usage)
{

    *ucpu_usage = (cur_usage->utime_ticks + cur_usage->cutime_ticks) -
                  (last_usage->utime_ticks + last_usage->cutime_ticks);

    *scpu_usage = (cur_usage->stime_ticks + cur_usage->cstime_ticks) -
                  (last_usage->stime_ticks + last_usage->cstime_ticks);
}

int main(int argc, char **argv) {
    pid_t pid = 1;
    int slp = 10;
    int it = 1;

    if (argc >= 2) {
        pid = atoi(argv[1]);
    }

    if (argc >= 3) {
        slp = atoi(argv[2]);
    }

    if (argc >= 4) {
        it = atoi(argv[3]);
    }

    if (it == -1) it = INT_MAX;

    for (int i=0; i<it; i++) {
        struct pstat stat1, stat2;
        long unsigned int user_ticks, system_ticks;
        double user_pct, system_pct;

        get_usage(pid, &stat1);
        sleep(slp);
        get_usage(pid, &stat2);

        calc_cpu_usage(&stat2, &stat1, &user_ticks, &system_ticks);
        calc_cpu_usage_pct(&stat2, &stat1, &user_pct, &system_pct);

        printf("  Ticks:\tuser = %lu\tsystem = %lu\tsum = %lu\n",
                user_ticks, system_ticks, user_ticks + system_ticks);
        printf("Percent:\tuser = %.1f\tsystem = %.1f\tsum = %.1f\n\n",
                user_pct, system_pct, user_pct + system_pct);
    }

    return (0);
}

compile: gcc -o ticks ticks.c
run: ./ticks $(pgrep awesome) 20 3

This makes 3 measurements of each 20 seconds

Best used with short timeouts in the widgets and everything else idling.

@ff2000
Copy link
Author

ff2000 commented Aug 30, 2015

In order to preserve the possibility to change timers for certain widgets one could add a way so that the user wants to share timers. E.G. a property "use_shared_timers" in lain.helpers table. If set to true helpers.newtimer will index with timeout-id, otherwise it uses the passed arg "name".
Another possibility could be to add a "timer_name" value to each widget. If it is nil (default) use the default name (as it is done now). If it is a non-empty string use that as key, if it is an emtpy string use the timeout as key.
The first possibility would be quite easy to implement, as it doesn't touch every single widget, and new widgets wouldn't have to add yet more code... But the second one is more flexible...
(A third option would be to store the timer_name in helpers. IMHO not really a good solution, as it makes things less obvious, config might be borked without the user noticing it, and the whole intention of the effort (better performance through optional timer sharing) might be for the birds...)

@everslick
Copy link
Contributor

I improved a little bit on the test program. I put it here for reference, and in case it is helpful for lain performance testing. I edited my previous comment accordingly.

@lcpz
Copy link
Owner

lcpz commented Sep 4, 2015

I improved a little bit on the test program. I put it here for reference, and in case it is helpful for lain performance testing.

I'm a bit rusty in C, after adding stdio, stdlib and limits libraries to your code, I get these errors and warnings while compiling.

ticks.c: In functionmain’:
ticks.c:25:22: error: storage size ofstat1isnt known
         struct pstat stat1, stat2;
                      ^
ticks.c:25:29: error: storage size ofstat2isnt known
         struct pstat stat1, stat2;
                             ^
ticks.c:29:9: warning: implicit declaration of functionget_usage’ [-Wimplicit-function-declaration]
         get_usage(pid, &stat1);
         ^
ticks.c:30:9: warning: implicit declaration of functionsleep’ [-Wimplicit-function-declaration]
         sleep(slp);
         ^
ticks.c:33:9: warning: implicit declaration of functioncalc_cpu_usage’ [-Wimplicit-function-declaration]
         calc_cpu_usage(&stat2, &stat1, &user_ticks, &system_ticks);
         ^
ticks.c:34:9: warning: implicit declaration of functioncalc_cpu_usage_pct’ [-Wimplicit-function-declaration]
         calc_cpu_usage_pct(&stat2, &stat1, &user_pct, &system_pct);

How do I fix?

@everslick
Copy link
Contributor

You need to combine the two listings I posted. :)

To avoid any further confusion, I updated the original comment with my improvements.

@everslick
Copy link
Contributor

what is your conclusion on that? i'd vote for as much timer sharing as
possible. :-)

On Tue, Sep 15, 2015 at 9:24 AM, Luke Bonham notifications@github.com
wrote:

For convenience, I created a gist
https://gist.github.com/copycat-killer/277e6cb8468d6d1bf958.

I think I'll close this one within the week end.


Reply to this email directly or view it on GitHub
#114 (comment)
.

lcpz pushed a commit that referenced this issue Oct 12, 2015
@lcpz
Copy link
Owner

lcpz commented Oct 12, 2015

Can you please test 912bd26?

It has the proposed patches extended to all widgets. I hope it's ok.

@lcpz
Copy link
Owner

lcpz commented Oct 13, 2015

I tried latest master, but I am experiencing a heavier cpu load.

This is a current ticks output:

 ./ticks $(pgrep awesome) 20 3

  Ticks:    user = 106  system = 41 sum = 147
Percent:    user = 1.3  system = 0.5    sum = 1.8

  Ticks:    user = 102  system = 37 sum = 139
Percent:    user = 1.3  system = 0.5    sum = 1.7

  Ticks:    user = 111  system = 39 sum = 150
Percent:    user = 1.4  system = 0.5    sum = 1.9

While this is one with a previous commit:

 ./ticks $(pgrep awesome) 20 3

  Ticks:    user = 94   system = 11 sum = 105
Percent:    user = 1.2  system = 0.1    sum = 1.3

  Ticks:    user = 94   system = 10 sum = 104
Percent:    user = 1.2  system = 0.1    sum = 1.3

  Ticks:    user = 104  system = 14 sum = 118
Percent:    user = 1.3  system = 0.2    sum = 1.5

I don't know if this is related to the fact that I used maps instead of locally instantiating things like @everslick did in it's pull request. I'll investigate further.

@lcpz
Copy link
Owner

lcpz commented Oct 13, 2015

Moved to enhancement_tests branch.

lcpz pushed a commit that referenced this issue Mar 2, 2016
@lcpz
Copy link
Owner

lcpz commented Mar 2, 2016

Testing timer sharing patch:

diff --git a/helpers.lua b/helpers.lua
index dbee617..d3f1db5 100644
--- a/helpers.lua
+++ b/helpers.lua
@@ -75,8 +75,11 @@ end

 helpers.timer_table = {}

-function helpers.newtimer(name, timeout, fun, nostart)
-    helpers.timer_table[name] = capi.timer({ timeout = timeout })
+function helpers.newtimer(_name, timeout, fun, nostart)
+    local name = timeout
+    if not helpers.timer_table[name] then
+        helpers.timer_table[name] = capi.timer({ timeout = timeout })
+    end
     helpers.timer_table[name]:connect_signal("timeout", fun)
     helpers.timer_table[name]:start()

Test 1

System: idle
Theme: multicolor
Timeout: 1 second for cpu, coretemp, battery, alsa, net, mem, mpd; default for remaining widgets

With timer sharing

$ ./ticks $(pgrep awesome) 20 3
  Ticks:    user = 43   system = 10 sum = 53
Percent:    user = 0.5  system = 0.1    sum = 0.7

  Ticks:    user = 42   system = 10 sum = 52
Percent:    user = 0.5  system = 0.1    sum = 0.7

  Ticks:    user = 40   system = 8  sum = 48
Percent:    user = 0.5  system = 0.1    sum = 0.6

Without timer sharing

$ ./ticks $(pgrep awesome) 20 3
  Ticks:    user = 54   system = 7  sum = 61
Percent:    user = 0.7  system = 0.1    sum = 0.8

  Ticks:    user = 55   system = 7  sum = 62
Percent:    user = 0.7  system = 0.1    sum = 0.8

  Ticks:    user = 56   system = 5  sum = 61
Percent:    user = 0.7  system = 0.1    sum = 0.8

Test 2

System: idle
Theme: multicolor
Timeout: default for every widgets

With timer sharing

$ ./ticks $(pgrep awesome) 20 3
  Ticks:    user = 31   system = 2  sum = 33
Percent:    user = 0.4  system = 0.0    sum = 0.4

  Ticks:    user = 15   system = 1  sum = 16
Percent:    user = 0.2  system = 0.0    sum = 0.2

  Ticks:    user = 24   system = 4  sum = 28
Percent:    user = 0.3  system = 0.1    sum = 0.4

Without timer sharing

$ ./ticks $(pgrep awesome) 20 3
  Ticks:    user = 43   system = 11 sum = 54
Percent:    user = 0.5  system = 0.1    sum = 0.7

  Ticks:    user = 48   system = 6  sum = 54
Percent:    user = 0.6  system = 0.1    sum = 0.7

  Ticks:    user = 65   system = 8  sum = 73
Percent:    user = 0.8  system = 0.1    sum = 0.9

Test 3

Using everslick fork at commit 54b6aba
System: idle
Theme: multicolor
Timeout: 1 second for cpu, coretemp, battery, alsa, net, mem, mpd; default for remaining widgets

  Ticks:    user = 57   system = 10 sum = 67
Percent:    user = 0.7  system = 0.1    sum = 0.8

  Ticks:    user = 38   system = 13 sum = 51
Percent:    user = 0.5  system = 0.2    sum = 0.6

  Ticks:    user = 38   system = 7  sum = 45
Percent:    user = 0.5  system = 0.1    sum = 0.6

Test 4

Using everslick fork at commit 54b6aba
System: idle
Theme: multicolor
Timeout: default for every widgets

  Ticks:    user = 20   system = 3  sum = 23
Percent:    user = 0.3  system = 0.0    sum = 0.3

  Ticks:    user = 28   system = 6  sum = 34
Percent:    user = 0.4  system = 0.1    sum = 0.4

  Ticks:    user = 25   system = 4  sum = 29
Percent:    user = 0.3  system = 0.1    sum = 0.4

Result

I'll use timer sharing with default timeouts.

@lcpz lcpz closed this as completed Mar 2, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants