Commits on Mar 18, 2014
  1. [u_proc] Add u_proc.need_update field

    gajdusek committed Mar 18, 2014
    Mask of properties that may be invalid and need to be updated. These
    are set if process called a function from `exec()` family and used by
    update was not requested by the caller.
    process_new_delay() does no more update cmdline and exe, but marks
    them as
  2. [u_proc] Change and rename ENSURE_WHAT enum to be usable as a mask

    gajdusek committed Mar 18, 2014
    It no more enumerates 'tags' (BASIC, CGROUP, ...) as a sequence of
    integers but rather as binary flags. So the enumeration may be now
    used as a mask of properties; and I renamed the enum to
    u_proc.ensured, which was structure of booleans, is replaced with
    guint mask of U_PROC_PROPERTIES.
    Structure u_proc_ensured is removed.
  3. [LUA] ulatency.list_processes(): Exclude vanished and zombie processes

    gajdusek committed Mar 18, 2014
    This could make the scheduler iterations a bit faster and should cause
    no unexpected side-effects, as vanished process were already removed
    from `processes` table before filters and scheduler were called.
  4. [linux_netlink] Increase socket timeout from 10s to 60s

    gajdusek committed Mar 18, 2014
    10 seconds is not enough to netlink being initialized on slow
    computers, as the ulatencyd startup may take longer. Actually 10s
    is too short for my computer if running ulatencyd DEVELOP_MODE and
    observing detailed logs on slow terminal.
    Another issue is running ulatencyd under valgrind, in this case even
    60s may not be enough.
    I hope this will not introduce any clashes with netlink_fallback
    module, unless it will be unloaded somewhat later.
  5. [core] Print critical warning if process tree is broken

    gajdusek committed Mar 18, 2014
    Log critical warning each time rebuild_tree() is invoked, as such
    errors should be closely examined. Maybe I have missed something, but
    couldn't found out how this may happen.
  6. [u_proc] Rework u_proc_ensure() and U_PROC_STAT stuff

    gajdusek committed Mar 18, 2014
    Reasoning behind this is to make assumptions about processes state
    more precise; it allows us to distinguish unexpected errors, while
    manipulating with processes or cgroups, from expected ones (e.g.
    errors caused by process vanished, becomes zombies or it's a kernel
    process). Expected errors should be correctly and quietly handled,
    unexpected errors should trigger critical warning, as these are signs
    of some ulatencyd assumptions are simply wrong or tests are not
    enough precise.
    == U_PROC_STATE enum
    Add UPROC_VANISHED state:
            Added to processes we are sure they vanished from /proc/
    Add UPROC_ZOMBIE state:
            Added to processes we are sure are zombies (or vanished).
    Add UPROC_KERNEL state:
            Tracks kernel processes. In rare cases, this state may be set
            to user-space vanished or zombie processes (only root process
            But it is guaranteed that every kernel process will have it.
            This way rules may distinguish kernel processes much more
            reliable than testing vm_size == 0, which would include
            Mask of vanished and zombie processes
    == sysinfo.c
    Stop using g_file_get_content(), functions use u_pid_read_file()
    u_read_0file() renamed to u_pid_read_0file()
    == u_proc_ensure() stuff
    Rework ENSURE_UPDATE enum:
    	do not update fields
    	update conditions are selected according the field type
    	fields are updated unless already set and unless another
    	attempt to update them occurred since `/proc/<PID>/`
    	directory was parsed last time
    	update fields unless an attempt to update them occurred since
    	`/proc/<PID>/` directory was parsed last time, i.e. since the
    	last time the process was passed to #update_processes_run()
    	update fields now
    It's behaviour is slightly changed: You may ensure if the set of
    properties is set and say whether (on what circumstances) they should
    be updated. But you cannot elaborate if they actually were updated or
    if the update was successful; only if the properties are set. If
    the properties should be updated but weren't, because the process
    died, became a zombie, or it is actually a kernel process you may test
    u_proc states which are always set by this function if some expected
    error occurred. If some other (unexpected) error occurred, there is
    nothing you can do with it anyway, user has been already informed by
    critical warning, and the code should continue if older values of
    properties are available (u_proce_ensure returned TRUE) or give up
    if not (u_proc_encuse returned FALSE).
    	- Each set of properties is handled in separate function:
                + u_proc_update_basic()
                + u_proc_update_environment()
                + u_proc_update_cgroup()
                + u_proc_update_cmdline()
                + u_proc_update_exe()
    	- Doesn't loose time on kernel, vanished or zombie processes,
    	  if it is useless.
    	- Doesn't erase old values of properties if update fails,
    	  process vanished etc. Therefore once u_proc_ensure()
    	  returns TRUE for some set of properties, there will stay
    	  available forever (until the u_proc instance is destroyed).
    	- If the set of properties is available, always returns TRUE;
    	  FALSE is returned only if properties are not set (i.e. were
    	  __never__ successfully updated)
    	- To read contents of files inside /proc/<pid>/ directory,
     	  u_pid_read() is used.
  7. [u_proc] Make U_PROC_*_STATE macros more robust

    gajdusek committed Mar 18, 2014
    Allows to set/unset/test multiple states efficiently with single
    U_PROC_*_STATE() macro usage, which would be more likely merged
    into single condition by compiler.
    you can use `U_PROC_HAS_STATE (proc, state1 | state2) == state1`
    to test if a proc has state1 but not state2;
    instead of `proc->ustate & (state1 | state2) == state1` or
    `U_PROC_HAS_STATE(proc, state1) && !U_PROC_HAS_STATE(proc, state2)`
    This also allows to define and use state masks.
  8. [u_proc] Add U_PROC_FORMAT and U_PROC_FORMAT_ARGS(P) macros

    gajdusek committed Mar 18, 2014
    Which may be used e.g. for log messages containing u_proc instance.
    g_warning("Process " U_PROC_FORMAT " should not exist!", U_PROC_FORMAT_ARGS(proc));
  9. [core] Add u_pid_read_file() which will replace g_file_get_contents()

    gajdusek committed Mar 18, 2014
    g_file_get_contents() is not suitable for reading files in procfs:
    Major g_file_get_contents() issue:
    - GFileError enumeration misses value corresponding to errno ESRCH,
      which we need to determine whether process vanished.
    Minor g_file_get_contents() issues:
    - It uses fread <stdio.h> family instead of read syscall <unistd.h>,
      if the file has 0 bytes length reported from `stat` (every file in
      procfs, sysfs, cgroupfs etc.) which adds unnecessary layer.
    - It may allocate unnecessary much memory.
    - We often need to distinguish between an empty file and a file
      containing only null byte ('\0'). If there wasn't an error,
      g_file_get_contents() _always_ allocates string (and adds null
      byte), therefore we must track returned length value and free the
      string if not needed, which both is uncomfortable for us.
      u_pid_read_file() simply returns NULL and sets errno to EEXIST if
      the file was empty.
  10. [u_proc] Remove UPROC_NEW and UPROC_ALIVE states

    gajdusek committed Mar 16, 2014
    They are confusing, especially UPROC_ALIVE, and not needed anywhere.
    If you need, check UPROC_BASIC instead of UPROC_ALIVE and
    ! UPROC_BASIC instead of UPROC_NEW.
    Currently nothing calls update_processes_run() with a PROCTAB opened
    without OPENPROC_FLAGS, so every updated process has UPROC_BASIC state
    Should we be able to update only a subset of basic properties, e.g.
    from fast filters invoked by process_new_delay(), we should add other
    states, like UPROC_MINIMAL or, better, states named after /proc/<pid/
    files which has been parsed or after openproc flags, and set them
    in update_process_run() accordingly. And/or add enumerations to
    ENSURE_WHAT and handle them in u_proc_ensure(). BTW, u_proc states
    and u_proc_ensure() facilities overlap and should be refactored in
    some way.
  11. [core] proc_by_pid_with_retry(): Parse not yet known PIDs and TIDs

    gajdusek committed Mar 17, 2014
    Parse a PID if not already known by the core and return updated
    u_proc instance.
    If passed PID is an unknown thread, parse its thread leader and returns
    its u_proc instance.
    NULL is returned only if a passed PID vanished from /proc/ or if it
    is a thread which thread leader does no more exists.
  12. [netlink_fallback] Skip processes that are already known by the core

    gajdusek committed Mar 17, 2014
    Just to be sure we not do unnecessary work, if netlink_fallback
    inotify callback is delayed for some reason and the core is informed
    about the new process sooner.
  13. [core] Really make sure no thread goes through update_processes_run()

    gajdusek committed Mar 17, 2014
    update_processes_run() skips every parsed process which is actually
    a thread. This may not happen in full run but may happen if passed
    proctab is polluted by TIDs. I already removed PROC_LOOSE_TASKS from
    OPENPROC_FLAGS before, but this flag is only used in readproc2() which
    we don't use, so it's useless to include or exclude it.
  14. [netlink] Fix fork events

    gajdusek committed Mar 17, 2014
    Fix bug introduced by erroneous condition attempting to skip threads,
    which caused all fork events have been unintentionally ignored.
    Question is whether we should handle netlink FORK events at all, as
    it was non functional for a year and this probably didn't make the
    latency noticeable worse. One use for handling forks could be
    determining the parent of ongoing EXEC, but we fully update EXEC'd
    processes through libproc anyway. Another use is early tracking of
    their resources by protector rules.
    Also revert commit c04260 (early 2011) which following assumption is
    > fix a grave bug in the new delay code. the parent of the fork message
    > is of course, not the parent of the new process. it is the process
    > who called fork and therefore, the new processes parent is parent->parent.
    This commit causes many reparenting in the tree after the fork is
    actually updated through libproc.
    Correct fix is to use fork.child_tgid and fork.parent_tgid; not
    fork.child_pid and fork.parent_pid, as the later are actually not
    PIDs, but TIDs (thread IDs). And because the process who called fork
    may be actually a thread, i.e. its  tid ( (sic!)
    differs from its tgid, we easily poisoned process_new_delay()
    with threads. It's surely very unfortunate, the Linux process events
    connector says pid, but means tid (but /proc/#/status do this too).
  15. [UAssert] Add u_assert_process_dead (proc)

    gajdusek committed Mar 15, 2014
    Print a warning if process exists in /proc/ directory and is not
    a zombie.
    In DEVELOP_MODE, this assertion is called everytime some process
    is marked as dead, specifically everytime U_PROC_STATE(P, UPROC_DEAD)
    macro is called. Assertion failure in this case is a symptom that
    ulatencyd core is making false assumptions about /proc/ behaviour,
    i.e. it falsely assumes some process is dead or it is a zombie but
    instead it is still alive.
  16. [core] Fix DEC_REF underflow test and make DEC_REF more robust

    gajdusek committed Mar 14, 2014
    Change `ret` type from guint to gint.
    Remove spurious semicolon after macro block.
    Enclose macro argument with parenthesis on assignment.
  17. [consolekit] Fix possible access to free'd memory

    gajdusek committed Mar 10, 2014
    Increase reference count to proc->environ hash table in
Commits on Mar 17, 2014
  1. Additions to simple.d: Dropbox, Ubuntu-One, Deja-dup, Duplicity, and …

    Feminist-Software-Foundation committed Mar 17, 2014
Commits on Mar 14, 2014
  1. [core] Make sure `processes` table is not poisoned by tasks

    gajdusek committed Mar 9, 2014
    ulatencyd core must avoid tasks being handled as processes, there is
    even test in u_proc_add_task() triggering a warning if u_proc is not
    a thread leader.
    But PIDs given by user (via DBUS call) must not be thread leaders and
    if pushed to process_update_pid() they will poison our #processes
    This commit therefore removes PROC_LOOSE_TASKS from OPENPROC_FLAGS
    to ensure process_update_pid() and companions cannot poison the
    #processes table with tasks.
    proc_by_pid_with_retry() was updated to return u_proc of thread leader
    if passed pid is a task. But it will be returned only if the task is
    already known (in #tasks table). If not, we probably should check
    /proc/ to found out if it is an existent task and if it is then
    parse its thread leader with process_update_pid().
  2. [dbus.c] Ensure correctness of PID arguments passed by dbus

    gajdusek committed Mar 8, 2014
    They are read from dbus as uint64 but in reality pid_t is 32bit
    signed integer; so make sure (pid_t) uint64 > 0. This has been causing
    problems at least in g_hash_lookup() /read access to uninitialized
    memory/ called by proc_by_pid() and later in update_processes_run().
  3. [memleak] Fix memory leak in l_scheduler_list_configs()

    gajdusek committed Mar 8, 2014
    Leaked names of all configs everytime called (from dbus.c)
  4. [doxygen] Set TYPEDEF_HIDES_STRUCT = NO

    gajdusek committed Mar 8, 2014
    Because it does not handle forward typedef declarations.
  5. 7/8 [UFocus] Update xwatch, LUA binding, rules and docblocks

    gajdusek committed Mar 8, 2014
    LUA binding:
    Removed from ulatency table (fortunately not used anywhere):
     - get_active_uids, l_get_active_uids
     - get_active_pids, l_get_active_pids
    These should be reimplemented by UFocus being exported to LUA.
      - ulatency.set_active_pid() replaced by U_PROC:set_focused()
    U_PROC.is_active and U_PROC.active_pos replaced by U_PROC.focus_position
  6. 6/8 [UFocus] Update DBUS interface and ulatency client

    gajdusek committed Mar 8, 2014
    DBUS user interface:
    Remove setActiveWithUser
    Remove setActiveControl
    Remove property setActtiveListLength:q
    Comment out activeList property - need to be reimplemented it later
    Rename setActive => setFocusedPid
    Add getFocusStackLength in pid:t, out length:q
    Add setFocusStackLength in pid:t, in length:q
    ulatency client:
    ulatency get activelistlength => ulatency get focusStackLength
    ulatency set active => ulatency set focused
  7. 5/8 [UFocus] Add u_proc_* functions to set focus and get proc position

    gajdusek committed Mar 8, 2014
    + gboolean u_proc_set_focused (u_proc *proc, time_t timestamp);
    + guint16 u_proc_get_focus_position (u_proc *proc, gboolean force);
  8. 4/8 [UFocus] USession: Focus tracker may attach itself to a session

    gajdusek committed Mar 8, 2014
    A focus tracker agent (module) may register a hook to
    U_HOOK_TYPE_SESSION_FOCUS_TRACKER_CHANGED list, wait for a session which
    focus_tracker field changed to NULL, either because a new session was
    created or some other tracker released it.
    Then the new tracker may attach itself with
  9. 3/8 [UFocus] Implement UFocusStack (replaces active_users list)

    gajdusek committed Mar 8, 2014
    Currently it's a prototype implemented in a flat list, just as
    active_users was. This allows me to adapt other parts of code more
    quickly, especially adapt and test the xwatch module.
    But later this must be reworked into a tree, which will allow accept
    user space hints and also make the focus inherited by children processes,
    without breaking everything if user focuses terminal with dozens tabs
    and applications started from it. Also it should be much faster to
    get current process position.
    You may find a proposal in docs/UEP/UFocus/focus_tree.dia