Skip to content
This repository has been archived by the owner on Nov 9, 2017. It is now read-only.

Improve git-status performance #94

Closed
wants to merge 12 commits into from
Closed

Commits on Sep 10, 2013

  1. MSVC: fix compile errors due to missing libintl.h

    Set NO_GETTEXT in config.mak.uname to get rid of libintl.h dependency.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Sep 10, 2013
    Configuration menu
    Copy the full SHA
    60d7821 View commit details
    Browse the repository at this point in the history
  2. MSVC: fix compile errors due to macro redefinitions

    Skip errno.h definitions if they are already defined.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Sep 10, 2013
    Configuration menu
    Copy the full SHA
    d82a934 View commit details
    Browse the repository at this point in the history
  3. MSVC: fix stat definition hell

    In msvc.h, there's a couple of stat related functions defined diffently
    from mingw.h. When we remove these definitions, the only problem we get is
    "warning C4005: '_stati64' : macro redefinition" for this line in mingw.h:
    
    #define _stati64(x,y) mingw_stat(x,y)
    
    The reason is that as of MSVCR80.dll (distributed with MSVC 2005), the
    original _stati64 family of functions was renamed to _stat32i64, and the
    former function names became macros (pointing to the appropriate function
    based on the definition of _USE_32BIT_TIME_T).
    
    Defining _stati64 works on MinGW because MinGW by default compiles against
    the MSVCRT.DLL that is part of Windows (i.e. _stati64 is a function rather
    than a macro).
    
    Note: MinGW *can* compile for newer MSVC runtime versions, and MSVC
    apparently can also compile for the Windows MSVCRT.DLL via the DDK (see
    http://www.syndicateofideas.com/posts/fighting-the-msvcrt-dll-hell ).
    
    Remove the stat definitions from msvc.h, as they are not compiler related.
    
    In mingw.h, determine the runtime version in use from the definitions of
    _stati64 and _USE_32BIT_TIME_T, and define stat() accordingly.
    
    This also fixes that stat() in MSVC builds still resolves to mingw_lstat()
    instead of mingw_stat().
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Sep 10, 2013
    Configuration menu
    Copy the full SHA
    3a05a04 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2013

  1. add a hashtable implementation that supports O(1) removal

    The existing hashtable implementation (in hash.[ch]) uses open addressing
    (i.e. resolve hash collisions by distributing entries across the table).
    Thus, removal is difficult to implement with less than O(n) complexity.
    Resolving collisions of entries with identical hashes (e.g. via chaining)
    is left to the client code.
    
    Add a hashtable implementation that supports O(1) removal and is slightly
    easier to use due to builtin entry chaining.
    
    Supports all basic operations init, free, get, add, remove and iteration.
    
    Also includes ready-to-use hash functions based on the public domain FNV-1
    algorithm (http://www.isthe.com/chongo/tech/comp/fnv).
    
    The per-entry data structure (hashmap_entry) is piggybacked in front of
    the client's data structure to save memory. See test-hashmap.c for usage
    examples.
    
    The hashtable is resized by a factor of four when 80% full. With these
    settings, average memory consumption is about 2/3 of hash.[ch], and
    insertion is about twice as fast due to less frequent resizing.
    
    Lookups are also slightly faster, because entries are strictly confined to
    their bucket (i.e. no data of other buckets needs to be traversed).
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Sep 29, 2013
    Configuration menu
    Copy the full SHA
    506d885 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2013

  1. Configuration menu
    Copy the full SHA
    6c99f68 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2013

  1. Configuration menu
    Copy the full SHA
    bb1aac3 View commit details
    Browse the repository at this point in the history
  2. Win32: make FILETIME conversion functions public

    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Oct 10, 2013
    Configuration menu
    Copy the full SHA
    70887a6 View commit details
    Browse the repository at this point in the history
  3. Win32: dirent.c: Move opendir down

    Move opendir down in preparation for the next patch.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Oct 10, 2013
    Configuration menu
    Copy the full SHA
    97b6df3 View commit details
    Browse the repository at this point in the history
  4. Win32: Make the dirent implementation pluggable

    Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is
    pretty staightforward, however, most of the information provided in the
    WIN32_FIND_DATA structure is thrown away in the process. A more
    sophisticated implementation may cache this data, e.g. for later reuse in
    calls to lstat.
    
    Make the dirent implementation pluggable so that it can be switched at
    runtime, e.g. based on a config option.
    
    Define a base DIR structure with pointers to readdir/closedir that match
    the opendir implementation (i.e. similar to vtable pointers in OOP).
    Define readdir/closedir so that they call the function pointers in the DIR
    structure. This allows to choose the opendir implementation on a
    call-by-call basis.
    
    Move the fixed sized dirent.d_name buffer to the dirent-specific DIR
    structure, as d_name may be implementation specific (e.g. a caching
    implementation may just set d_name to point into the cache instead of
    copying the entire file name string).
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Oct 10, 2013
    Configuration menu
    Copy the full SHA
    3176000 View commit details
    Browse the repository at this point in the history
  5. Win32: make the lstat implementation pluggable

    Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite
    slow. Windows operating system APIs seem to be much better at scanning the
    status of entire directories than checking single files. A caching
    implementation may improve performance by bulk-reading entire directories
    or reusing data obtained via opendir / readdir.
    
    Make the lstat implementation pluggable so that it can be switched at
    runtime, e.g. based on a config option.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Oct 10, 2013
    Configuration menu
    Copy the full SHA
    5801433 View commit details
    Browse the repository at this point in the history
  6. add infrastructure for read-only file system level caches

    Add a macro to mark code sections that only read from the file system,
    along with a config option and documentation.
    
    This facilitates implementation of relatively simple file system level
    caches without the need to synchronize with the file system.
    
    Enable read-only sections for 'git status' and preload_index.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Oct 10, 2013
    Configuration menu
    Copy the full SHA
    9ce4a71 View commit details
    Browse the repository at this point in the history
  7. Win32: add a cache below mingw's lstat and dirent implementations

    Checking the work tree status is quite slow on Windows, due to slow lstat
    emulation (git calls lstat once for each file in the index). Windows
    operating system APIs seem to be much better at scanning the status
    of entire directories than checking single files.
    
    Add an lstat implementation that uses a cache for lstat data. Cache misses
    read the entire parent directory and add it to the cache. Subsequent lstat
    calls for the same directory are served directly from the cache.
    
    Also implement opendir / readdir / closedir so that they create and use
    directory listings in the cache.
    
    The cache doesn't track file system changes and doesn't plug into any
    modifying file APIs, so it has to be explicitly enabled for git functions
    that don't modify the working copy.
    
    Note: in an earlier version of this patch, the cache was always active and
    tracked file system changes via ReadDirectoryChangesW. However, this was
    much more complex and had negative impact on the performance of modifying
    git commands such as 'git checkout'.
    
    Signed-off-by: Karsten Blees <blees@dcon.de>
    kblees committed Oct 10, 2013
    Configuration menu
    Copy the full SHA
    88f5ed0 View commit details
    Browse the repository at this point in the history