Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Commits on Jun 6, 2009
  1. @glandium

    Don't expect verify_pack() callers to set pack_size

    glandium authored committed
    Since use_pack() will end up populating pack_size if it is not already set,
    we can just adapt the code in verify_packfile() such that it doesn't require
    pack_size to be set beforehand.
    
    This allows callers not to have to set pack_size themselves, and we can thus
    revert changes from 1c23d79 (Don't die in git-http-fetch when fetching packs).
    
    Signed-off-by: Mike Hommey <mh@glandium.org>
    Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commits on Oct 3, 2008
  1. @spearce

    fix openssl headers conflicting with custom SHA1 implementations

    Nicolas Pitre authored spearce committed
    On ARM I have the following compilation errors:
    
        CC fast-import.o
    In file included from cache.h:8,
                     from builtin.h:6,
                     from fast-import.c:142:
    arm/sha1.h:14: error: conflicting types for 'SHA_CTX'
    /usr/include/openssl/sha.h:105: error: previous declaration of 'SHA_CTX' was here
    arm/sha1.h:16: error: conflicting types for 'SHA1_Init'
    /usr/include/openssl/sha.h:115: error: previous declaration of 'SHA1_Init' was here
    arm/sha1.h:17: error: conflicting types for 'SHA1_Update'
    /usr/include/openssl/sha.h:116: error: previous declaration of 'SHA1_Update' was here
    arm/sha1.h:18: error: conflicting types for 'SHA1_Final'
    /usr/include/openssl/sha.h:117: error: previous declaration of 'SHA1_Final' was here
    make: *** [fast-import.o] Error 1
    
    This is because openssl header files are always included in
    git-compat-util.h since commit 684ec6c whenever NO_OPENSSL is not
    set, which somehow brings in <openssl/sha1.h> clashing with the custom
    ARM version.  Compilation of git is probably broken on PPC too for the
    same reason.
    
    Turns out that the only file requiring openssl/ssl.h and openssl/err.h
    is imap-send.c.  But only moving those problematic includes there
    doesn't solve the issue as it also includes cache.h which brings in the
    conflicting local SHA1 header file.
    
    As suggested by Jeff King, the best solution is to rename our references
    to SHA1 functions and structure to something git specific, and define those
    according to the implementation used.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Commits on Jun 25, 2008
  1. verify-pack: check packed object CRC when using index version 2

    Nicolas Pitre authored committed
    To do so, check_pack_crc() moved from builtin-pack-objects.c to
    pack-check.c where it is more logical to share.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
  2. move show_pack_info() where it belongs

    Nicolas Pitre authored committed
    This is called when verify_pack() has its verbose argument set, and
    verbose in this context makes sense only for the actual 'git verify-pack'
    command.  Therefore let's move show_pack_info() to builtin-verify-pack.c
    instead and remove useless verbose argument from verify_pack().
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
  3. optimize verify-pack a bit

    Nicolas Pitre authored committed
    Using find_pack_entry_one() to get object offsets is rather suboptimal
    when nth_packed_object_offset() can be used directly.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commits on Jun 24, 2008
  1. call init_pack_revindex() lazily

    Nicolas Pitre authored committed
    This makes life much easier for next patch, as well as being more efficient
    when the revindex is actually not used.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commits on Jun 2, 2008
  1. make verify-pack a bit more useful with bad packs

    Nicolas Pitre authored committed
    When a pack gets corrupted, its SHA1 checksum will fail.  However, this
    is more useful to let the test go on in order to find the actual
    problem location than only complain about the SHA1 mismatch and
    bail out.
    
    Also, it is more useful to compare the stored pack SHA1 with the one in
    the index file instead of the computed SHA1 since the computed SHA1
    from a corrupted pack won't match the one stored in the index either.
    
    Finally a few code and message cleanups were thrown in as a bonus.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commits on Mar 1, 2008
  1. add storage size output to 'git verify-pack -v'

    Nicolas Pitre authored committed
    This can possibly break external scripts that depend on the previous
    output, but those script can't possibly be critical to Git usage, and
    fixing them should be trivial.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
  2. fix unimplemented packed_object_info_detail() features

    Nicolas Pitre authored committed
    Since commit eb32d23, there was a TODO
    comment in packed_object_info_detail() about the SHA1 of base object to
    OBJ_OFS_DELTA objects.  So here it is at last.
    
    While at it, providing the actual storage size information as well is now
    trivial.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commits on Jun 6, 2007
  1. @julliard

    pack-check: Sort entries by pack offset before unpacking them.

    julliard authored committed
    Because of the way objects are sorted in a pack, unpacking them in
    disk order is much more efficient than random access. Tests on the
    Wine repository show a gain in pack validation time of about 35%.
    
    Signed-off-by: Alexandre Julliard <julliard@winehq.org>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commits on May 27, 2007
  1. @spearce

    Lazily open pack index files on demand

    spearce authored Junio C Hamano committed
    In some repository configurations the user may have many packfiles,
    but all of the recent commits/trees/tags/blobs are likely to
    be in the most recent packfile (the one with the newest mtime).
    It is therefore common to be able to complete an entire operation
    by accessing only one packfile, even if there are 25 packfiles
    available to the repository.
    
    Rather than opening and mmaping the corresponding .idx file for
    every pack found, we now only open and map the .idx when we suspect
    there might be an object of interest in there.
    
    Of course we cannot known in advance which packfile contains an
    object, so we still need to scan the entire packed_git list to
    locate anything.  But odds are users want to access objects in the
    most recently created packfiles first, and that may be all they
    ever need for the current operation.
    
    Junio observed in b867092 that placing recent packfiles before
    older ones can slightly improve access times for recent objects,
    without degrading it for historical object access.
    
    This change improves upon Junio's observations by trying even harder
    to avoid the .idx files that we won't need.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on May 26, 2007
  1. fixes to output of git-verify-pack -v

    Nicolas Pitre authored Junio C Hamano committed
    Now that the default delta depth is 50, it is a good idea to also bump
    MAX_CHAIN to 50.
    
    While at it, make the display a bit prettier by making the MAX_CHAIN
    limit inclusive, and display the number of deltas that are above that
    limit at the end instead of the beginning.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Apr 10, 2007
  1. get rid of num_packed_objects()

    Nicolas Pitre authored Junio C Hamano committed
    The coming index format change doesn't allow for the number of objects
    to be determined from the size of the index file directly.  Instead, Let's
    initialize a field in the packed_git structure with the object count when
    the index is validated since the count is always known at that point.
    
    While at it let's reorder some struct packed_git fields to avoid padding
    due to needed 64-bit alignment for some of them.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Apr 5, 2007
  1. clean up and optimize nth_packed_object_sha1() usage

    Nicolas Pitre authored Junio C Hamano committed
    Let's avoid the open coded pack index reference in pack-object and use
    nth_packed_object_sha1() instead.  This will help encapsulating index
    format differences in one place.
    
    And while at it there is no reason to copy SHA1's over and over while a
    direct pointer to it in the index will do just fine.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Acked-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Mar 17, 2007
  1. [PATCH] clean up pack index handling a bit

    Nicolas Pitre authored Junio C Hamano committed
    Especially with the new index format to come, it is more appropriate
    to encapsulate more into check_packed_git_idx() and assume less of the
    index format in struct packed_git.
    
    To that effect, the index_base is renamed to index_data with void * type
    so it is not used directly but other pointers initialized with it. This
    allows for a couple pointer cast removal, as well as providing a better
    generic name to grep for when adding support for new index versions or
    formats.
    
    And index_data is declared const too while at it.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Mar 7, 2007
  1. @spearce

    Use off_t when we really mean a file offset.

    spearce authored Junio C Hamano committed
    Not all platforms have declared 'unsigned long' to be a 64 bit value,
    but we want to support a 64 bit packfile (or close enough anyway)
    in the near future as some projects are getting large enough that
    their packed size exceeds 4 GiB.
    
    By using off_t, the POSIX type that is declared to mean an offset
    within a file, we support whatever maximum file size the underlying
    operating system will handle.  For most modern systems this is up
    around 2^60 or higher.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
  2. @spearce

    Use uint32_t for all packed object counts.

    spearce authored Junio C Hamano committed
    As we permit up to 2^32-1 objects in a single packfile we cannot
    use a signed int to represent the object offset within a packfile,
    after 2^31-1 objects we will start seeing negative indexes and
    error out or compute bad addresses within the mmap'd index.
    
    This is a minor cleanup that does not introduce any significant
    logic changes.  It is roach free.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Feb 27, 2007
  1. convert object type handling from a string to a number

    Nicolas Pitre authored Junio C Hamano committed
    We currently have two parallel notation for dealing with object types
    in the code: a string and a numerical value.  One of them is obviously
    redundent, and the most used one requires more stack space and a bunch
    of strcmp() all over the place.
    
    This is an initial step for the removal of the version using a char array
    found in object reading code paths.  The patch is unfortunately large but
    there is no sane way to split it in smaller parts without breaking the
    system.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Dec 29, 2006
  1. @spearce

    Loop over pack_windows when inflating/accessing data.

    spearce authored Junio C Hamano committed
    When multiple mmaps start getting used for all pack file access it
    is not possible to get all data associated with a specific object
    in one contiguous memory region.  This limitation prevents simply
    passing a single address and length to SHA1_Update or to inflate.
    
    Instead we need to loop until we have processed all data of interest.
    
    As we loop over the data we are always interested in reusing the same
    window 'cursor', as the prior window will no longer be of any use
    to us.  This allows the use_pack() call to automatically decrement
    the use count of the prior window before setting up access for us
    to the next window.
    
    Within each loop we need to make use of the available length output
    parameter of use_pack() to tell us how many bytes are available in
    the current memory region, as we cannot tell otherwise.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
  2. @spearce

    Replace use_packed_git with window cursors.

    spearce authored Junio C Hamano committed
    Part of the implementation concept of the sliding mmap window for
    pack access is to permit multiple windows per pack to be mapped
    independently.  Since the inuse_cnt is associated with the mmap and
    not with the file, this value is in struct pack_window and needs to
    be incremented/decremented for each pack_window accessed by any code.
    
    To faciliate that implementation we need to replace all uses of
    use_packed_git() and unuse_packed_git() with a different API that
    follows struct pack_window objects rather than struct packed_git.
    
    The way this works is when we need to start accessing a pack for
    the first time we should setup a new window 'cursor' by declaring
    a local and setting it to NULL:
    
      struct pack_windows *w_curs = NULL;
    
    To obtain the memory region which contains a specific section of
    the pack file we invoke use_pack(), supplying the address of our
    current window cursor:
    
      unsigned int len;
      unsigned char *addr = use_pack(p, &w_curs, offset, &len);
    
    the returned address `addr` will be the first byte at `offset`
    within the pack file.  The optional variable len will also be
    updated with the number of bytes remaining following the address.
    
    Multiple calls to use_pack() with the same window cursor will
    update the window cursor, moving it from one window to another
    when necessary.  In this way each window cursor variable maintains
    only one struct pack_window inuse at a time.
    
    Finally before exiting the scope which originally declared the window
    cursor we must invoke unuse_pack() to unuse the current window (which
    may be different from the one that was first obtained from use_pack):
    
      unuse_pack(&w_curs);
    
    This implementation is still not complete with regards to multiple
    windows, as only one window per pack file is supported right now.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
  3. @spearce

    Refactor packed_git to prepare for sliding mmap windows.

    spearce authored Junio C Hamano committed
    The idea behind the sliding mmap window pack reader implementation
    is to have multiple mmap regions active against the same pack file,
    thereby allowing the process to mmap in only the active/hot sections
    of the pack and reduce overall virtual address space usage.
    
    To implement this we need to refactor the mmap related data
    (pack_base, pack_use_cnt) out of struct packed_git and move them
    into a new struct pack_window.
    
    We are refactoring the code to support a single struct pack_window
    per packfile, thereby emulating the prior behavior of mmap'ing the
    entire pack file.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
  4. @spearce

    Replace unpack_entry_gently with unpack_entry.

    spearce authored Junio C Hamano committed
    The unpack_entry_gently function currently has only two callers:
    the delta base resolution in sha1_file.c and the main loop of
    pack-check.c.  Both of these must change to using unpack_entry
    directly when we implement sliding window mmap logic, so I'm doing
    it earlier to help break down the change set.
    
    This may cause a slight performance decrease for delta base
    resolution as well as for pack-check.c's verify_packfile(), as
    the pack use counter will be incremented and decremented for every
    object that is unpacked.
    
    Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Sep 23, 2006
  1. many cleanups to sha1_file.c

    Nicolas Pitre authored Junio C Hamano committed
    Those cleanups are mainly to set the table for the support of deltas
    with base objects referenced by offsets instead of sha1.  This means
    that many pack lookup functions are converted to take a pack/offset
    tuple instead of a sha1.
    
    This eliminates many struct pack_entry usages since this structure
    carried redundent information in many cases, and it increased stack
    footprint needlessly for a couple recursively called functions that used
    to declare a local copy of it for every recursion loop.
    
    In the process, packed_object_info_detail() has been reorganized as well
    so to look much saner and more amenable to deltas with offset support.
    
    Finally the appropriate adjustments have been made to functions that
    depend on the above changes.  But there is no functionality changes yet
    simply some code refactoring at this point.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Aug 17, 2006
  1. Do not use memcmp(sha1_1, sha1_2, 20) with hardcoded length.

    David Rientjes authored Junio C Hamano committed
    Introduces global inline:
    
    	hashcmp(const unsigned char *sha1, const unsigned char *sha2)
    
    Uses memcmp for comparison and returns the result based on the length of
    the hash name (a future runtime decision).
    
    Acked-by: Alex Riesen <raa.lkml@gmail.com>
    Signed-off-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Jun 20, 2006
  1. Remove all void-pointer arithmetic.

    Florian Forster authored Junio C Hamano committed
    ANSI C99 doesn't allow void-pointer arithmetic. This patch fixes this in
    various ways. Usually the strategy that required the least changes was used.
    
    Signed-off-by: Florian Forster <octo@verplant.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Apr 27, 2006
  1. verify-pack: check integrity in a saner order.

    Junio C Hamano authored
    Check internal integrity to report corrupt pack or idx, and
    then check cross-integrity between idx and pack.
    
    Signed-off-by: Linus Torvalds <torvalds@osdl.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Mar 5, 2006
  1. verify-pack -v: show delta-chain histogram.

    Junio C Hamano authored
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Feb 15, 2006
  1. packed objects: minor cleanup

    Junio C Hamano authored
    The delta depth is unsigned.
    
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Feb 10, 2006
  1. remove delta-against-self bit

    Nicolas Pitre authored Junio C Hamano committed
    After experimenting with code to add the ability to encode a delta
    against part of the deltified file, it turns out that resulting packs
    are _bigger_ than when this ability is not used.  The raw delta output
    might be smaller, but it doesn't compress as well using gzip with a
    negative net saving on average.
    
    Said bit would in fact be more useful to allow for encoding the copying
    of chunks larger than 64KB providing more savings with large files.
    This will correspond to packs version 3.
    
    While the current code still produces packs version 2, it is made future
    proof so pack versions 2 and 3 are accepted.  Any pack version 2 are
    compatible with version 3 since the redefined bit was never used before.
    When enough time has passed, code to use that bit to produce version 3
    packs could be added.
    
    Signed-off-by: Nicolas Pitre <nico@cam.org>
    Signed-off-by: Junio C Hamano <junkio@cox.net>
Commits on Aug 10, 2005
  1. @sirainen

    [PATCH] -Werror fixes

    sirainen authored Junio C Hamano committed
    GCC's format __attribute__ is good for checking errors, especially
    with -Wformat=2 parameter. This fixes most of the reported problems
    against 2005-08-09 snapshot.
Commits on Jul 7, 2005
  1. [PATCH] Typofix an error message in pack-check.c

    Junio C Hamano authored Linus Torvalds committed
    The current error message does not make any sense.
    
    Signed-off-by: Junio C Hamano <junkio@cox.net>
    Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Commits on Jul 1, 2005
  1. [PATCH] Show more details of packfile with verify-pack -v.

    Junio C Hamano authored Linus Torvalds committed
    This implements show_pack_info() function used in verify-pack
    command when -v flag is used to obtain something like
    unpack-objects used to give when it was first written.
    
    It shows the following for each non-deltified object found in
    the pack:
    
        SHA1 type size offset
    
    For deltified objects, it shows this instead:
    
        SHA1 type size offset depth base_sha1
    
    In order to get the output in the order that appear in the pack
    file for debugging purposes, you can do this:
    
     $ git-verify-pack -v packfile | sort -n -k 4,4
    
    Signed-off-by: Junio C Hamano <junkio@cox.net>
    Signed-off-by: Linus Torvalds <torvalds@osdl.org>
  2. [PATCH] verify-pack updates.

    Junio C Hamano authored Linus Torvalds committed
    Nico pointed out that having verify_pack.c and verify-pack.c was
    confusing.  Rename verify_pack.c to pack-check.c as suggested,
    and enhances the verification done quite a bit.
    
     - Built-in sha1_file unpacking knows that a base object of a
       deltified object _must_ be in the same pack, and takes
       advantage of that fact.
    
     - Earlier verify-pack command only checked the SHA1 sum for the
       entire pack file and did not look into its contents.  It now
       checks everything idx file claims to have unpacks correctly.
    
     - It now has a hook to give more detailed information for
       objects contained in the pack under -v flag.
    
    Signed-off-by: Junio C Hamano <junkio@cox.net>
    Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Something went wrong with that request. Please try again.