Skip to content
Commits on May 21, 2010
  1. @gitster

    Merge branch 'sp/maint-dumb-http-pack-reidx'

    * sp/maint-dumb-http-pack-reidx:
      http.c::new_http_pack_request: do away with the temp variable filename
      http-fetch: Use temporary files for pack-*.idx until verified
      http-fetch: Use index-pack rather than verify-pack to check packs
      Allow parse_pack_index on temporary files
      Extract verify_pack_index for reuse from verify_pack
      Introduce close_pack_index to permit replacement
      http.c: Remove unnecessary strdup of sha1_to_hex result
      http.c: Don't store destination name in request structures
      http.c: Drop useless != NULL test in finish_http_pack_request
      http.c: Tiny refactoring of finish_http_pack_request
      t5550-http-fetch: Use subshell for repository operations
      http.c: Remove bad free of static block
    gitster committed May 21, 2010
Commits on Apr 20, 2010
  1. @spearce @gitster

    Extract verify_pack_index for reuse from verify_pack

    The dumb HTTP transport should verify an index is completely valid
    before trying to use it.  That requires checking the header/footer
    but also checking the complete content SHA-1.  All of this logic is
    already in the front half of verify_pack, so pull it out into a new
    function that can be reused.
    Signed-off-by: Shawn O. Pearce <>
    Signed-off-by: Junio C Hamano <>
    spearce committed with gitster Apr 19, 2010
Commits on Feb 23, 2010
  1. @npitre @gitster

    move encode_in_pack_object_header() to a better place

    Commit 1b22b6c made duplicated versions of encode_header() into a
    common version called encode_in_pack_object_header(). There is however
    a better location that sha1_file.c for such a function though, as
    sha1_file.c contains nothing related to the creation of packs, and
    it is quite populated already.
    Also the comment that was moved to the header file should really remain
    near the function as it covers implementation details and provides no
    information about the actual function interface.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    npitre committed with gitster Feb 23, 2010
Commits on Jan 22, 2010
  1. @torvalds @gitster

    make "index-pack" a built-in

    This required some fairly trivial packfile function 'const' cleanup,
    since the builtin commands get a const char *argv[] array.
    Signed-off-by: Linus Torvalds <>
    Signed-off-by: Junio C Hamano <>
    torvalds committed with gitster Jan 22, 2010
Commits on Aug 30, 2008
  1. @gitster

    improve reliability of fixup_pack_header_footer()

    Currently, this function has the potential to read corrupted pack data
    from disk and give it a valid SHA1 checksum.  Let's add the ability to
    validate SHA1 checksum of existing data along the way, including before
    and after any arbitrary point in the pack.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with gitster Aug 29, 2008
Commits on Jun 25, 2008
  1. @gitster

    verify-pack: check packed object CRC when using index version 2

    To do so, check_pack_crc() moved from builtin-pack-objects.c to
    pack-check.c where it is more logical to share.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with gitster Jun 24, 2008
  2. @gitster

    move show_pack_info() where it belongs

    This is called when verify_pack() has its verbose argument set, and
    verbose in this context makes sense only for the actual 'git verify-pack'
    command.  Therefore let's move show_pack_info() to builtin-verify-pack.c
    instead and remove useless verbose argument from verify_pack().
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with gitster Jun 24, 2008
Commits on Nov 2, 2007
  1. @gitster

    Merge branch 'np/progress'

    * np/progress:
      Show total transferred as part of throughput progress
      make sure throughput display gets updated even if progress doesn't move
      return the prune-packed progress display to the inner loop
      add throughput display to git-push
      add some copyright notice to the progress display code
      add throughput display to index-pack
      add throughput to progress display
      relax usage of the progress API
      make struct progress an opaque type
      prune-packed: don't call display_progress() for every file
      Stop displaying "Pack pack-$ID created." during git-gc
      Teach prune-packed to use the standard progress meter
      Change 'Deltifying objects' to 'Compressing objects'
      fix for more minor memory leaks
      fix const issues with some functions
      pack-objects.c: fix some global variable abuse and memory leaks
      pack-objects: no delta possible with only one object in the list
      cope with multiple line breaks within sideband progress messages
      more compact progress display
    gitster committed Nov 2, 2007
Commits on Oct 17, 2007
  1. @spearce

    fix const issues with some functions

    Two functions, namely write_idx_file() and open_pack_file(), currently
    return a const pointer.  However that pointer is either a copy of the
    first argument, or set to a malloc'd buffer when that first argument
    is null.  In the later case it is wrong to qualify that pointer as const
    since ownership of the buffer is transferred to the caller to dispose of,
    and obviously the free() function is not meant to be passed const
    Making the return pointer not const causes a warning when the first
    argument is returned since that argument is also marked const.
    The correct thing to do is therefore to remove the const qualifiers,
    avoiding the need for ugly casts only to silence some warnings.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Shawn O. Pearce <>
    Nicolas Pitre committed with spearce Oct 16, 2007
Commits on Sep 19, 2007
  1. @spearce @gitster

    Refactor index-pack "keep $sha1" handling for reuse

    There is a subtle (but important) linkage between receive-pack and
    index-pack that allows index-pack to create a packfile but protect
    it from being deleted by a concurrent `git repack -a -d` operation.
    The linkage works by having index-pack mark the newly created pack
    with a ".keep" file and then it passes the SHA-1 name of that new
    packfile to receive-pack along its stdout channel.
    The receive-pack process must unkeep the packfile by deleting the
    .keep file, but can it can only do so after all elgible refs have
    been updated in the receiving repository.  This ensures that the
    packfile is either kept or its objects are reachable, preventing
    a concurrent repacker from deleting the packfile before it can
    determine that its objects are actually needed by the repository.
    The new builtin-fetch code needs to perform the same actions if
    it choose to run index-pack rather than unpack-objects, so I am
    moving this code out to its own function where both receive-pack
    and fetch-pack are able to invoke it when necessary.  The caller
    is responsible for deleting the returned ".keep" and freeing the
    path if the returned path is not NULL.
    Signed-off-by: Shawn O. Pearce <>
    Signed-off-by: Junio C Hamano <>
    spearce committed with gitster Sep 14, 2007
Commits on Jun 2, 2007
  1. Unify write_index_file functions

    This patch unifies the write_index_file functions in
    builtin-pack-objects.c and index-pack.c.  As the name
    "index" is overloaded in git, move in the direction of
    using "idx" and "pack idx" when refering to the pack index.
    There should be no change in functionality.
    Signed-off-by: Geert Bosch <>
    Acked-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Geert Bosch committed with Junio C Hamano Jun 1, 2007
Commits on May 2, 2007
  1. @spearce

    Create pack-write.c for common pack writing code

    Include a generalized fixup_pack_header_footer() in this new file.
    Needed by git-repack --max-pack-size feature in a later patchset.
    [sp: Moved close(pack_fd) to callers, to support index-pack, and
         changed name to better indicate it is for packfiles.]
    Signed-off-by: Dana L. How <>
    Signed-off-by: Shawn O. Pearce <>
    Dana L. How committed with spearce May 2, 2007
Commits on Mar 17, 2007
  1. [PATCH] clean up pack index handling a bit

    Especially with the new index format to come, it is more appropriate
    to encapsulate more into check_packed_git_idx() and assume less of the
    index format in struct packed_git.
    To that effect, the index_base is renamed to index_data with void * type
    so it is not used directly but other pointers initialized with it. This
    allows for a couple pointer cast removal, as well as providing a better
    generic name to grep for when adding support for new index versions or
    And index_data is declared const too while at it.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with Junio C Hamano Mar 16, 2007
Commits on Jan 25, 2007
  1. Refactor the pack header reading function out of receive-pack.c

    Signed-off-by: Junio C Hamano <>
    Junio C Hamano committed Jan 22, 2007
Commits on Jan 18, 2007
  1. @corecode

    Use fixed-size integers for the on-disk pack structure.

    Plain integer types without a fixed size can vary between platforms.  Even
    though all common platforms use 32-bit ints, there is no guarantee that
    this won't change at some point.  Furthermore, specifying an integer type
    with explicit size makes the definition of structures more obvious.
    Signed-off-by: Simon 'corecode' Schubert <>
    Signed-off-by: Junio C Hamano <>
    corecode committed with Junio C Hamano Jan 17, 2007
  2. @spearce

    Document pack .idx file format upgrade strategy.

    Way back when Junio developed the 64 bit index topic he came up
    with a means of changing the .idx file format so that older Git
    clients would recognize that they don't understand the file and
    refuse to read it, while newer clients could tell the difference
    between the old-style and new-style .idx files.  Unfortunately
    this wasn't recorded anywhere.
    This change documents how we might go about changing the .idx
    file format by using a special signature in the first four bytes.
    Credit (and possible blame) goes completely to Junio for thinking
    up this technique.
    The change also modifies the error message of the current Git code
    so that users get a recommendation to upgrade their Git software
    should this version or later encounter a new-style .idx which it
    cannot process.  We already do this for the .pack files, but since
    we usually process the .idx files first its important that these
    files are recognized and encourage an upgrade.
    Signed-off-by: Shawn O. Pearce <>
    Signed-off-by: Junio C Hamano <>
    spearce committed with Junio C Hamano Jan 17, 2007
Commits on Oct 23, 2006
  1. Merge branch 'np/pack'

    * np/pack:
      add the capability for index-pack to read from a stream
      index-pack: compare only the first 20-bytes of the key.
      git-repack: repo.usedeltabaseoffset
      pack-objects: document --delta-base-offset option
      allow delta data reuse even if base object is a preferred base
      zap a debug remnant
      let the GIT native protocol use offsets to delta base when possible
      make pack data reuse compatible with both delta types
      make git-pack-objects able to create deltas with offset to base
      teach git-index-pack about deltas with offset to base
      teach git-unpack-objects about deltas with offset to base
      introduce delta objects with offset to base
    Junio C Hamano committed Oct 22, 2006
Commits on Oct 15, 2006
  1. Revert "move pack creation to version 3"

    This reverts commit 1685457.
    Git as recent as v1.1.6 do not understand version 3 delta.
    v1.2.0 is Ok and I personally would say it is old enough, but
    the improvement between version 2 and version 3 delta is not
    bit enough to justify breaking older clients.
    We should resurrect this later, but when we do so we shold
    make it conditional.
    Signed-off-by: Junio C Hamano <>
    Junio C Hamano committed Oct 14, 2006
Commits on Sep 27, 2006
  1. make pack data reuse compatible with both delta types

    This is the missing part to git-pack-objects allowing it to reuse delta
    data to/from any of the two delta types.  It can reuse delta from any
    type, and it outputs base offsets when --allow-delta-base-offset is
    provided and the base is also included in the pack.  Otherwise it
    outputs base sha1 references just like it always did.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with Junio C Hamano Sep 22, 2006
Commits on Sep 23, 2006
  1. move pack creation to version 3

    It's been quite a while now that GIT is able to read version 3 packs.
    Let's create them at last.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with Junio C Hamano Sep 21, 2006
Commits on Jul 13, 2006
  1. Remove TYPE_* constant macros and use object_type enums consistently.

    This updates the type-enumeration constants introduced to reduce
    the memory footprint of "struct object" to match the type bits
    already used in the packfile format, by removing the former
    (i.e. TYPE_* constant macros) and using the latter (i.e. enum
    object_type) throughout the code for consistency.
    Eventually we can stop passing around the "type strings"
    entirely, and this will help - no confusion about two different
    integer enumeration.
    Signed-off-by: Linus Torvalds <>
    Signed-off-by: Junio C Hamano <>
    Linus Torvalds committed with Junio C Hamano Jul 11, 2006
Commits on Feb 17, 2006
  1. pack-objects: reuse data from existing packs.

    When generating a new pack, notice if we have already needed
    objects in existing packs.  If an object is stored deltified,
    and its base object is also what we are going to pack, then
    reuse the existing deltified representation unconditionally,
    bypassing all the expensive find_deltas() and try_deltas()
    Also, notice if what we are going to write out exactly match
    what is already in an existing pack (either deltified or just
    compressed).  In such a case, we can just copy it instead of
    going through the usual uncompressing & recompressing cycle.
    Without this patch, in linux-2.6 repository with about 1500
    loose objects and a single mega pack:
        $ git-rev-list --objects v2.6.16-rc3 >RL
        $ wc -l RL
        184141 RL
        $ time git-pack-objects p <RL
        Generating pack...
        Done counting 184141 objects.
        Packing 184141 objects....................
        real    12m4.323s
        user    11m2.560s
        sys     0m55.950s
    With this patch, the same input:
        $ time ../git.junio/git-pack-objects q <RL
        Generating pack...
        Done counting 184141 objects.
        Packing 184141 objects.....................
        Total 184141, written 184141, reused 182441
        real    1m2.608s
        user    0m55.090s
        sys     0m1.830s
    Signed-off-by: Junio C Hamano <>
    Junio C Hamano committed Feb 15, 2006
Commits on Feb 10, 2006
  1. remove delta-against-self bit

    After experimenting with code to add the ability to encode a delta
    against part of the deltified file, it turns out that resulting packs
    are _bigger_ than when this ability is not used.  The raw delta output
    might be smaller, but it doesn't compress as well using gzip with a
    negative net saving on average.
    Said bit would in fact be more useful to allow for encoding the copying
    of chunks larger than 64KB providing more savings with large files.
    This will correspond to packs version 3.
    While the current code still produces packs version 2, it is made future
    proof so pack versions 2 and 3 are accepted.  Any pack version 2 are
    compatible with version 3 since the redefined bit was never used before.
    When enough time has passed, code to use that bit to produce version 3
    packs could be added.
    Signed-off-by: Nicolas Pitre <>
    Signed-off-by: Junio C Hamano <>
    Nicolas Pitre committed with Junio C Hamano Feb 9, 2006
Commits on Jul 1, 2005
  1. [PATCH] verify-pack updates.

    Nico pointed out that having verify_pack.c and verify-pack.c was
    confusing.  Rename verify_pack.c to pack-check.c as suggested,
    and enhances the verification done quite a bit.
     - Built-in sha1_file unpacking knows that a base object of a
       deltified object _must_ be in the same pack, and takes
       advantage of that fact.
     - Earlier verify-pack command only checked the SHA1 sum for the
       entire pack file and did not look into its contents.  It now
       checks everything idx file claims to have unpacks correctly.
     - It now has a hook to give more detailed information for
       objects contained in the pack under -v flag.
    Signed-off-by: Junio C Hamano <>
    Signed-off-by: Linus Torvalds <>
    Junio C Hamano committed with Linus Torvalds Jun 30, 2005
Commits on Jun 29, 2005
  1. [PATCH] Add git-verify-pack command.

    Given a list of <pack>.idx files, this command validates the
    index file and the corresponding .pack file for consistency.
    This patch also uses the same validation mechanism in fsck-cache
    when the --full flag is used.
    During normal operation, sha1_file.c verifies that a given .idx
    file matches the .pack file by comparing the SHA1 checksum
    stored in .idx file and .pack file as a minimum sanity check.
    We may further want to check the pack signature and version when
    we map the pack, but that would be a separate patch.
    Earlier, errors to map a pack file was not flagged fatal but led
    to a random fatal error later.  This version explicitly die()s
    when such an error is detected.
    Signed-off-by: Junio C Hamano <>
    Signed-off-by: Linus Torvalds <>
    Junio C Hamano committed with Linus Torvalds Jun 29, 2005
  2. Make git pack files use little-endian size encoding

    This makes it match the new delta encoding, and admittedly makes the
    code easier to follow.
    This also updates the PACK file version to 2, since this (and the delta
    encoding change in the previous commit) are incompatible with the old
    Linus Torvalds committed Jun 28, 2005
Commits on Jun 28, 2005
  1. Change pack file format. Hopefully for the last time.

    This also adds a header with a signature, version info, and the number
    of objects to the pack file.  It also encodes the file length and type
    more efficiently.
    Linus Torvalds committed Jun 28, 2005
Something went wrong with that request. Please try again.