Version 3 drops the packed-git commit I mentioned earlier, simplifies the
finish_if_sparse API as suggested by Eric, and splits the object sort commit
into 2 commits: one to add the additional progress indicators and one to
sort the objects by packfile. This makes it easier to follow.

----------------------------------------------------------------------------

Version 2 addresses progress-related concerns raised in the previous version
of the midx verify code.

This version also extends the existing progress.[ch] code and adds a
"sparse" mode that automatically ensures the 100% message is issued.

----------------------------------------------------------------------------

Teach "multi-pack-index verify" to handle cases where the number of
packfiles exceeds the open file handle limit.

The first commit fixes a problem that prevented the LRU-style
close_one_pack() mechanism from working which caused midx verify to run out
of file descriptors.

The second commit teaches midx verify to sort the set of objects to verify
by packfile rather than verifying them in OID order. This eliminates the
need to have more than one packfile/idx open at the same time.

With the second commit, runtime on 3600 packfiles went from 12 minutes to 25
seconds.

Thanks, Jeff

Cc: dstolee@microsoft.com

Jeff Hostetler (4):
  progress: add sparse mode to force 100% complete message
  trace2:data: add trace2 data to midx
  midx: add progress indicators in multi-pack-index verify
  midx: during verify group objects by packfile to speed verification

 builtin/multi-pack-index.c |  3 ++
 midx.c                     | 79 +++++++++++++++++++++++++++++++++++---
 packfile.c                 |  2 +-
 packfile.h                 |  2 +
 progress.c                 | 38 ++++++++++++++++--
 progress.h                 |  3 ++
 6 files changed, 118 insertions(+), 9 deletions(-)

base-commit: e902e9bcae2010bc42648c80ab6adc6c5a16a4a5

Submitted-As: https://public-inbox.org/git/pull.166.v3.git.gitgitgadget@gmail.com
In-Reply-To: https://public-inbox.org/git/pull.166.git.gitgitgadget@gmail.com
In-Reply-To: https://public-inbox.org/git/pull.166.v2.git.gitgitgadget@gmail.com