Skip to content
/ git Public
forked from git/git

Commit

Permalink
[PATCH] Avoid wasting memory in git-rev-list
Browse files Browse the repository at this point in the history
As pointed out on the list, git-rev-list can use a lot of memory.

One low-hanging fruit is to free the commit buffer for commits that we
parse. By default, parse_commit() will save away the buffer, since a lot
of cases do want it, and re-reading it continually would be unnecessary.
However, in many cases the buffer isn't actually necessary and saving it
just wastes memory.

We could just free the buffer ourselves, but especially in git-rev-list,
we actually end up using the helper functions that automatically add
parent commits to the commit lists, so we don't actually control the
commit parsing directly.

Instead, just make this behaviour of "parse_commit()" a global flag.
Maybe this is a bit tasteless, but it's very simple, and it makes a
noticable difference in memory usage.

Before the change:

	[torvalds@g5 linux]$ /usr/bin/time git-rev-list v2.6.12..HEAD > /dev/null
	0.26user 0.02system 0:00.28elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+3714minor)pagefaults 0swaps

after the change:

	[torvalds@g5 linux]$ /usr/bin/time git-rev-list v2.6.12..HEAD > /dev/null
	0.26user 0.00system 0:00.27elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
	0inputs+0outputs (0major+2433minor)pagefaults 0swaps

note how the minor faults have decreased from 3714 pages to 2433 pages.
That's all due to the fewer anonymous pages allocated to hold the comment
buffers and their metadata.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
  • Loading branch information
Linus Torvalds authored and Junio C Hamano committed Sep 15, 2005
1 parent f71a69a commit 60ab26d
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 1 deletion.
4 changes: 3 additions & 1 deletion commit.c
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
#include "commit.h"
#include "cache.h"

int save_commit_buffer = 1;

struct sort_node
{
/*
Expand Down Expand Up @@ -264,7 +266,7 @@ int parse_commit(struct commit *item)
sha1_to_hex(item->object.sha1));
}
ret = parse_commit_buffer(item, buffer, size);
if (!ret) {
if (save_commit_buffer && !ret) {
item->buffer = buffer;
return 0;
}
Expand Down
1 change: 1 addition & 0 deletions commit.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ struct commit {
char *buffer;
};

extern int save_commit_buffer;
extern const char *commit_type;

struct commit *lookup_commit(const unsigned char *sha1);
Expand Down
2 changes: 2 additions & 0 deletions rev-list.c
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,8 @@ int main(int argc, char **argv)
handle_one_commit(commit, &list);
}

save_commit_buffer = verbose_header;

if (!merge_order) {
sort_by_date(&list);
if (limited)
Expand Down

0 comments on commit 60ab26d

Please sign in to comment.