New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/cmd/godoc: allow index merging #6813

Open
bradfitz opened this Issue Nov 21, 2013 · 3 comments

Comments

Projects
None yet
2 participants
@bradfitz
Member

bradfitz commented Nov 21, 2013

We should support incremental indexing of large corpuses.  We can already save a corpus
to an io.Writer and load a corpus from an io.Reader, but once we have a large, saved
Corpus, there is no way to update just part of it.

I think we should add a method on Index, like:

func (x *Index) Add(o *Index) *Index

That returns the merge of x + o (with any redundant text in the bytes.Buffer/Fileset
removed and compacted)

Then incremental indexing just involves loading the last huge index from disk, building
small indexes of subtrees, merging them in, writing to disk, etc.

Perhaps the Index.Add method only adds and is quick, but we could also have a Compact
method to return a compacted index (a slower operation), so the common case is a bunch
of adds, followed by a less-frequent compact + write to disk.
@rsc

This comment has been minimized.

Contributor

rsc commented Nov 27, 2013

Comment 1:

Labels changed: added go1.3maybe.

@rsc

This comment has been minimized.

Contributor

rsc commented Dec 4, 2013

Comment 2:

Labels changed: added release-none, removed go1.3maybe.

@rsc

This comment has been minimized.

Contributor

rsc commented Dec 4, 2013

Comment 3:

Labels changed: added repo-tools.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015

@rsc rsc removed release-none labels Apr 10, 2015

@rsc rsc changed the title from godoc: allow index merging to x/tools/cmd/godoc: allow index merging Apr 14, 2015

@rsc rsc removed the repo-tools label Apr 14, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment