Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
x/tools/cmd/godoc: allow index merging #6813
We should support incremental indexing of large corpuses. We can already save a corpus to an io.Writer and load a corpus from an io.Reader, but once we have a large, saved Corpus, there is no way to update just part of it. I think we should add a method on Index, like: func (x *Index) Add(o *Index) *Index That returns the merge of x + o (with any redundant text in the bytes.Buffer/Fileset removed and compacted) Then incremental indexing just involves loading the last huge index from disk, building small indexes of subtrees, merging them in, writing to disk, etc. Perhaps the Index.Add method only adds and is quick, but we could also have a Compact method to return a compacted index (a slower operation), so the common case is a bunch of adds, followed by a less-frequent compact + write to disk.