plumbing: object, add APIs for traversing over commit graphs

[filipnavara]

Follow-up to #1128. Only the last commit is relevant, rest will be disapper with rebase eventually.

This wires up the commit-graph API on the object layer. It introduces a concept of a lightweight CommitNode interface that is used to encapsulate either real Commit or a node loaded from the commit graph. A second interface called CommitNodeIndex is introduced to access the graph of CommitNode objects.

Two implementations of CommitNode and CommitNodeIndex are provided. One of them operates on top of storer.EncodedObjectStorer and regular git objects. The other one operates on commit-graph files and has fallback to storer.EncodedObjectStorer for cases where the commit-graph file doesn't cover the entire repository.

NewCommitNodeIterCTime iterator is added, analogous to NewCommitIterCTime, that can operate on the CommitNodeIndex implementations.

[filipnavara]

This mimics the interface from NewCommitIterCTime. As far as I can see ignore and seenExternal essentially serve the same purpose, perhaps with different performance characteristics. I'd be fine with keeping it this way or removing one of them.

[jfontan]

Is it possible to get the generation of the commit? This information is interesting to compare two commits or to stop walking at a certain level.

[filipnavara]

It is available at the format level, but I didn't propagate it here and it is not exposed through API at this level. I will look into it. One thing to consider though is that the Git folks are looking into replacing generation numbers with "corrected commit date". I will link to the relevent discussion once I get back to my computer...

[jfontan]

This? https://public-inbox.org/git/6367e30a-1b3a-4fe9-611b-d931f51effef@gmail.com/

[filipnavara]

Yep, V3 was the one I most recently read about. There's no final decision about any particular approach, but I didn't feel confident about designing API around something that is likely to change.

[filipnavara]

Looks like there are three things we can expose with a reasonable level of forward compatibility with the new proposals:

• Generation() uint64 API on CommitNode that will uphold the property that if A.Generation() < B.Generation() then B is unreachable by A. While the current format allows the generation to be only 30-bit integer extending it to 64-bit should account for any of the other proposals to work with the same API (except maybe the FELINE version), including the 34-bit corrected commit dates.
• A high-level API like IsUnreachableFrom(a, b CommitNode). The idea is to abstract away the concept of the generations from the user and just expose the most useful underlying property. This may need some thinking since we basically have three states: a) definitely unreachable (property coming from the generations) b) definitely reachable (not in the commit-graph, but perhaps something that is stored in reachability bitmap for some subset of commits?) c) we don't know.
• High-level API for priority heaps for traversing the graph in topological order by reusing the Generation property without directly exposing it.

Thoughts? I don't feel like implementing the last one since I currently have no use case for it, but maybe it would be useful for merge-base or something of that sort.

[filipnavara]

I didn't feel confident enough to establish API shape for two of the above options. I've added the Generation() method to CommitNode in a fashion similar to what git uses internally. It should be fairly easy to expand on it later.

[filipnavara]

Any news about this one? I have the next PR in the pipeline and this one is approved... /cc @mcuadros

[mcuadros]

I rather not add new objects for unstable APIs to the object package, maybe make sense add this to any package called commitgraph, and mark this as EXPERIMENTAL, I am not very sure about this API or where is going.

[filipnavara]

I was considering that. The reason I put it into the object package was that git itself has only one Commit object and it hides from the API user whether it was loaded from commit-graph or object storage. Unfortunately that's not possible to do in go-git without incompatible API changes because the current Commit struct exposes data as fields, not through methods.

I do consider this somewhat experimental which is one of the reasons why it's designed to be quite independent of the rest of the code.

As for where it's going I plan to write it up all the way to the Repository and provide methods for the following:

• Getting a CommitNodeIndex object that will transparently return either one using commit-graph, if available, or one using the object storage
• Writing a commit-graph file for existing repository.

Additionally I may rewrite the Log, Blame or other similar methods to use the new APIs and validate its shape.

[mcuadros]

I understand the situation that's why I rather keep this package and his usage limited to some packages until we reach the v5 of the library.

Meanwhile, we can place all the code in a package.
Otherwise will be hard to understand by the users.

[filipnavara]

Fine with me. The only problem with the commitgraph name is that it will inevitably clash with the name of the plumbing/fmt/commitgraph package and it will have to be aliased. If that's acceptable I will just do it.

[mcuadros]

The format is used inside of the object/commitgraph, so it's not a problem.

[mcuadros]

Awesome thanks! Can you add a doc.go file with the package documentation explaining the goal and also that is an experimental package (this implies that the API is also unstable), also will be great if you can add an example. Thanks!

[filipnavara]

Can you add a doc.go file with the package documentation explaining the goal and also that is an experimental package (this implies that the API is also unstable), also will be great if you can add an example.

Sure. I'll add the doc.go for both this and the plumbing/fmt/commitgraph package. The examples would make more sense once I add the high-level API but I will see what can I do.

[filipnavara]

I added basic documentation and one high-level usage example.

[filipnavara]

@mcuadros Anything else that is missing and needs to be addressed?