Skip to content

Don't provide closure retrieve mode in simple bulk API #201

Closed

Description

@enikao

For bulk API retrieve operation, we discussed to have a closure mode.
This mode would include the transitive closure of all requested nodes, i.e. the nodes themselves, all their descendants + annotations, and all (directly or indirectly) contained, annotating, or referenced nodes.

All other variants of the retrieve operation only look at children or descendants, thus they always stay within the same partition.
This variant could "cross over" into a different partition, as we can refer to nodes anywhere in the repository.

We do not include parents of nodes into the transitive closure, as including the parents, and their descendants, would effectively include big parts of the repository in lots of cases.

However, then we could end up with a result that's hard to interpret: Imagine partition A contains class C with a Method M.
M contains a MethodCall c that refers to method P. P is contained in class Q, which in turn is part of partition B.
It is hard to make sense of P without knowing Q (and maybe even B).

The main reason for closure mode is to avoid repetitive calls from the client. This is a known antipattern from SQL: retrieve something from table a, then for each of them, retrieve something related from table b, etc. SQL solves this with joins. But SQL has a rich language to specify such operations -- conceptually similar to #39.

In our case, there are two acceptable alternatives:

  1. Our whole repository is not that big in the first place. Then we can query everything, and be sure all references are in there.
    (We still need to deal with dead references, they can occur).
  2. We can batch the repeated queries: 1/ retrieve the first set of trees 2/ find all unresolved references 3/ retrieve the trees for all of them /4 repeat from (2) until all interesting references are resolved.
    In practice, this approach should terminate quickly. If not, we're dealing with a complex model, and need to be smarter about references anyway.

Because of the inconsistency with other variants, the questionable applicability, and the available alternatives, we won't have a closure retrieve mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions