Store bitmap result directly

[alanbernstein]

Description

There is currently no way to compute a bitmap query and store it directly as a new row.

Example: Store(<bitmap_query>, frame=foo, rowID=10) -> {"results":[{"rowID": 10}]}
(maybe rowID input is optional)

Success criteria (What criteria will consider this ticket closeable?)

Any query that returns a bitmap result can be immediately written to a new frame+row, without returning the full bitmap result to the client.

[jaffee]

I think we need a concrete user story to motivate this. There are a lot of open questions around the implementation - i.e. what do you do if there is data in the row to be stored to, how do figure out what a good row to store to is, etc.

Going to icebox until we have a reason to priotitize.

[alanbernstein]

@travisturner pointed out a good use case: "group by"-like queries, or any situation with multiple repeated queries with small differences, would benefit from being able to store the static portion of a Bitmap query, and reuse the result within the "group by". In some of the bigger SSB queries, this might significantly cut back on repeated Intersects.

Also, we might call these things "registers".

[jaffee]

#1241 is a subset of this functionality - we will close that in favor of this.

Travis and I were just discussing a few possible use cases for this - there might be 3 different modes of operation, from most specific, to least:

Store the result in the specified field at row 10:
Store(<bitmap_query>, <fieldname>=10)

Store the result in the specified field in the first empty row.
Store(<bitmap_query>, <fieldname>)

Store the result in a new field, or perhaps a temp field allocated for this purpose:
Store(<bitmap_query>)

In all cases, the return value would be something like:

{
  "field": "fieldname",
  "row": 10,
}

[travisturner]

The first case (case 1) is pretty straight forward:

Store(<bitmap_query>, <fieldname>=10)

I think it's reasonable to implement this now.

The next two cases, however, introduce the need to safely allocate a rowID across the cluster.

Store(<bitmap_query>, <fieldname>)
Store(<bitmap_query>)

Rather than implement that logic again, it seems like we should leverage key translation. Of course, that requires that <fieldname> (case 2) and the temp field (case 3) support keys. Does anyone object to that approach?