Alternator: support Projection in GSI and LSI

[nyh]

When creating a GSI or LSI with CreateTable, the DynamoDB API provides a Projection parameter determining what gets copied to the materialized view: Just the keys, all the attributes, or the keys plus a few select attributes. We should support this parameter - currently we ignore it and always project all attributes.

Note that for LSI, there is a delicate issue: Even when only specific attributes are projected, DynamoDB allows to read also unprojected attributes (we verify this in test_lsi_get_not_projected_attribute). We can implement this either by projecting everything (as we do today) and not returning the unprojected columns when not specifically requested, or by doing a second query to get the unprojected columns. The second approach is probably closer to what DynamoDB does. Note that the second query is always on the same shard as the first one (same partition key).

We have existing xfailing tests for this issue:

1. In test_gsi.py: test_gsi_projection_keys_only, test_gsi_projection_include, test_gsi_missing_projection_type, test_gsi_query_select_2, test_gsi_projection_include_keyattributes, test_gsi_projection_include_otherkey, test_gsi_projection_error_missing_nonkeyattributes, test_gsi_projection_error_superflous_nonkeyattributes, test_gsi_projection_error_duplicate, test_gsi_projection_error_duplicate, test_gsi_bad_projection_type,
2. In test_lsi.py: test_lsi_get_all_projected_attributes, test_lsi_query_select.

And as mentioned above, test_lsi.py::test_lsi_get_not_projected_attribute needs to keep working and not get broken by the solution for LSIs.

[tzach]

or by doing a second query to get the unprojected columns.

AFAIU, this is what DynamoDB is doing
" However, to retrieve any additional attributes, DynamoDB must perform additional read operations against the Thread table. These extra reads are known as fetches, and they can increase the total amount of provisioned throughput required for a query."
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LSI.html#LSI.Projections

[nyh]

It will be significantly easier to implement KEYS_ONLY projection, because it just means not to include the ":attrs" in the view - no need to slice it. So we should start with implementing the KEYS_ONLY option.

[doublex]

@nyh
As a temporary workaround:
Is it possible to use an "ALTER MATERIALIZED VIEW" statement to simulate KEYS_ONLY?

[nyh]

@nyh As a temporary workaround: Is it possible to use an "ALTER MATERIALIZED VIEW" statement to simulate KEYS_ONLY?

CQL's ALTER MATERIALIZED VIEW cannot change the SELECTion of the view, so you cannot use it to get rid of the :attrs column in the view (which is what you'd want to do).
One hack that may still be possible without any code changes is to use CQL to DROP MATERIALIZED VIEW the view created by Alternator, and create a new one with CREATE MATERIALIZED VIEW whose SELECT doesn't include :attrs. This may work, but I didn't test it actually does. One of the questions is whether the code that reads from a view is ok with the fact there is no :attrs column in that table.

[doublex]

@nyh
Thank you for your assistance.
ScyllaDB does not allow to create the alternator materialized-view manually.
Alternator could create the appropriate materialized view.

E.g.:

base-table "data":
{
    primary_key: binary,
    si_hash_key: binary,
    si_range_key: binary,
    payload: binary,
}
CREATE MATERIALIZED VIEW alternator_data."data:index" AS
    SELECT primary_key, si_hash_key, si_range_key
    FROM alternator_data.data
    WHERE si_hash_key IS NOT NULL AND si_range_key IS NOT NULL
    PRIMARY KEY (si_hash_key, si_range_key, primary_key)
    WITH CLUSTERING ORDER BY (si_range_key ASC, primary_key ASC);

Error message, docs: https://opensource.docs.scylladb.com/stable/cql/mv.html
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot include more than one non-primary key column 'si_range_key' in materialized view primary key"

The create materialzed viewstatement was taken from DESCRIBE MATERIALIZED VIEW alternator_data."data:index";

[mykaul]

It will be significantly easier to implement KEYS_ONLY projection, because it just means not to include the ":attrs" in the view - no need to slice it. So we should start with implementing the KEYS_ONLY option.

Agreed, looks like an easy win with high 'profit' - but I don't know how common is the use of KEYS_ONLY.