High level context
Benchmark RLI for tables on an existing table with large number of record keys (~100B).
Incrementally ingest about 10GB of data, MoR table, partitioned with ~500 partitions.
Use Hfile size of 2GB.
- Ensure the bootstrap of RLI works as expected.
- Measure the read and write latencies for the RLI index
- Find and measure all bottlenecks
- Report any issue with the core indexing or RLI or MDT DAGs.
JIRA info
High level context
Benchmark RLI for tables on an existing table with large number of record keys (~100B).
Incrementally ingest about 10GB of data, MoR table, partitioned with ~500 partitions.
Use Hfile size of 2GB.
JIRA info