Summary:
This diff fixes two issues that could block vector index backfill forever:
1) Vector index backfill uses the same thread pool as vector index insert.
It could happen that all threads in this thread pool are used by backfill.
In this case insert cannot happen until backfill is done.
But when there are already too many insert tasks pending, backfill cannot proceed
since it is waiting to allocate new insert tasks.
In other words backfill waits for inserts while insert waits for backfill.
Fixed by using separate thread pool for backfill.
2) We limit number of insert tasks in particular vector index using vector_index_max_insert_tasks flag.
When there is very big insert batch, we could get into situation when number of tasks for single batch is greater than
vector_index_max_insert_tasks.
It results in infinite waiting for ability to allocate more tasks than allowed by vector_index_max_insert_tasks.
Fixed by adjusting wait logic. If there are no allocated tasks, then we could allocate any number of insert tasks.
Jira: DB-15690
Original commit: 20072e2791218fcb32b269a54a4da428daf0d845 / D42443
Test Plan: PgVectorIndexTest.ManyRowsWithBackfill/*
Reviewers: arybochkin, slingam, #db-approvers
Reviewed By: arybochkin, slingam
Subscribers: svc_phabricator, yql, ybase
Tags: #jenkins-ready
Differential Revision: https://phorge.dev.yugabyte.com/D42470