Support basic optimizer

[ShawnShawnYou]

• support optimize() on BasicSearcher #278
• baseline:

+-----------+------------+-----+----------+------------+----------------------------------------+-----------+-----+-----------------------------+-------------+----------------+-----------+
| Name      | NumVectors | Dim | DataType | MetricType | IndexParam                             | BuildTime | TPS | SearchParam                 | QPS         | LatencyAvg(ms) | RecallAvg |
+-----------+------------+-----+----------+------------+----------------------------------------+-----------+-----+-----------------------------+-------------+----------------+-----------+
| sq4u-fp32 | 1000000    | 960 | float32  | l2         | {"base_quantization_type":"sq4_unifor- | N/A       | N/A | {"hgraph":{"ef_search":80}} | 1000.059570 | 0.999940       | 0.860900  |
|           |            |     |          |            | m","build_thread_count":32,"ef_constr- |           |     |                             |             |                |           |
|           |            |     |          |            | uction":500,"max_degree":72,"precise_- |           |     |                             |             |                |           |
|           |            |     |          |            | io_type":"block_memory_io","precise_q- |           |     |                             |             |                |           |
|           |            |     |          |            | uantization_type":"fp32","use_reorder- |           |     |                             |             |                |           |
|           |            |     |          |            | ":true}                                |           |     |                             |             |                |           |
+-----------+------------+-----+----------+------------+----------------------------------------+-----------+-----+-----------------------------+-------------+----------------+-----------+

• running example:

[2025-04-22 16:11:59.162] [debug] created a hgraph index
[2025-04-22 16:11:59.167] [debug] train.shape: [1000000,960]
[2025-04-22 16:11:59.167] [debug] test.shape: [1000,960]
[2025-04-22 16:11:59.167] [debug] neighbors.shape: [1000,100]
[2025-04-22 16:12:06.081] [info] ============Start Optimize===========
[2025-04-22 16:12:25.314] [info] setting prefetch_depth_codes -> 1, get new loss = 6.428 from before = 6.354, improving = -1.156 %
[2025-04-22 16:12:31.458] [info] setting prefetch_depth_codes -> 2, get new loss = 6.113 from before = 6.354, improving = 3.795 %
[2025-04-22 16:12:37.719] [info] setting prefetch_depth_codes -> 3, get new loss = 6.230 from before = 6.354, improving = 1.957 %
[2025-04-22 16:12:43.543] [info] setting prefetch_depth_codes -> 4, get new loss = 5.793 from before = 6.354, improving = 8.823 %
[2025-04-22 16:12:49.729] [info] setting prefetch_depth_codes -> 5, get new loss = 6.154 from before = 6.354, improving = 3.156 %
[2025-04-22 16:12:55.482] [info] setting prefetch_depth_codes -> 6, get new loss = 5.722 from before = 6.354, improving = 9.948 %
[2025-04-22 16:13:01.755] [info] setting prefetch_depth_codes -> 7, get new loss = 6.241 from before = 6.354, improving = 1.782 %
[2025-04-22 16:13:06.819] [info] setting prefetch_depth_codes -> 8, get new loss = 5.032 from before = 6.354, improving = 20.804 %
[2025-04-22 16:13:06.819] [info] setting to best param: prefetch_depth_codes -> 8, improving 20.804%
[2025-04-22 16:13:17.205] [info] setting prefetch_stride_codes -> 1, get new loss = 5.091 from before = 5.232, improving = 2.701 %
[2025-04-22 16:13:21.871] [info] setting prefetch_stride_codes -> 2, get new loss = 4.635 from before = 5.232, improving = 11.416 %
[2025-04-22 16:13:26.601] [info] setting prefetch_stride_codes -> 3, get new loss = 4.698 from before = 5.232, improving = 10.206 %
[2025-04-22 16:13:31.125] [info] setting prefetch_stride_codes -> 4, get new loss = 4.493 from before = 5.232, improving = 14.125 %
[2025-04-22 16:13:35.767] [info] setting prefetch_stride_codes -> 5, get new loss = 4.611 from before = 5.232, improving = 11.876 %
[2025-04-22 16:13:40.326] [info] setting prefetch_stride_codes -> 6, get new loss = 4.528 from before = 5.232, improving = 13.451 %
[2025-04-22 16:13:44.553] [info] setting prefetch_stride_codes -> 7, get new loss = 4.196 from before = 5.232, improving = 19.813 %
[2025-04-22 16:13:49.087] [info] setting prefetch_stride_codes -> 8, get new loss = 4.503 from before = 5.232, improving = 13.927 %
[2025-04-22 16:13:53.656] [info] setting prefetch_stride_codes -> 9, get new loss = 4.538 from before = 5.232, improving = 13.268 %
[2025-04-22 16:13:58.162] [info] setting prefetch_stride_codes -> 10, get new loss = 4.475 from before = 5.232, improving = 14.478 %
[2025-04-22 16:13:58.162] [info] setting to best param: prefetch_stride_codes -> 7, improving 19.813%
[2025-04-22 16:14:06.966] [info] setting prefetch_stride_visit -> 1, get new loss = 4.278 from before = 4.464, improving = 4.162 %
[2025-04-22 16:14:11.539] [info] setting prefetch_stride_visit -> 2, get new loss = 4.541 from before = 4.464, improving = -1.714 %
[2025-04-22 16:14:16.007] [info] setting prefetch_stride_visit -> 3, get new loss = 4.438 from before = 4.464, improving = 0.584 %
[2025-04-22 16:14:20.211] [info] setting prefetch_stride_visit -> 4, get new loss = 4.173 from before = 4.464, improving = 6.514 %
[2025-04-22 16:14:24.599] [info] setting prefetch_stride_visit -> 5, get new loss = 4.357 from before = 4.464, improving = 2.404 %
[2025-04-22 16:14:28.624] [info] setting prefetch_stride_visit -> 6, get new loss = 3.995 from before = 4.464, improving = 10.507 %
[2025-04-22 16:14:32.914] [info] setting prefetch_stride_visit -> 7, get new loss = 4.259 from before = 4.464, improving = 4.593 %
[2025-04-22 16:14:37.322] [info] setting prefetch_stride_visit -> 8, get new loss = 4.376 from before = 4.464, improving = 1.967 %
[2025-04-22 16:14:41.607] [info] setting prefetch_stride_visit -> 9, get new loss = 4.254 from before = 4.464, improving = 4.705 %
[2025-04-22 16:14:46.026] [info] setting prefetch_stride_visit -> 10, get new loss = 4.389 from before = 4.464, improving = 1.685 %
[2025-04-22 16:14:46.027] [info] setting to best param: prefetch_stride_visit -> 6, improving 10.507%
[2025-04-22 16:14:46.027] [info] ============Optimize Report===========
[2025-04-22 16:14:50.349] [info] setting prefetch_stride_codes -> 7
[2025-04-22 16:14:50.349] [info] setting prefetch_stride_visit -> 6
[2025-04-22 16:14:50.349] [info] setting prefetch_depth_codes -> 8
[2025-04-22 16:14:50.349] [info] improving: 32.496%
[2025-04-22 16:14:50.349] [info] ============Finish Optimize===========
[2025-04-22 16:14:50.349] [info] hgraph Deserialize cost 168.330s
[2025-04-22 16:14:50.349] [debug] query count is 1000
+-----------+------------+-----+----------+------------+----------------------------------------+-----------+-----+-----------------------------+-------------+----------------+-----------+
| Name      | NumVectors | Dim | DataType | MetricType | IndexParam                             | BuildTime | TPS | SearchParam                 | QPS         | LatencyAvg(ms) | RecallAvg |
+-----------+------------+-----+----------+------------+----------------------------------------+-----------+-----+-----------------------------+-------------+----------------+-----------+
| sq4u-fp32 | 1000000    | 960 | float32  | l2         | {"base_quantization_type":"sq4_unifor- | N/A       | N/A | {"hgraph":{"ef_search":80}} | 1446.742065 | 0.691208       | 0.860900  |
|           |            |     |          |            | m","build_thread_count":32,"ef_constr- |           |     |                             |             |                |           |
|           |            |     |          |            | uction":500,"max_degree":72,"precise_- |           |     |                             |             |                |           |
|           |            |     |          |            | io_type":"block_memory_io","precise_q- |           |     |                             |             |                |           |
|           |            |     |          |            | uantization_type":"fp32","use_elp_opt- |           |     |                             |             |                |           |
|           |            |     |          |            | imizer":true,"use_reorder":true}       |           |     |                             |             |                |           |
+-----------+------------+-----+----------+------------+----------------------------------------+-----------+-----+-----------------------------+-------------+----------------+-----------+

[inabao]

LGTM

[codecov]

Codecov Report

Attention: Patch coverage is 95.50562% with 8 lines in your changes missing coverage. Please review.

@@            Coverage Diff             @@
##             main     #523      +/-   ##
==========================================
+ Coverage   91.31%   91.56%   +0.25%     
==========================================
  Files         182      186       +4     
  Lines       10870    11043     +173     
==========================================
+ Hits         9926    10112     +186     
+ Misses        944      931      -13     

Flag	Coverage Δ
cpp	91.56% <95.50%> (+0.25%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
common	93.28% <ø> (ø)
datacell	92.85% <90.00%> (+0.80%)	⬆️
index	89.40% <92.00%> (-0.01%)	⬇️
simd	100.00% <ø> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8407bb0...bcb1983. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wxyucs]

lgtm

[LHT129]

LGTM