Approximate selection #107
Conversation
|
@ddkang Could you please review this PR? |
Sorry, I didn't state clearly. For example, we have a query 'select blob_id, entity_id from entities00 where type LIKE 'EVENT' '. Because one derived row is 'EVENT', another is 'ORG'. Suppose blob 0 is the cluster representative, should we set true score of blob 0 as 0.5 or 1? |
|
@ddkang Could you please review this PR? |
aidb/engine/approx_select_engine.py
Outdated
| dataset = self.get_sampled_proxy_blob(proxy_score_for_all_blobs) | ||
|
|
||
| # This is used for parallel test | ||
| seed = (mp.current_process().pid * np.random.randint(100000, size=1)[0]) % (2**32 - 1) |
There was a problem hiding this comment.
Can you pass in the seed as an argument to the base engine? I don't think it's good to set here
aidb/engine/approx_select_engine.py
Outdated
|
|
||
| logger.info(f'num_samples: {len(additional_samples)}') | ||
|
|
||
| additional_satisfied_sampled_results, additional_all_sampled_results = await self.get_inference_results( |
There was a problem hiding this comment.
Don't we also want to take the positive records from the pilot sample?
There was a problem hiding this comment.
Yes, we have taken the positive records from the pilot sample. The variable names are ambiguous, I will change them.
aidb/aidb/engine/approx_select_engine.py
Lines 212 to 215 in 4c67799
|
@ddkang Could you please review the new commit? |
https://github.com/ddkang/aidb/wiki/Apporx-selection-%E2%80%90-Design-Document