Filtering by match key in comparison viewer #2667
-
|
Hi, It would be useful to filter by match_key when using the comparison viewer. Is there a way to achieve this, or would it be feasible to implement? On a similar note, is there a way to filter the results generated by, for example, match_key? Thank you |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 2 replies
-
|
+1 for a match_key focused viewer. I wrote a little metric for checking the spread of match weights over different match keys which can be useful for telling if a blocking rule can be safely removed when all edges it produces fall below a threshold. It creates a horrific visualisation though due to the number of match keys in my models, so a nice viewer dashboard would be useful. When you say "filter the results generated", do you mean from df_predict? You can put a WHERE condition on a query into that prediction table for |
Beta Was this translation helpful? Give feedback.
-
|
Thank you! Could you possibly share the syntax for putting the WHERE query in? |
Beta Was this translation helpful? Give feedback.
-
|
Unless I've misunderstood what you're after (quite possible, been a long Monday), you can filter the df_predict by match_key with a For example, the following would return only the edges allowed by the match key 0 (your first blocking rule). The match_keys are sequentially numbered as provided in the list within the settings dictionary, so you can loop over that list and corelate the edges with the appropriate blocking rule by their index. Note that this doesn't necessarily mean that a given match key is the best/tightest blocking rule for a given edge, as the ordering would make a difference in how to interpret them. I try to keep my blocking rules ordered tightest to loosest, and use fairly granular rules (some models have >20 rules) to make match keys quite focused. |
Beta Was this translation helpful? Give feedback.
-
|
That is exactly what I meant, thank you! There is a missing f-string marker in your line of code, in case anyone else stumbles across this and wants to try it, it should be:
|
Beta Was this translation helpful? Give feedback.
-
|
Good spot 😅 |
Beta Was this translation helpful? Give feedback.
+1 for a match_key focused viewer.
I wrote a little metric for checking the spread of match weights over different match keys which can be useful for telling if a blocking rule can be safely removed when all edges it produces fall below a threshold. It creates a horrific visualisation though due to the number of match keys in my models, so a nice viewer dashboard would be useful.
When you say "filter the results generated", do you mean from df_predict? You can put a WHERE condition on a query into that prediction table for
WHERE match_key =..., this helped me separate out different kinds of edges in my use case, but obviously then you need to keep tabs on which match_keys are which.