wip messing around with sentence similarity #4989
Conversation
|
|
||
| const docEmbeds = await extractor(funcs.map(f => f[0] + ": " + f[1].doc.slice(0, 512)), { pooling: 'mean', normalize: true }) // need to check max length. takes a few seconds | ||
|
|
||
| probe = 'easter egg' |
There was a problem hiding this comment.
results: "hint", "wintitle", "snow_mouse_mode", "neo_mouse_mode", "escapehatch", "back", "markjumplocal", "yankimage", "markjump", "forward"
for "version control": "issue", "credits", "updatecheck", "nativeinstall", "updatenative", "containerupdate", "mktridactylrc", "firefoxsyncpull", "setnull", "markaddlocal"
|
Worth checking if we can speed up search with a nearest neighbour index https://unum-cloud.github.io/usearch/javascript/ I think the slowness is in the embedding of the search term but I could be wrong Edit: that library needs wasm. I can't get my head around it, |
For our data, it is much, much slower than just searching all the data. And gets worse results. Committing because I might want the reshape function again at some point
This reverts commit 5ba8438.
Closes #936 if ever finished
thoughts:
irrelevant, bizarre, totally irrelevant, THING YOU ARE LOOKING FOR, irrelvant, kooky... but given that:aproposfor the same pattern often gives zero results, maybe that's useful. example: searching for "version control" gives you lots of random stuff but ~5th among them is mktridactylrc which is what you probably want. Which you might not have found otherwise