Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multisearch #37

Open
Vandivier opened this issue Jul 2, 2023 · 2 comments
Open

multisearch #37

Vandivier opened this issue Jul 2, 2023 · 2 comments

Comments

@Vandivier
Copy link
Owner

this can be a premium feature

basic users can access 1 or 2 kinds of search but premium users can access 3 or 4, among:

  1. Text Search: Use GitHub, Google, and other tools to search the Ladderly site, repo, and curriculum files specifically. Also good ol' cmd+F on the curriculum page...blog this, so multisearch depends on the blog #27 feature
  2. Similarity Search using pgvector
  3. Material Search: Text and Similarity versions. in this form of advanced search, text and similarity search are applied to source material and proprietary intermediary material (PIM) rather than simply to the public inputs and outputs. see abstract content, variant, and PIM entities #35

do we want an outputs-only repo? or can we search just fine with the outputs being inside the ladderly-slides files+folders?

@Vandivier
Copy link
Owner Author

pgvector needs to be a higher priority because we know we have too much info for even claude to handle
i would put this at equal priority compared to finetuning; mainly bc idk the performance/cost comparison of these two approaches, so let's do em both...we can also look for research on perf between them.

@Vandivier
Copy link
Owner Author

so premium search is:

  1. finetuned model
  2. pgvector material search
  3. keyword search

anyone can use the claude search as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant