Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ai: Add a node embedding watcher #27204

Merged
merged 18 commits into from
Jun 21, 2023
Merged

ai: Add a node embedding watcher #27204

merged 18 commits into from
Jun 21, 2023

Commits on Jun 9, 2023

  1. ai: add embeddings basic support 

    - add Embeddings service and its local implementation
- add Embedding type and proto message
- add nodeEmbeddingCollector tracking nodes
- add NodeEmbeddingWatcher watching for events adn sending them to the
  collector
- add the Embedder interface and its openai implementation
    @hugoShaka
    hugoShaka committed Jun 9, 2023
    Configuration menu
    Copy the full SHA
    418a592 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2023

  1. ai: adapt embeddings to the vector index

    @hugoShaka
    hugoShaka committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    82de770 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into hugo/ai-embedding-watcher

    @hugoShaka
    hugoShaka committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    be800cf View commit details
    Browse the repository at this point in the history

  3. fixup! ai: adapt embeddings to the vector index

    @hugoShaka
    hugoShaka committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    56f07f7 View commit details
    Browse the repository at this point in the history

  4. fixup! fixup! ai: adapt embeddings to the vector index

    @hugoShaka
    hugoShaka committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    a45d571 View commit details
    Browse the repository at this point in the history

  5. Update lib/service/service.go 

    Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
    @hugoShaka @jakule
    hugoShaka and jakule committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    9a39a5f View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2023

  1. address feedback pt.1

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    a2103e2 View commit details
    Browse the repository at this point in the history

  2. address feedback pt.2: store protobuf message in backend

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    002bb8c View commit details
    Browse the repository at this point in the history

  3. address feedback pt.3: have GetEmbeddings return a stream

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    8d3d703 View commit details
    Browse the repository at this point in the history

  4. Update lib/services/embeddings.go 

    Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
    @hugoShaka @rosstimothy
    hugoShaka and rosstimothy committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    e02c251 View commit details
    Browse the repository at this point in the history

  5. Apply suggestions from code review 

    Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
    @hugoShaka @espadolini
    hugoShaka and espadolini committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    04cf97a View commit details
    Browse the repository at this point in the history

  6. address feedback pt.4: extract embedding logic out of Embeddings service

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    15186dc View commit details
    Browse the repository at this point in the history

  7. fixup! address feedback pt.4: extract embedding logic out of Embeddin… 

    …gs service
    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    98eab77 View commit details
    Browse the repository at this point in the history

  8. address feedback pt.5: simpler error handling when embedding fails

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    f8c86cc View commit details
    Browse the repository at this point in the history

  9. fix tests pt.1

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    6a7d87e View commit details
    Browse the repository at this point in the history

  10. fix tests pt.2

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    5ec0661 View commit details
    Browse the repository at this point in the history

  11. fix tests pt.3

    @hugoShaka
    hugoShaka committed Jun 14, 2023
    Configuration menu
    Copy the full SHA
    5cbd303 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2023

  1. [Assist] Replace embedding watcher (#27953) 

    Change the way how the embeddings are calculated. Instead of creating a watcher in Auth, we will process all nodes every hour and process embeddings if any embeddings are missing or any node has been updated.
    @jakule
    jakule committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    3895105 View commit details
    Browse the repository at this point in the history