feat: prepare embedder for Production #9517

mattkrick · 2024-03-08T03:16:54Z

Description

Refactors the embedder to handle production workload.

Out of scope

detect when there's not a enough data, or the data is not english
consider adversarial models to grade a set of teamIds
handle priority (e.g. process on feature flag or teamId, etc)

Demo
https://www.loom.com/share/e2d9d43d6d924676bb8a3cce84edb336?sid=82842bf2-68c5-42e5-b4f5-3c7f482b8e0b

How it works:

1 worker from the cluster becomes primary and builds the tables, triggers, and historical metadata
when a meeting ends, the gql executor sends a message to the embedder via redis streams so only 1 worker gets the job to add the discussion to metadata table
When a row is inserted into the metadata table, it also gets added to the job queue table. This happens via trigger. there is 1 trigger per embedding model
The embedder polls the PG job queue table to get a job. it creates fullText and embeds that. it then writes that embedding to the model's table.
Can add items to the metadata by adhoc query passing in startAt and/or endAt
every 5 mins the primary worker resets stalled embedding jobs

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-08T03:17:15Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

packages/server/postgres/migrations/1709860378296_embeddingsMetadataId.ts

mattkrick · 2024-03-08T03:22:27Z

packages/embedder/ai_models/AbstractModel.ts

+          "id" INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+          "embedText" TEXT,
+          "embedding" vector(${sql.raw(vectorDimensions.toString())}),
+          "embeddingsMetadataId" INTEGER UNIQUE NOT NULL,


@jordanh safe to add the unique constraint here?

Yes, I think that is ok. It would imply that if we ever have multiple versions of the same embedding models, they'll need to live on their own table which I think is what we'd want

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-13T18:09:05Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-13T21:21:25Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-18T21:40:21Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-19T02:16:05Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

github-actions · 2024-03-19T15:58:33Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

github-actions · 2024-03-19T16:00:12Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-19T20:39:46Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-19T21:03:13Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

github-actions · 2024-03-19T21:53:12Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

rafaelromcar-parabol · 2024-03-26T15:51:44Z

Updating a readme whenever the version changes isn't great. While looking over that readme again, the build section is basically just build.yml in markdown, right? Can we remove that section & just say refer to the build.yml to see how to build it?

PR #9558

I answered the PR. As it is a different PR and we will be discussing it over there, I suggest you do the changes I proposed, just to be consistent with the current state of things, and we then see what to do with the future.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-27T02:01:33Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-27T17:30:27Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-27T22:57:51Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-27T23:16:47Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-27T23:46:55Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-28T00:18:53Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-28T00:31:35Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-28T00:35:32Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-28T19:43:09Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

github-actions · 2024-03-28T23:35:17Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions · 2024-03-29T00:10:41Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR will be delayed and might be rejected due to its size.

mattkrick added 3 commits March 7, 2024 14:09

feat: support ad-hoc metadata inserts

d86eb2d

Signed-off-by: Matt Krick <matt.krick@gmail.com>

feat: triggers from metadata to queue

8628e3e

Signed-off-by: Matt Krick <matt.krick@gmail.com>

feat: move job queue to pg

285f69b

Signed-off-by: Matt Krick <matt.krick@gmail.com>

github-actions bot added the size/xl label Mar 8, 2024

mattkrick commented Mar 8, 2024

View reviewed changes

packages/server/postgres/migrations/1709860378296_embeddingsMetadataId.ts Outdated Show resolved Hide resolved

mattkrick commented Mar 8, 2024

View reviewed changes

mattkrick added 2 commits March 13, 2024 10:14

begin retry

0b024dd

feat: support retries after queue goes dry

ae6126f

Signed-off-by: Matt Krick <matt.krick@gmail.com>

chore: separate abstract models into their own files

9fb9c4f

Signed-off-by: Matt Krick <matt.krick@gmail.com>

mattkrick added 4 commits March 13, 2024 19:08

feat: support recursive text splitting for chunks

20f5d96

Signed-off-by: Matt Krick <matt.krick@gmail.com>

validate discussions to only include ended meetings

eff8eb9

Signed-off-by: Matt Krick <matt.krick@gmail.com>

support historical info

7ec3d6a

Signed-off-by: Matt Krick <matt.krick@gmail.com>

Merge branch 'master' into feat/embedder1

49c3eae

feat: support listening to jobs from app

46dafac

Signed-off-by: Matt Krick <matt.krick@gmail.com>

mattkrick added 2 commits March 19, 2024 08:56

feat: support calls from app

47e83d9

merge master

7cee59e

fix migration conflict

5c96c41

fix: import history

49b349b

Signed-off-by: Matt Krick <matt.krick@gmail.com>

fix: dataloader mem leak

bf9a379

Signed-off-by: Matt Krick <matt.krick@gmail.com>

feat: handle stalled jobs

f7ccdc6

feat: custom text splitter

59b0647

Signed-off-by: Matt Krick <matt.krick@gmail.com>

feat: add language to metadata table

3faa442

Signed-off-by: Matt Krick <matt.krick@gmail.com>

feat: support multiple workers

2ab9f61

Signed-off-by: Matt Krick <matt.krick@gmail.com>

feat: set workers via env var

7fc4136

Signed-off-by: Matt Krick <matt.krick@gmail.com>

fix: handle shutdown and stalled jobs

ea32be7

Signed-off-by: Matt Krick <matt.krick@gmail.com>

update readme for parabol-ubi

bc1cf74

Signed-off-by: Matt Krick <matt.krick@gmail.com>

remove unused trigger logic

7d82b28

Signed-off-by: Matt Krick <matt.krick@gmail.com>

turn on dev servers

b609f3e

Signed-off-by: Matt Krick <matt.krick@gmail.com>

mattkrick requested review from jordanh and rafaelromcar-parabol March 28, 2024 00:37

mattkrick added 2 commits March 28, 2024 12:13

Merge branch 'master' into feat/embedder1

8679d2c

fix: rename migration

0c80dd6

Signed-off-by: Matt Krick <matt.krick@gmail.com>

Merge branch 'master' into feat/embedder1

826a044

Signed-off-by: Matt Krick <matt.krick@gmail.com>

mattkrick force-pushed the feat/embedder1 branch from ce8c1b4 to 826a044 Compare March 29, 2024 00:10

mattkrick merged commit 538c95c into master Mar 29, 2024
6 checks passed

mattkrick deleted the feat/embedder1 branch March 29, 2024 00:19

parabol-release-bot bot mentioned this pull request Mar 29, 2024

chore(release): release v7.24.0 #9581

Merged

github-actions bot mentioned this pull request Apr 1, 2024

chore(release): Test v7.24.0 #9583

Closed

24 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: prepare embedder for Production #9517

feat: prepare embedder for Production #9517

mattkrick commented Mar 8, 2024 •

edited

github-actions bot commented Mar 8, 2024

mattkrick Mar 8, 2024

jordanh Mar 8, 2024

github-actions bot commented Mar 13, 2024

github-actions bot commented Mar 13, 2024

github-actions bot commented Mar 18, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

rafaelromcar-parabol commented Mar 26, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 29, 2024

feat: prepare embedder for Production #9517

feat: prepare embedder for Production #9517

Conversation

mattkrick commented Mar 8, 2024 • edited

Description

github-actions bot commented Mar 8, 2024

mattkrick Mar 8, 2024

Choose a reason for hiding this comment

jordanh Mar 8, 2024

Choose a reason for hiding this comment

github-actions bot commented Mar 13, 2024

github-actions bot commented Mar 13, 2024

github-actions bot commented Mar 18, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Mar 19, 2024

rafaelromcar-parabol commented Mar 26, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 27, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 28, 2024

github-actions bot commented Mar 29, 2024

mattkrick commented Mar 8, 2024 •

edited