Add ai analysis demo data to the demo account#115
Conversation
|
🚅 Deployed to the reqcore-pr-115 environment in applirank
|
📝 WalkthroughWalkthroughAdds AI-driven scoring data and seeding logic to the demo seed flow and a new reset script to remove demo AI scoring and demo org/user; seeds criteria, per-application scores, analysis runs, an audit snapshot, and an encrypted demo AI config, then enables auto-score on the first job. Changes
Sequence Diagram(s)sequenceDiagram
participant Seed as "Seed Script"
participant Encrypt as "Encryption Util"
participant DB as "Postgres (drizzle-orm)"
participant Logger as "Console/Logs"
Note over Seed,DB: AI scoring seeding flow
Seed->>DB: Insert scoring criteria per job (JOB_*_CRITERIA)
DB-->>Seed: Criteria created (IDs)
Seed->>Encrypt: encrypt(demoApiKey, secret)
Encrypt-->>Seed: encryptedKey
Seed->>DB: Insert aiConfig with encryptedKey
DB-->>Seed: aiConfig persisted
loop per application in AI_SCORING_DATA
Seed->>DB: Insert CriterionScore records + AnalysisRun (rawResponse, token counts)
DB-->>Seed: Scores + AnalysisRun IDs
Seed->>DB: Create analysis audit with criteria snapshot & composite score
DB-->>Seed: Audit created
end
Seed->>DB: Update first job: set autoScoreOnApply = true
DB-->>Seed: Update acknowledged
Seed->>Logger: Log totals and demo cost
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@server/scripts/seed.ts`:
- Around line 504-510: The seed currently stores a hand-maintained
compositeScore in ApplicationScoringSeed and writes criterionScore rows,
application.score and analysisRun in separate statements (see the block around
lines 2127-2181), which can drift from computeCompositeScore(); change the seed
to remove/ignore the manual compositeScore and instead call the shared
computeCompositeScore() (from server/utils/ai/autoScore.ts) to derive the
composite, then persist the criterionScore rows, update application.score and
insert analysisRun inside a single database transaction so the three related
writes are atomic and reruns don't leave partial state.
- Around line 2187-2191: The seed turns on autoScoreOnApply for the first seeded
job (using jobIds and db.update(schema.job).set({ autoScoreOnApply: true })
where eq(schema.job.id, firstJobId)), which can trigger real AI calls; change
this so autoScoreOnApply remains false by default and only gets set when an
explicit env var is present (e.g., process.env.ENABLE_AUTO_SCORE_IN_SEEDS ===
'true'). Modify the block around firstJobId to check that env var before calling
db.update(schema.job).set(...), or simply remove the update to keep the flag off
in seeds; reference autoScoreOnApply, jobIds, db.update, and schema.job when
making the change.
| interface ApplicationScoringSeed { | ||
| jobIndex: number | ||
| candidateIndex: number | ||
| compositeScore: number | ||
| scores: CriterionScoreSeed[] | ||
| summary: string | ||
| } |
There was a problem hiding this comment.
Derive seeded composite scores from the rubric and persist them atomically.
Line 507 makes compositeScore another hand-maintained field, and Lines 2127-2181 then write the related score rows and analysisRun in separate statements. The first full-stack seed entry already drifts from the rubric — 10/9/10/9/9 over weights 70/55/40/50/30 is about 94, not 96 — so this can create demo data that computeCompositeScore() in server/utils/ai/autoScore.ts would never produce. Please compute the score once with the shared helper and persist criterionScore, application.score, and analysisRun in a single transaction; otherwise a mid-loop failure leaves a partial seed that reruns will skip.
Also applies to: 2127-2181
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@server/scripts/seed.ts` around lines 504 - 510, The seed currently stores a
hand-maintained compositeScore in ApplicationScoringSeed and writes
criterionScore rows, application.score and analysisRun in separate statements
(see the block around lines 2127-2181), which can drift from
computeCompositeScore(); change the seed to remove/ignore the manual
compositeScore and instead call the shared computeCompositeScore() (from
server/utils/ai/autoScore.ts) to derive the composite, then persist the
criterionScore rows, update application.score and insert analysisRun inside a
single database transaction so the three related writes are atomic and reruns
don't leave partial state.
| // Enable autoScoreOnApply on the Senior Full-Stack Engineer job to showcase the feature | ||
| const firstJobId = jobIds[0] | ||
| if (firstJobId) { | ||
| await db.update(schema.job).set({ autoScoreOnApply: true }).where(eq(schema.job.id, firstJobId)) | ||
| console.log(`✅ Enabled auto-score on apply for: ${JOBS_DATA[0]?.title}`) |
There was a problem hiding this comment.
Keep autoScoreOnApply off by default in seed data.
Lines 2187-2191 turn this on for a seeded public job. server/api/public/jobs/[slug]/apply.post.ts will call autoScoreApplication() on every new application when this flag is true, and server/utils/ai/autoScore.ts can then hit the external provider once AI config exists. That makes the seed quietly opt environments into real AI spend; gate it behind an env var or leave it disabled by default.
⚙️ Suggested guard
- const firstJobId = jobIds[0]
- if (firstJobId) {
+ const firstJobId = jobIds[0]
+ if (process.env.SEED_ENABLE_AUTO_SCORE_DEMO === 'true' && firstJobId) {
await db.update(schema.job).set({ autoScoreOnApply: true }).where(eq(schema.job.id, firstJobId))
console.log(`✅ Enabled auto-score on apply for: ${JOBS_DATA[0]?.title}`)
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Enable autoScoreOnApply on the Senior Full-Stack Engineer job to showcase the feature | |
| const firstJobId = jobIds[0] | |
| if (firstJobId) { | |
| await db.update(schema.job).set({ autoScoreOnApply: true }).where(eq(schema.job.id, firstJobId)) | |
| console.log(`✅ Enabled auto-score on apply for: ${JOBS_DATA[0]?.title}`) | |
| // Enable autoScoreOnApply on the Senior Full-Stack Engineer job to showcase the feature | |
| const firstJobId = jobIds[0] | |
| if (process.env.SEED_ENABLE_AUTO_SCORE_DEMO === 'true' && firstJobId) { | |
| await db.update(schema.job).set({ autoScoreOnApply: true }).where(eq(schema.job.id, firstJobId)) | |
| console.log(`✅ Enabled auto-score on apply for: ${JOBS_DATA[0]?.title}`) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@server/scripts/seed.ts` around lines 2187 - 2191, The seed turns on
autoScoreOnApply for the first seeded job (using jobIds and
db.update(schema.job).set({ autoScoreOnApply: true }) where eq(schema.job.id,
firstJobId)), which can trigger real AI calls; change this so autoScoreOnApply
remains false by default and only gets set when an explicit env var is present
(e.g., process.env.ENABLE_AUTO_SCORE_IN_SEEDS === 'true'). Modify the block
around firstJobId to check that env var before calling
db.update(schema.job).set(...), or simply remove the update to keep the flag off
in seeds; reference autoScoreOnApply, jobIds, db.update, and schema.job when
making the change.
JoachimLK
changed the title
1 participant
Summary
Type of change
Validation
DCO
Signed-off-by) viagit commit -sSummary by CodeRabbit
New Features
Chores