[Job Launcher] GCV moderation: Batches processing by 0xVoronov · Pull Request #3081 · humanprotocol/human-protocol

0xVoronov · 2025-02-07T11:52:06Z

Issue tracking

This PR addresses issues related to job moderation processing and result collection.
Tracking: [Job Launcher] GCV integration
#2905.

Context behind the change

This PR introduces batch processing for large datasets, ensuring that datasets with more than 2000 images are split into multiple tasks for better processing efficiency.

Changes made:

Implemented batch processing:

If a dataset contains more than 2000 images, it is split into smaller tasks to improve parallel processing.
Added a new migration:
Introduced the job-moderation-tasks table to store individual task details.

How has this been tested?

Unit Tests:
- Tested valid and invalid moderation results.
- Ensured missing files throw expected errors.
Manual Testing:
- Triggered moderation flow with various scenarios (successful, partial passed, fully passed).

Release plan

Remove all data from cron-jobs table.
Run the new migration before deploying to create the job-moderation-tasks table.

…gcv-moderation-multiple-tasks

vercel · 2025-02-07T11:52:12Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
human-app	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 26, 2025 0:04am
human-dashboard-frontend	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 26, 2025 0:04am
staking-dashboard	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 26, 2025 0:04am

2 Skipped Deployments

Name	Status	Preview	Comments	Updated (UTC)
faucet-frontend	⬜️ Ignored (Inspect)	Visit Preview		Feb 26, 2025 0:04am
faucet-server	⬜️ Ignored (Inspect)	Visit Preview		Feb 26, 2025 0:04am

portuu3

I think job moderation should be in a different module.

portuu3 · 2025-02-07T14:20:35Z

 export const JOB_MODERATION_ASYNC_BATCH_SIZE = 100;
+export const JOB_MODERATION_BATCH_SIZE_PER_TASK = 2000;


I would add GCV to the names of these constants, just in case we ever add other external service for moderation

portuu3 · 2025-02-07T14:26:19Z

+        `);
+    await queryRunner.query(`
+            ALTER TABLE "hmt"."job-moderation-tasks"
+            ALTER COLUMN "job_id" DROP NOT NULL


portuu3 · 2025-02-10T11:33:52Z

    }

-    this.logger.log('Parse job moderation results START');
+    this.logger.log('Process job moderation tasks START');


Suggested change

this.logger.log('Process job moderation tasks START');

this.logger.log('Parse job moderation results START');

portuu3 · 2025-02-10T12:15:13Z

+          this.logger.error(
+            `Error process job moderation task. Error ID: ${errorId}, Job ID: ${jobModerationTaskEntity.id}, Reason: ${failedReason}, Message: ${err.message}`,
+          );
+          jobModerationTaskEntity.status = JobModerationTaskStatus.FAILED;


it is missing a notification

portuu3 · 2025-02-10T12:23:07Z


-  @Cron('*/3 * * * *')
+  @Cron('*/2 * * * *')
+  public async processJobModerationTasksCronJob() {


I think this cron job is unnecessary. As far as I can see we have 4 cron jobs:

Create moderation task entity.

Send the data for moderation.

Parse results.

Send notification.

At least 1 and 2 could be combined.

portuu3

I think job moderation should have its own module

…update environment variables, and remove deprecated job moderation tasks.

…service tests

…gcv-moderation-multiple-tasks

…ce for better control over mock behavior

…ssary console logs and ensure consistent output format

… a new gcv.ts file and updating imports accordingly

portuu3 · 2025-02-26T12:00:42Z

+    it('should do nothing if requests already exist', async () => {
+      const jobEntity = {
+        id: faker.number.int(),
+        status: JobStatus.PAID,


Job entity can be defined globally inside ensureRequests or maybe even for the whole test file.
Check this for other mocked entities

portuu3 · 2025-02-26T14:41:55Z

+      }
+
+      jobEntity.contentModerationRequests = [
+        ...(jobEntity.contentModerationRequests || []),


this will always be null, we are checking if the job has requests in line 92 and returning the function

portuu3 · 2025-02-27T09:04:21Z

+      (fileName) => `${gcDataUrl}/${fileName.split('/').pop()}`,
+    );
+
+    const fileName = getFileName(


rename to a more descriptive name

portuu3 · 2025-02-27T09:08:49Z

+    const outputUri = constructGcsPath(
+      this.visionConfigService.moderationResultsBucket,
+      this.visionConfigService.moderationResultsFilesPath,
+      hashString(fileName),


it makes no sense to hash the file name

portuu3 · 2025-02-27T09:11:44Z

+   */
+  private async collectModerationResults(fileName: string) {
+    try {
+      const hash = hashString(fileName);


unnecessary

* Implemented job moderation service * Implemented storage * Added updates * Updated names, updated migration * Added migration file * Improved job moderation flow, added new unit tests * Implemented async batch annotation logic * Resolved comments, fixed dependencies, updated tests * Implemented new GCS and Job Moderation utils * [Job Launcher] GCV moderation: Batches processing (#3081) * Implemented batch processing logic * Added entity, repository and migration * Updated job moderation service * Implemented unit tests * Refactor content moderation module: rename and restructure entities, update environment variables, and remove deprecated job moderation tasks. * Add @faker-js/faker dependency, improve test cases, and clean up job service tests * Improved content moderation tests * Faker ussage for Content Moderation tests * Refactor GCVContentModerationService tests to use mockResolvedValueOnce for better control over mock behavior * Fix GCV content moderation tests * Refactor GCS URL conversion and validation functions to remove unnecessary console logs and ensure consistent output format * Refactor content moderation enums by moving ContentModerationLevel to a new gcv.ts file and updating imports accordingly * Refactor categorize methods and clean some useless code --------- Co-authored-by: Francisco López <francislopez977@gmail.com> Co-authored-by: portuu3 <adrian.portugues.mas@gmail.com> * feat: add node-cache for caching GCS object listings and remove wrong migration * Update gcv-content-moderation.service.ts --------- Co-authored-by: Francisco López <francislopez977@gmail.com> Co-authored-by: portuu3 <adrian.portugues.mas@gmail.com> Co-authored-by: Francisco López <50665615+flopez7@users.noreply.github.com>

0xVoronov added 5 commits February 6, 2025 12:29

Implemented batch processing logic

8f9804c

Added entity, repository and migration

728d1fa

Merge branch 'feature/launcher/gcv-moderation' into feature/launcher/…

101bc7f

…gcv-moderation-multiple-tasks

Updated job moderation service

b346c67

Implemented unit tests

e54cffb

0xVoronov self-assigned this Feb 7, 2025

0xVoronov requested a review from portuu3 February 7, 2025 11:52

portuu3 requested changes Feb 10, 2025

View reviewed changes

Refactor content moderation module: rename and restructure entities, …

ede3569

…update environment variables, and remove deprecated job moderation tasks.

vercel Bot deployed to Preview – human-dashboard-frontend February 20, 2025 12:17 View deployment

vercel Bot deployed to Preview – human-app February 20, 2025 12:20 View deployment

vercel Bot deployed to Preview – staking-dashboard February 20, 2025 12:25 View deployment

Add @faker-js/faker dependency, improve test cases, and clean up job …

8ca08b5

…service tests

vercel Bot deployed to Preview – human-app February 21, 2025 14:34 View deployment

vercel Bot deployed to Preview – human-dashboard-frontend February 21, 2025 14:38 View deployment

vercel Bot deployed to Preview – staking-dashboard February 21, 2025 14:42 View deployment

Merge branch 'feature/launcher/gcv-moderation' into feature/launcher/…

19077e4

…gcv-moderation-multiple-tasks

vercel Bot temporarily deployed to Preview – human-app February 21, 2025 15:05 Inactive

vercel Bot deployed to Preview – human-dashboard-frontend February 21, 2025 15:16 View deployment

vercel Bot deployed to Preview – faucet-server February 21, 2025 15:19 View deployment

vercel Bot deployed to Preview – staking-dashboard February 21, 2025 15:24 View deployment

Improved content moderation tests

e60fb33

vercel Bot deployed to Preview – human-dashboard-frontend February 21, 2025 17:00 View deployment

vercel Bot deployed to Preview – staking-dashboard February 21, 2025 17:02 View deployment

vercel Bot deployed to Preview – human-app February 21, 2025 17:06 View deployment

Faker ussage for Content Moderation tests

e2d6170

vercel Bot deployed to Preview – human-dashboard-frontend February 24, 2025 12:54 View deployment

vercel Bot deployed to Preview – human-app February 24, 2025 12:55 View deployment

vercel Bot deployed to Preview – staking-dashboard February 24, 2025 12:57 View deployment

Refactor GCVContentModerationService tests to use mockResolvedValueOn…

1ffd7f7

…ce for better control over mock behavior

vercel Bot deployed to Preview – human-dashboard-frontend February 25, 2025 13:45 View deployment

vercel Bot deployed to Preview – staking-dashboard February 25, 2025 13:47 View deployment

vercel Bot deployed to Preview – human-app February 25, 2025 13:48 View deployment

Fix GCV content moderation tests

e52cd49

vercel Bot deployed to Preview – human-app February 25, 2025 15:01 View deployment

vercel Bot deployed to Preview – staking-dashboard February 25, 2025 15:03 View deployment

vercel Bot deployed to Preview – human-dashboard-frontend February 25, 2025 15:05 View deployment

portuu3 assigned flopez7 and unassigned 0xVoronov Feb 26, 2025

portuu3 reviewed Feb 26, 2025

View reviewed changes

Comment thread packages/apps/job-launcher/server/src/common/enums/content-moderation.ts Outdated

Refactor GCS URL conversion and validation functions to remove unnece…

59710d4

…ssary console logs and ensure consistent output format

vercel Bot deployed to Preview – staking-dashboard February 26, 2025 11:56 View deployment

Refactor content moderation enums by moving ContentModerationLevel to…

bc7004d

… a new gcv.ts file and updating imports accordingly

vercel Bot deployed to Preview – human-app February 26, 2025 12:00 View deployment

vercel Bot deployed to Preview – staking-dashboard February 26, 2025 12:02 View deployment

vercel Bot deployed to Preview – human-dashboard-frontend February 26, 2025 12:04 View deployment

flopez7 requested review from portuu3 February 26, 2025 12:09

portuu3 requested changes Feb 26, 2025

View reviewed changes

portuu3 requested changes Feb 27, 2025

View reviewed changes

Refactor categorize methods and clean some useless code

9a7177c

portuu3 approved these changes Mar 6, 2025

View reviewed changes

portuu3 merged commit 6670dee into feature/launcher/gcv-moderation Mar 6, 2025

portuu3 deleted the feature/launcher/gcv-moderation-multiple-tasks branch March 6, 2025 12:45

		export const JOB_MODERATION_ASYNC_BATCH_SIZE = 100;
		export const JOB_MODERATION_BATCH_SIZE_PER_TASK = 2000;

	this.logger.log('Process job moderation tasks START');
	this.logger.log('Parse job moderation results START');

Conversation

0xVoronov commented Feb 7, 2025

Issue tracking

Context behind the change

Changes made:

How has this been tested?

Release plan

Uh oh!

vercel Bot commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

portuu3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

portuu3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vercel Bot commented Feb 7, 2025 •

edited

Loading