Research Papers | ||
---|---|---|
Model Name | Description | Link to Paper and Git repo |
NA |
|
|
MOMENTA |
![]() Use of online google vision APIs for OCR, object detection, attribute detection. |
|
PromptHate |
|
|
Hate-CLIPper |
|
All the Datasets for OSPC AI Singapore can be found here. It contains the following datasets:
- Facebook harmful meme detection challenge dataset
- Total defence memes - singapore
- Palash's Sir Dataset (RMMHS)
- Propanganda Meme Dataset
-
Tesseract OCR (
tessdata_best
) : Takes around "1 hrs 30 mins" (2.9 it/s) for 1800 images. Quite slow !!! -
Tesseract OCR (
tessdata
) : Takes around "1 hr 10 mins" (2.3 it/s). Faster thantessdata_best
. -
In above two cases, turbo-boost was on. Now, turning off the turbo-boost, ran the
tessdata_best
on 272 images. Usingmultiprocessing.Pool(4)
, it took "8 mins 33 secs". Using simple for-loop, it takes ">20 mins". Usingmultiprocessing.Pool(3)
, it took "9 mins 06 secs". -
Scaling the time taken above to 1800 images, using multiprocess.Pool(4), it would take around "1 hr" only.
-
CLIP can handle images of size 224x224 upto 336x336.
-
Using HateCLIPper, the (auroc, acc) obtained on fb-meme data validation-set are:
- run_4_easyocr : (0.729, 0.614)
- run-3-easyocr : (0.739, 0.634)
- run-2-easyocr : (0.743, 0.646)
- run-1-easyocr : (0.740, 0.632)
- run-10 : (0.70, 0.656)
- run-9 : (0.733, 0.634)
- run-8 : (0.5, 0.5)
- run-7 : (0.737, 0.642)
- run-6 : (0.7408, 0.666)
-
Using HateCLIPper, the (auroc, acc) obtained on RMMHS data are:
- run_4_easyocr : (0.815, 0.68)
- run-3-easyocr : (0.843, 0.67)
- run-2-easyocr : (0.865, 0.75)
- run-1-easyocr : (0.87, 0.73)
- run-10 : (0.888, 0.789)
- run-9 : (0.854, 0.68)
- run-8 : (0.5, 0.45)
- run-7 : (0.847, 0.789)
- run-6 : (0.872, 0.835)
-
So, based on above data and charts from wandb, I decided to go with
run-1-easyocr
(run-9
was second best contender). -
Running the above model on translated val-set of fb-meme data, the (auroc, acc) obtained was (0.7456, 0.632)
-
Running the above model on translated val-set (sampling randomly 500) of fb-meme data, the (auroc, acc) obtained was (0.761, 0.722)