added cookbook

LAION-AI · Mar 4, 2023 · b141e50 · b141e50
1 parent 19a9339
commit b141e50
Show file tree

Hide file tree

Showing 4 changed files with 810 additions and 101 deletions.
diff --git a/data/datasets/nsfw_csam_reddit/.gitignore b/data/datasets/nsfw_csam_reddit/.gitignore
@@ -1,2 +1,3 @@
 dataframes/*.csv
 comments_cache/*.csv
+*.csv
diff --git a/data/datasets/nsfw_csam_reddit/README.md b/data/datasets/nsfw_csam_reddit/README.md
@@ -0,0 +1,6 @@
+# NSFW - CSAM from Reddit
+
+Note(TODO): this is the pipeline, will need to scale this dataset by getting
+data from file.pushshift.io
+The scripts and notebooks in the directory are used to create the NSFW and CSAM
+dataset from Reddit that can be used to train the safety model.