Added Caltech10k Web Faces dataset #324
Conversation
| "landmarks": {name: tf.int64 for name in LANDMARK_HEADINGS} | ||
| }), | ||
| urls=['http://www.vision.caltech.edu/Image_Datasets/Caltech_10K_WebFaces/'], | ||
| citation="" |
There was a problem hiding this comment.
The datasets must be have citation. Please add citation.
| """ | ||
| class Caltech10K_WebFaces(tfds.core.GeneratorBasedBuilder): | ||
|
|
||
| VERSION = tfds.core.Version("0.1.0") |
There was a problem hiding this comment.
Indentation should be 2 as in other files.
| "landmarks": {name: tf.int64 for name in LANDMARK_HEADINGS} | ||
| }), | ||
| urls=['http://www.vision.caltech.edu/Image_Datasets/Caltech_10K_WebFaces/'], | ||
| citation="" |
| name=tfds.Split.TRAIN, | ||
| num_shards=10, | ||
| gen_kwargs={ | ||
| "file_id": 0, |
There was a problem hiding this comment.
You didn't use file_id why returning that ?
There was a problem hiding this comment.
Forgot to remove it after updating logic
| name=tfds.Split.VALIDATION, | ||
| num_shards=4, | ||
| gen_kwargs={ | ||
| "extracted_dirs": extracted_dirs, |
There was a problem hiding this comment.
I don't understand. It seems that you are passing the same gen_kwargs to your validations and test set. So all train/valid/test sets will contains the same examples no ?
There was a problem hiding this comment.
@Conchylicultor some incorrect replication of logic on my end. This dataset doesn't have any original splits given, so will return a single tfds.Split.TRAIN split.
Fixed now
| from __future__ import print_function | ||
|
|
||
| import os | ||
| import tensorflow as tf |
There was a problem hiding this comment.
import tensorflow.compat.v2 as tf
| return [ | ||
| tfds.core.SplitGenerator( | ||
| name=tfds.Split.TRAIN, | ||
| num_shards=5, |
There was a problem hiding this comment.
remove - num_shards is no longer needed.
| for file_name in sorted(files): | ||
| path = os.path.join(filedir, file_name) | ||
|
|
||
| yield { |
There was a problem hiding this comment.
the S3 is the default now - so you have to yield a tuple: of (file_name and everything else)
|
|
||
| landmarks = self._process_caltech10k_config_file(landmarks_path) | ||
|
|
||
| for file_name in sorted(files): |
There was a problem hiding this comment.
nit: doesn't have to be sorted.
| row_values = line.strip().split() | ||
| # Each row start with the 'file_name' and then space-separated values. | ||
| values[row_values[0]] = [v for v in row_values[1:]] | ||
| return keys, values |
There was a problem hiding this comment.
please explain more what are the "keys" and "values" here..
keys should be the same as "LANDMARK_HEADINGS" right ?
| SPLITS = { | ||
| "train": 10, | ||
| } | ||
|
|
There was a problem hiding this comment.
does the test pass ? the CSV file is not present in the fake_examples dir.
cyfra
added
author:please_respond
Dataset Request #318
@rsepassi @cyfra @Conchylicultor