FIW Data Development Kit

Table of Contents

• Introduction
• Families In the Wild Database
• Publications
• Organization
• To Do
• License
• Getting Involved

---

Introduction

Python tools for automatic kinship recognition in images and videos.Python tools for automatic kinship recognition in images and videos.

• This is the documentation of the visual kinship recognition toolbox and tools specific to the FIW dataset (i.e., FIW data development kit). If you want the Places-CNN models instead of the training data, please refer to the FIW-models.
• Demos for Recognizing Families In the Wild (RFIW) 5th Edition:
  • Track 1, Verification;
  • Track 2, Tri-Subject Verification;
  • Track 3, Search & Retrieval.

This repo is a work in progress and all contributions and requests are welcome.

In summary, the following items are available:

• Version and contact information, download links, along with a brief description of the different download options.

• Overview of the API, its purpose, contents, and premiere features.

• Image data details for FIW-Standard and RFIW-Challenge.

  1. Image list and annotations
  2. Submission format
  3. Evaluation routines

• Overview of the FIW data development kit.

• List of action items (tentative; open to requests and PRs)

This repo serves as the main set of tools for kinship recognition effort, including the FIW database. Besides, the next section is detailed description of database (i.e., data and label) structure.

Please contact Joseph Robinson robinson.jo@northeastern.edu for questions, comments, or bug reports.

Download data and learn more about it here https://web.northeastern.edu/smilelab/fiw/.

---

Families In the Wild Database

FIW can be obtained from two primary locations: the main dataset (i.e., raw data, experimental splits, and more) downloads page, along with task-specific data splits on codalab (i.e., Task 1, Task 2, and Task 3), which were at one time used for data challenge (i.e., 2020 RFIW in conjunction with the IEEE FG Conference). Oncce download, we suggest to decompress the files in the data to their own folder.

FIW Data and Labels

This documentation describes FIW DB and (working) development kit. This is work in progress (i.e., still to come are FIW-CNN models, updated benchmarks, more in README (this), and more).

Check out FIW project page

Download

Download here

DB Contents and Structure

• FIW_PIDs.csv: Photo lookup table. Each row is an image instance, containing the following fields:

  • PID: Photo ID
  • Name: Surname.firstName (root reference for given family)
  • URL: Photo URL on web
  • Metadata: Text caption for photo

• FIW_FIDs.csv: FID (family)/ Surname lookup table.

  • FID: Unique ID key assigned to each family.
  • Surname: Family Name corresponding to FID key.

• FIW_RIDs.csv: Relationship lookup table with keys [1-9] assigned to relationship types.

• FIDs/

  • FID####/ Contains labels and cropped facial images for members of family (1-1000)
    • MID#/: Face images of family member with ID key , i.e., MID #.
    • F####.csv: File containing member information of each family:
      • relationships matrix: representation of relationships.
      • name: First name of family member.
      • gender: gender of family member.

For example:

FID0001.csv

	MID     1     2     3     Name    Gender
	 1      0     4     5     name1     F
	 2      1     0     1     name2     F
	 3      5     4     0     name3     M

Here we have 3 family members, as listed under the MID column (far-left). Each MID reads acorss its row.

We can see that MID1 is related to MID2 by 4->1 (Parent->Sibling), which of course can be viewed as the inverse, i.e., MID2->MID1 is 1->4. It can also be seen that MID1 and MID3 are Spouses of one another, i.e., 5->5. And so on, and so forth.

Statistics

Publications

Papers on FIW describe the data collection processes and details; supplemental to this is the FIW Data Card below. Note that the Latex source file for the datasheet could be borrowed as a tempalate for another dataset of similar structure. Check out repo, as well as DatasheetForFiw/main.pdf.

2021

Joseph P. Robinson, Ming Shao, and Yun Fu. Survey on the Analysis and Modeling of Visual Kinship: A Decade in the Making. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).

PDF

@ARTICLE{robinsonSurvey2021,
  author={Robinson, Joseph Peter and Shao, Ming and Fu, Yun},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  title={Survey on the Analysis and Modeling of Visual Kinship: A Decade in the Making},
  year={2021},
  pages={1-1},
  doi={10.1109/TPAMI.2021.3063078},
  }

Joseph P. Robinson, Yu Yin, Zaid Khan, Ming Shao, and Yun Fu. Families In Wild Multimedia (FIW MM): A Multimodal Database for Recognizing Kinship. IEEE Transactions on Multimedia (2021).

PDF

@ARTICLE{robinsonfiwmm,
  author={Robinson, Joseph Peter and Yin, Yu and Khan, Zaid and Shao, Ming and Fu, Yun},
  journal={IEEE Transactions on Multimedia (TMM)},
  title={Families In Wild Multimedia (FIW MM): A Multimodal Database for Recognizing Kinship},
  year={2021},
  }

2020

Joseph P. Robinson, Yu Yin, Zaid Khan, Ming Shao, Siyu Xia, Michael Stopa, Samson Timoner, Matthew A. Turk, Rama Chellappa, and Yun Fu. Recognizing Families In the Wild (RFIW): The 4th Edition IEEE International Conference on Automatic Face & Gesture Recognition.

PDF

@article{robinson2020recognizing,
    title={Recognizing Families In the Wild (RFIW): The 4th Edition},
    author={Robinson, Joseph P and Yin, Yu and Khan, Zaid and Shao, Ming and Xia, Siyu and
            Stopa, Michael and Timoner, Samson and Turk, Matthew A and Chellappa, Rama and Fu, Yun},
    journal={arXiv preprint arXiv:2002.06303},
    year={2020}
}

2018

Joseph P. Robinson, Ming Shao, Yue Wu, Hongfu Liu, Timothy Gillis, and Yun Fu. Visual Kinship Recognition of Families in the Wild. IEEE International Conference on Automatic Face & Gesture Recognition (2018).

PDF

@article{robinson2018visulkinship,
	title={Visual Kinship Recognition of Families in the Wild},
	author={Robinson, Joseph P and Shao, Ming and Wu, Yue and Liu, Hongfu and Gillis, Timothy and Fu, Yun},
	journal={IEEE Transactions on pattern analysis and machine intelligence (TPAMI) Special Issue: Computational Face},
	year={2020}
}

2016

Joseph P. Robinson, Ming Shao, Yue Wu, Yun Fu. Families in the Wild (FIW): Large-scale Kinship Image Database and Benchmarks IEEE International Conference on Automatic Face & Gesture Recognition (2016).

PDF

@article{robinson2016families,
  title="Visual Kinship Recognition of Families in the Wild",
  author="Robinson, Joseph P and Shao, Ming and Wu, Yue and Liu, Hongfu and Gillis, Timothy and Fu, Yun",
  journal="ACM on Multimedia Conference",
  year="2016"
}

A more complete list of references can be found here.

---

Organization

License

By downloading the image data you agree to the following terms:

1. You will use the data only for non-commercial research and educational purposes.
2. You will NOT distribute the above images.
3. Northeastern University makes no representations or warranties regarding the data, including but not limited to warranties of non-infringement or fitness for a particular purpose.
4. You accept full responsibility for your use of the data and shall defend and indemnify Northeastern University, including its employees, officers and agents, against any and all claims arising from your use of the data, including but not limited to your use of any copies of copyrighted images that you may create from the data.

See Download links (and Terms and Conditions) here.

Version

0.1.0 Created: 16 January 2020

Authors

• Joseph Robinson - Github - web
• Zaid Khan - Github

Bugs and Issues

Please bring up any questions, comments, bugs, PRs, etc.

---

To Do

• Finish documentation
• Demo for Track 1
• Demo for Track 2
• Demo for Track 3
• Dataloader
• Publish results (all baselines)

Disclaimer

If you found our data and resources useful please cite our works [above]().

---

Project based on the cookiecutter data science project template. #cookiecutterdatascience