This is an unofficial implementation of the 3MDAD dataset. Reference:
Imen Jegham, Anouar Ben Khalifa, Ihsen Alouani, Mohamed Ali Mahjoub, A novel public dataset for multimodal multiview and multispectral driver distraction analysis: 3MDAD, Signal Processing: Image Communication, Volume 88, October 2020, 115966, DOI: https://doi.org/10.1016/j.image.2020.115960.
The dataset is a multimodal multiview driver distraction detection dataset. that contains the following activities:
A1: Safe driving,
A2: Doing hair and makeup,
A3: Adjusting radio,
A4: GPS operating,
A5: Writing message using right hand,
A6: Writing message using left hand,
A7: Talking phone using right hand,
A8: Talking phone using left hand,
A9: Having picture,
A10: Talking to passenger,
A11: Singing or dancing,
A12: Fatigue and somnolence,
A13: Drinking using right hand,
A14: Drinking using left hand,
A15: Reaching behind,
A16: Smoking.
Download the dataset here
For each modality and view, download the zip files in a folder in the following format "{}_view_{}_data".format(view, time)
, where view is either side or front, time is either day or night according to the dataset. Create a zip folder under each category folder that contain the zip files. The dataset should look like the following:
3MDAD
front_view_day_data
zips
Depth2.rar
RGB2.rar
front_view_night_data
side_view_day_data
side_view_night_data
Annotations 3MDAD.rar
Run the script to extract 3MDAD dataset (please read the next before doing so):
python dataset/unzip_3MDAD.py --datadir /path_to_3MDAD
We have noticed some corrupted data from this data. The provided unzip_3MDAD.py will fix everything below. However, it's recommended you check out the code before running it to decide which conflict you want to solve.
3MDAD/front_view_night_data/Depth2/S6/AC2/DN2_SUB6ACT2F234.tiff
3MDAD/front_view_day_data/Depth2/S2/AC7/Vid_dep_S2_AC7.mj2
3MDAD/side_view_day_data/Depth1/S31/AC10 and AC9 should be swapped.
mv 3MDAD/side_view_day_data/Depth1/S31/AC10 3MDAD/side_view_day_data/Depth1/S31/AC9_
mv 3MDAD/side_view_day_data/Depth1/S31/AC9 3MDAD/side_view_day_data/Depth1/S31/AC10
mv 3MDAD/side_view_day_data/Depth1/S31/AC9_ 3MDAD/side_view_day_data/Depth1/S31/AC9
3MDAD also includes a list of .mat files in the annotations that encode the hand & face locations. To convert all .mat files to .csv files, upload the .mat files to Matlab and run convert_csv_3MDAD.m in matlab.
sub_id_lst: a list of subject ids. e.g. [1,2,3,4,5].
time: string. Choose from day, night
modality: a list of modalities to include in the dataset. e.g. [RGB, Depth]
bbox_gt: Boolean. Include bbox in the ground truth
view: a list of views from side, front. Can be [side, front] for multiview fusion
dataset_folder: parent dataset folder
video_fps: Int. if given, will sample 3D video tensors based on this fps. Otherwise, sample images.
video_sample_per_action: how many sample videos are extracted for a video
exhaustive_video_sampling: sample all continuous video clips based on video_fps and number of frames in video
# video training
threeMDAD = ThreeMDADDataset([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], time="day", view=["side"], modality=["RGB", "Depth"],
bbox_gt=False, dataset_folder="/home/lang/Data/project/3MDAD/", transform=None,
video_fps=16, exhaustive_video_sampling=True)