Module2_Step1b_Annotated_Images_proam_504_original+labeled_aug.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Module2_Step1b_Annotated_Images_proam_504_original+labeled_aug.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["## NOTEBOOK OBJECTIVE\n","The Objective of this notebook is to perform various augmentations (geometrical as well as color based) on the **504 professional+amateur tennis match frames** extracted in Module1_Step2 and whose tennis ball positions have been labeled in YOLO fromat using the opensource tool **labelimg**. We will be using the python package **abumentations** for video frame augmentation. Data Augmentation is expected to benefit the Deep Learning based YOLOV4-tiny Object Detection Model.\n","\n","Wherever there is a comment for **## UPDATE** the code (often a path) needs to be updated\n","\n","Libraries involved:\n","1. opencv-python-headless==4.5.5.62 \n","2. albumentations\n","3. os\n","4. shutil\n","5. random\n","6. zipfile\n","7. watermark\n","\n","Steps involved:\n","1. STEP1: Mounting Drive \n","2. STEP2: Installing & Importing Libraries and setting working directory\n","3. STEP3: Loading the Image Folder, cleaning the class and bounding box text file if needed (often the original bounding box would have multiple detected class IDs which need to be revised as we are only detecting a single class -the tennis ball)\n","4. STEP4. Zipping the original image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process\n","5. STEP5. Augmenting the original images using 3 albumenations based augmentations. Only labeled augmented images selected.\n","6. STEP6. Zipping the Augmented image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process\n","7. STEP7. Dependencies\n","\n","Inputs will include:\n","1. Set of labeled Image files\n","\n","Outputs will include:\n","1. Zipped file of labeled original images\n","2. Zipped file of labeled original+augmented images\n","\n","Source:\n","1. https://analyticsindiamag.com/hands-on-guide-to-albumentation/\n","2. https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/\n","3. https://github.com/tzutalin/labelImg"],"metadata":{"id":"MB5oJFBzIVND"}},{"cell_type":"markdown","source":["## STEP1: Mounting Drive"],"metadata":{"id":"UI0JHmHf3L3x"}},{"cell_type":"code","source":["from google.colab import drive\n","drive.mount('/content/drive')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"DKxF9be9Iw6N","executionInfo":{"status":"ok","timestamp":1650416690347,"user_tz":-330,"elapsed":25644,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"954b40a3-63b8-437e-acea-6d1a8d83ee6a"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}]},{"cell_type":"markdown","source":["## STEP2: Installing & Importing Libraries"],"metadata":{"id":"ujiMy4ln3PXq"}},{"cell_type":"code","source":["# Installing Libraries\n","\n","!pip uninstall opencv-python-headless==4.5.5.62 \n","!pip install opencv-python-headless==4.1.2.30\n","!pip install --upgrade albumentations\n","!pip install watermark"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"iyt-bGQ7KNb1","executionInfo":{"status":"ok","timestamp":1650416705749,"user_tz":-330,"elapsed":15417,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"c049aaf2-5d7f-48a4-b434-abf304184544"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["\u001b[33mWARNING: Skipping opencv-python-headless as it is not installed.\u001b[0m\n","Collecting opencv-python-headless==4.1.2.30\n","  Downloading opencv_python_headless-4.1.2.30-cp37-cp37m-manylinux1_x86_64.whl (21.8 MB)\n","\u001b[K     |████████████████████████████████| 21.8 MB 1.2 MB/s \n","\u001b[?25hRequirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/dist-packages (from opencv-python-headless==4.1.2.30) (1.21.6)\n","Installing collected packages: opencv-python-headless\n","Successfully installed opencv-python-headless-4.1.2.30\n","Requirement already satisfied: albumentations in /usr/local/lib/python3.7/dist-packages (0.1.12)\n","Collecting albumentations\n","  Downloading albumentations-1.1.0-py3-none-any.whl (102 kB)\n","\u001b[K     |████████████████████████████████| 102 kB 5.2 MB/s \n","\u001b[?25hRequirement already satisfied: numpy>=1.11.1 in /usr/local/lib/python3.7/dist-packages (from albumentations) (1.21.6)\n","Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from albumentations) (3.13)\n","Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from albumentations) (1.4.1)\n","Requirement already satisfied: opencv-python-headless>=4.1.1 in /usr/local/lib/python3.7/dist-packages (from albumentations) (4.1.2.30)\n","Requirement already satisfied: scikit-image>=0.16.1 in /usr/local/lib/python3.7/dist-packages (from albumentations) (0.18.3)\n","Collecting qudida>=0.0.4\n","  Downloading qudida-0.0.4-py3-none-any.whl (3.5 kB)\n","Requirement already satisfied: scikit-learn>=0.19.1 in /usr/local/lib/python3.7/dist-packages (from qudida>=0.0.4->albumentations) (1.0.2)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from qudida>=0.0.4->albumentations) (4.1.1)\n","Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (2021.11.2)\n","Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (3.2.2)\n","Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (2.4.1)\n","Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (1.3.0)\n","Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (2.6.3)\n","Requirement already satisfied: pillow!=7.1.0,!=7.1.1,>=4.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (7.1.2)\n","Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (2.8.2)\n","Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (3.0.8)\n","Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (0.11.0)\n","Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (1.4.2)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (1.15.0)\n","Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.19.1->qudida>=0.0.4->albumentations) (3.1.0)\n","Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.19.1->qudida>=0.0.4->albumentations) (1.1.0)\n","Installing collected packages: qudida, albumentations\n","  Attempting uninstall: albumentations\n","    Found existing installation: albumentations 0.1.12\n","    Uninstalling albumentations-0.1.12:\n","      Successfully uninstalled albumentations-0.1.12\n","Successfully installed albumentations-1.1.0 qudida-0.0.4\n","Collecting watermark\n","  Downloading watermark-2.3.0-py2.py3-none-any.whl (7.2 kB)\n","Collecting importlib-metadata<3.0\n","  Downloading importlib_metadata-2.1.3-py2.py3-none-any.whl (10 kB)\n","Requirement already satisfied: ipython in /usr/local/lib/python3.7/dist-packages (from watermark) (5.5.0)\n","Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata<3.0->watermark) (3.8.0)\n","Requirement already satisfied: pexpect in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (4.8.0)\n","Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (57.4.0)\n","Requirement already satisfied: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (0.7.5)\n","Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (1.0.18)\n","Requirement already satisfied: pygments in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (2.6.1)\n","Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (5.1.1)\n","Requirement already satisfied: simplegeneric>0.8 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (0.8.1)\n","Requirement already satisfied: decorator in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (4.4.2)\n","Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->watermark) (1.15.0)\n","Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->watermark) (0.2.5)\n","Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.7/dist-packages (from pexpect->ipython->watermark) (0.7.0)\n","Installing collected packages: importlib-metadata, watermark\n","  Attempting uninstall: importlib-metadata\n","    Found existing installation: importlib-metadata 4.11.3\n","    Uninstalling importlib-metadata-4.11.3:\n","      Successfully uninstalled importlib-metadata-4.11.3\n","\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n","markdown 3.3.6 requires importlib-metadata>=4.4; python_version < \"3.10\", but you have importlib-metadata 2.1.3 which is incompatible.\u001b[0m\n","Successfully installed importlib-metadata-2.1.3 watermark-2.3.0\n"]}]},{"cell_type":"code","source":["# Restarting post installing libraries -if needed\n","import os\n","os.kill(os.getpid(), 9)"],"metadata":{"id":"jOtBnGtu1AOs"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# importing libraries\n","import albumentations as A\n","import cv2\n","import os\n","import shutil\n","from zipfile import ZipFile\n","import random"],"metadata":{"id":"qvCQ2LwZIsNx"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# Set Directory path\n","\n","path=\"/content/drive/MyDrive/CAPSTONE/CAPSTONE_FINAL/Module2_Object_Detection_Yolov4_tiny\"  ## UPDATE\n","os.chdir(path) # Change Path"],"metadata":{"id":"XYdCmaAEBPSR"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## Ensure the following folder structure in the working directory where this notebooks is stored -->\n","\n","-Module2_Object_Detection_Yolov4_tiny\n","\n","--images_labels\n","\n","--INPUT_to_detection\n","\n","--OUTPUT_from_detection\n","\n","--SAVED_performance_charts\n","\n","--SAVED_training_weights\n","\n","--training\n","\n","--ZIPPED_images_labels"],"metadata":{"id":"CCP9ejDhi4Iu"}},{"cell_type":"markdown","source":["## STEP3: Loading the Image Folder, cleaning the class and bounding box text file if needed"],"metadata":{"id":"Bpg0LQl_RknW"}},{"cell_type":"markdown","source":["Manually copy the *obj_proam_504* folder (output of manual object labelling on selected extracted video frames from Module1_Step2 using labelimg ) from the BIG_Files_Folders folder to images_labels google drive folder link (https://drive.google.com/drive/folders/1-skl0_iiKYZDJPM6hhFxCqmW-cgqGh3q?usp=sharing) in Module2_Object_Detection_Yolov4_tiny folder."],"metadata":{"id":"1bR6aPCkKvGr"}},{"cell_type":"code","source":["# Copy initial image folder\n","source_fldr = r\"images_labels/obj_proam_504\" ## UPDATE\n","copy_folder=r\"images_labels/obj_proam_504_copy\"  ## UPDATE\n","try:\n","  shutil.copytree(source_fldr, copy_folder)\n","except:\n","  # delete previous folder if present\n","  shutil.rmtree(copy_folder)\n","  shutil.copytree(source_fldr, copy_folder)\n","\n","\n","# Open all files in the image folder\n","folder1=copy_folder\n","file_list_yolo=os.listdir(folder1)\n","\n","print(len(file_list_yolo))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ICz09XeAq10_","executionInfo":{"status":"ok","timestamp":1650416777026,"user_tz":-330,"elapsed":38175,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"0aa6886e-924b-4d2b-92e4-66406c222936"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["1009\n"]}]},{"cell_type":"code","source":["# Print available images and text files\n","image_file_list=[]\n","bbox_file_list=[]\n","\n","for fil in file_list_yolo[:]:\n","  if str(fil).split('.')[1]==\"jpg\":\n","    image_file_list.append(fil)\n","\n","  elif str(fil).split('.')[1]==\"txt\":\n","    bbox_file_list.append(fil)\n","\n","print('image_file_list')\n","print(len(image_file_list))\n","print(image_file_list[:5])\n","\n","print('-'*150)\n","print('-'*150)\n","print('bbox_file_list')\n","print(len(bbox_file_list))\n","print(bbox_file_list[:5])\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"VB7d8Gs91cvR","executionInfo":{"status":"ok","timestamp":1650416777028,"user_tz":-330,"elapsed":25,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"75af9677-c011-49f7-e6f2-af7b6e5f6935"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["image_file_list\n","504\n","['vid_pro_5_frame_2160.jpg', 'vid_pro_5_frame_2280.jpg', 'vid_pro_5_frame_1902.jpg', 'vid_pro_5_frame_1872.jpg', 'vid_pro_5_frame_1900.jpg']\n","------------------------------------------------------------------------------------------------------------------------------------------------------\n","------------------------------------------------------------------------------------------------------------------------------------------------------\n","bbox_file_list\n","505\n","['vid_am_4_frame_10680.txt', 'vid_am_4_frame_10020.txt', 'vid_am_4_frame_10290.txt', 'vid_am_4_frame_10140.txt', 'vid_am_4_frame_10740.txt']\n"]}]},{"cell_type":"code","source":["## Modify classes file and bbox file contents\n","\n","# Modifying classes text file\n","for fil in bbox_file_list:\n","  fil_path=folder1+\"/\"+str(fil)\n","  if str(fil).split('.')[0]==\"classes\":\n","    with open(fil_path,'r') as f:\n","      contents = f.readlines()\n","    contents='ball'\n","       \n","    with open(fil_path,'w') as f:\n","      f.writelines(contents)\n","\n","    \n","\n","\n","# Modifying bbox file\n","for fil in bbox_file_list[:]:\n","  fil_path=\"images_labels/obj_proam_504_copy\"+\"/\"+str(fil) ## UPDATE\n","  if str(fil).split('.')[0]!=\"classes\":\n","\n","    with open(fil_path,'r') as f:\n","      contents = f.readlines()\n","\n","    # # Update bbox file\n","    contents_2=[]\n","    for c1 in contents:\n","      if c1.split(' ')[0]=='15': # If the class is not updated while using labelimg, the additional class of ball becomes class 15\n","        c2=c1.split(' ')[1:]\n","        c2='0'+' '+ ' '.join(c2)\n","        contents_2.append(c2)\n","\n","      elif c1.split(' ')[0]=='0': # If the class is updated while using labelimg, the additional class of ball becomes class 15\n","        c2=c1.split(' ')[1:]\n","        c2='0'+' '+ ' '.join(c2)\n","        contents_2.append(c2)\n","\n","    #   # Update bbox file\n","    with open(fil_path,'w') as f:\n","      f.writelines(contents_2)\n","\n"],"metadata":{"id":"AsHO7jfx3aKh"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## STEP4. Zipping the original image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process"],"metadata":{"id":"LRVLZ6MR7vGu"}},{"cell_type":"code","source":["# Zipping the folder of original images\n","\n","# Zip the folder\n","full_folder=\"images_labels/obj_proam_504\" ## UPDATE\n","zip_folder=\"ZIPPED_images_labels/obj_proam_504_original_zip\" ## UPDATE\n","shutil.make_archive(zip_folder, 'zip',full_folder)\n","\n","# Original folder count\n","original_len=len(os.listdir(full_folder))\n","\n","\n","# Count contents in the zipped folder\n","with ZipFile(\"ZIPPED_images_labels/obj_proam_504_original_zip.zip\", 'r') as zipObj: ## UPDATE\n","   # Get list of files names in zip\n","  listOfiles = zipObj.namelist()\n","  zip_len=len(listOfiles)\n","\n","# Are the number of files pre and post zipping same\n","print ('Difference in file count pre and post zipping',original_len-zip_len)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Z6t4217M5f1c","executionInfo":{"status":"ok","timestamp":1650416795175,"user_tz":-330,"elapsed":12781,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"51221826-3d33-405d-9074-524a8327e93e"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Difference in file count pre and post zipping 0\n"]}]},{"cell_type":"markdown","source":["## STEP5. Augmenting the original images using 3 albumenations based augmentations. Only labeled augmented images selected.\n"],"metadata":{"id":"cWsSm3Yr8wpg"}},{"cell_type":"code","source":["## Setting random seed to ensure reproducibility of augmentation\n","random.seed(42)\n","\n","transform1 = A.Compose([\n","    A.RandomCrop(width=500, height=500),\n","   \n","], bbox_params=A.BboxParams(format='yolo', ))\n","\n","transform2 = A.Compose([\n","    \n","    A.HorizontalFlip(p=1),\n","    \n","], bbox_params=A.BboxParams(format='yolo', ))\n","\n","transform3 = A.Compose([\n","    \n","    A.RandomBrightnessContrast(p=1),\n","\n","], bbox_params=A.BboxParams(format='yolo',  ))\n","\n","\n","\n"],"metadata":{"id":"90vttqKZqTHe"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["%%time\n","# Save a set of Augmentations and BBoxes for Input Images\n","\n","# Create Folder and copy contents\n","source_fldr=\"images_labels/obj_proam_504\"  ## UPDATE\n","destination_fldr = r\"images_labels/obj_proam_504_aug\" ## UPDATE\n","try:\n","  shutil.copytree(source_fldr, destination_fldr)\n","except:\n","  # delete previous folder if present\n","  shutil.rmtree(destination_fldr)\n","  shutil.copytree(source_fldr, destination_fldr)\n","\n","\n","folder1=destination_fldr\n","\n","aug_lst=[transform1,transform2,transform3,\n","                 ]\n","\n","aug_lst_name=['transform1','transform2','transform3',\n","                ]\n","\n","for img in image_file_list[:]: \n","  bbox_file=img.split('.')[0]+'.txt'\n","  \n","  image = cv2.imread(folder1+\"/\"+img) \n","  image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n","\n"," \n","\n","  # Read Bbox\n","  bbox_file_path=folder1+\"/\"+bbox_file \n","  \n","  lst_of_lst=[[]]\n","  fil = open(bbox_file_path, 'r')\n","\n","  for line in fil:\n","    stripped_line = line.strip()\n","    line_list = stripped_line.split()\n","    lst_of_lst.append(line_list)\n","  fil.close()\n","\n","  # Pop 1st element in list of list\n","  lst_of_lst.pop(0)\n","\n","  # For each list in list of list pop 1st item\n","  lst_of_lst=[a[1:] for a in lst_of_lst]\n","\n","  # converting values to float\n","  lst_of_lst=[[float(a) for a in b] for b in lst_of_lst]\n"," \n","  # For each list in list of list append 'balls'\n","  abc=[]\n","\n","  for a in lst_of_lst:\n","    a.append('ball')\n","    abc.append(a) \n","\n","  lst_of_lst=abc\n","  bboxes=lst_of_lst\n","\n","\n","  for idx,val in enumerate(aug_lst[:]): # Update to blank\n","    transformed=val(image=image,bboxes=bboxes)\n","    transformed_image = transformed['image']\n","    transformed_bboxes = transformed['bboxes']\n","\n","\n","    aug_nam=aug_lst_name[idx]\n","\n","    # Save Transformed txt file\n","    save_txt_path=folder1+\"/\"+img.split('.')[0]+\"_\"+aug_nam+\".txt\"\n","  \n","    txt=''\n","    for s in transformed_bboxes:\n","      # Pop last item from set\n","      s=s[:-1]\n","\n","      s=' '.join(map(str,s))\n","      s='0 '+s\n","      txt+=s+'\\n'\n","\n","    # Only capturing the augmented images which have annotation, as augmentation could end up removing the part of the image with detected ball\n","    if len(transformed_bboxes)>0: \n","      # print('len(transformed_bboxes)>0')\n","       # Save Transformed image and text\n","      save_image_path=folder1+\"/\"+img.split('.')[0]+\"_\"+aug_nam+\".jpg\"\n","      cv2.imwrite(save_image_path, transformed_image)\n","      with open(save_txt_path, \"w\") as f:\n","        f.write(txt)\n","\n","\n","    \n","\n","\n","    \n","\n","  \n","\n","\n","\n","\n","  \n","\n","\n"],"metadata":{"id":"yoQyFHw1qUrr","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1650416881930,"user_tz":-330,"elapsed":86778,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"d05b5b21-7146-42e2-cb8e-637d9b957701"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["CPU times: user 46.6 s, sys: 3.17 s, total: 49.7 s\n","Wall time: 1min 27s\n"]}]},{"cell_type":"code","source":["import os\n","\n","folder_len=len(os.listdir(folder1))\n","\n","print('folder_len',folder_len)\n","\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ACi2Sa6eLr0V","executionInfo":{"status":"ok","timestamp":1650416881931,"user_tz":-330,"elapsed":9,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"27be6b7b-3019-42a4-c697-c170a60f2ad2"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["folder_len 3529\n"]}]},{"cell_type":"markdown","source":["## STEP6. Zipping the Augmented image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process"],"metadata":{"id":"z_j87e_o-Wo0"}},{"cell_type":"code","source":["# Zip the folder\n","full_folder=\"images_labels/obj_proam_504_aug\"  ## UPDATE\n","zip_folder=\"ZIPPED_images_labels/obj_proam_504_aug_zip\" ## UPDATE\n","shutil.make_archive(zip_folder, 'zip',full_folder)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"id":"MGdKv0Q64DwZ","executionInfo":{"status":"ok","timestamp":1650416923674,"user_tz":-330,"elapsed":41748,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"2be25ffc-d4b3-4855-b387-cf607bd63583"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'/content/drive/MyDrive/CAPSTONE/CAPSTONE_FINAL/Module2_Object_Detection_Yolov4_tiny/ZIPPED_images_labels/obj_proam_504_aug_zip.zip'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":12}]},{"cell_type":"code","source":["# Count contents in the zipped folder\n","from zipfile import ZipFile\n","\n","with ZipFile(\"ZIPPED_images_labels/obj_proam_504_aug_zip.zip\", 'r') as zipObj: # UPDATE\n","   # Get list of files names in zip\n","  listOfiles = zipObj.namelist()\n","  zip_len=len(listOfiles)\n","print(zip_len)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"TS8N7OgbZk-Q","executionInfo":{"status":"ok","timestamp":1650416923675,"user_tz":-330,"elapsed":23,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"b065e7f8-2e3e-4fcf-fae9-6fdca7d94a37"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["3529\n"]}]},{"cell_type":"code","source":["# Are the number of files pre and post zipping same\n","print ('Difference in file count pre and post zipping',folder_len-zip_len)"],"metadata":{"id":"LXtVN4MJaqcw","executionInfo":{"status":"ok","timestamp":1650416923676,"user_tz":-330,"elapsed":19,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"cb2447df-7722-481d-f30d-f03ff348dd4d"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Difference in file count pre and post zipping 0\n"]}]},{"cell_type":"code","source":["print('End of Augmentation for 504 proam Images')"],"metadata":{"id":"jVIjZYhnuiJ9","executionInfo":{"status":"ok","timestamp":1650416923677,"user_tz":-330,"elapsed":17,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"915b5273-419d-41b0-f786-d5d342563883"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["End of Augmentation for 504 proam Images\n"]}]},{"cell_type":"markdown","source":["## STEP7. Dependencies"],"metadata":{"id":"uImzHwh62-kv"}},{"cell_type":"code","source":["# Dependencies\n","%reload_ext watermark\n","%watermark\n","%watermark --iversions"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"sLkChGuk29hK","executionInfo":{"status":"ok","timestamp":1650416923677,"user_tz":-330,"elapsed":14,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"5194f41b-8423-45bc-9659-c67784113415"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Last updated: 2022-04-20T01:08:42.694674+00:00\n","\n","Python implementation: CPython\n","Python version       : 3.7.13\n","IPython version      : 5.5.0\n","\n","Compiler    : GCC 7.5.0\n","OS          : Linux\n","Release     : 5.4.144+\n","Machine     : x86_64\n","Processor   : x86_64\n","CPU cores   : 2\n","Architecture: 64bit\n","\n","albumentations: 1.1.0\n","IPython       : 5.5.0\n","google        : 2.0.3\n","cv2           : 4.1.2\n","\n"]}]}]}