Module2_Step1a_Annotated_Images_proam_256_original+labeled_aug.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Module2_Step1a_Annotated_Images_proam_256_original+labeled_aug.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["## NOTEBOOK OBJECTIVE\n","The Objective of this notebook is to perform various augmentations (geometrical as well as color based) on the **256 professional+amateur tennis match frames** extracted in Module1_Step2 and whose tennis ball positions have been labeled in YOLO fromat using the opensource tool **labelimg**. We will be using the python package **abumentations** for video frame augmentation. Data Augmentation is expected to benefit the Deep Learning based YOLOV4-tiny Object Detection Model.\n","\n","Wherever there is a comment for **## UPDATE** the code (often a path) needs to be updated\n","\n","Libraries involved:\n","1. opencv-python-headless==4.5.5.62 \n","2. albumentations\n","3. os\n","4. shutil\n","5. random\n","6. zipfile\n","7. watermark\n","\n","Steps involved:\n","1. STEP1: Mounting Drive \n","2. STEP2: Installing & Importing Libraries and setting working directory\n","3. STEP3: Loading the Image Folder, cleaning the class and bounding box text file if needed (often the original bounding box would have multiple detected class IDs which need to be revised as we are only detecting a single class -the tennis ball)\n","4. STEP4. Zipping the original image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process\n","5. STEP5. Augmenting the original images using 3 albumenations based augmentations. Only labeled augmented images selected.\n","6. STEP6. Zipping the Augmented image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process\n","7. STEP7. Dependencies\n","\n","Inputs will include:\n","1. Set of labeled Image files\n","\n","Outputs will include:\n","1. Zipped file of labeled original images\n","2. Zipped file of labeled original+augmented images\n","\n","Source:\n","1. https://analyticsindiamag.com/hands-on-guide-to-albumentation/\n","2. https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/\n","3. https://github.com/tzutalin/labelImg"],"metadata":{"id":"MB5oJFBzIVND"}},{"cell_type":"markdown","source":["## STEP1: Mounting Drive"],"metadata":{"id":"UI0JHmHf3L3x"}},{"cell_type":"code","source":["from google.colab import drive\n","drive.mount('/content/drive')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"DKxF9be9Iw6N","executionInfo":{"status":"ok","timestamp":1650414492549,"user_tz":-330,"elapsed":3507,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"30e8ff0c-0009-4865-fdab-62571a210e8c"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"]}]},{"cell_type":"markdown","source":["## STEP2: Installing & Importing Libraries"],"metadata":{"id":"ujiMy4ln3PXq"}},{"cell_type":"code","source":["# Installing Libraries\n","\n","!pip uninstall opencv-python-headless==4.5.5.62 \n","!pip install opencv-python-headless==4.1.2.30\n","!pip install --upgrade albumentations\n","!pip install watermark"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"iyt-bGQ7KNb1","executionInfo":{"status":"ok","timestamp":1650414567011,"user_tz":-330,"elapsed":16833,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"ab283fe7-5766-4df4-b53e-f7e06d4ac514"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Found existing installation: opencv-python-headless 4.1.2.30\n","Uninstalling opencv-python-headless-4.1.2.30:\n","  Would remove:\n","    /usr/local/lib/python3.7/dist-packages/cv2/*\n","    /usr/local/lib/python3.7/dist-packages/opencv_python_headless-4.1.2.30.dist-info/*\n","  Would not remove (might be manually added):\n","    /usr/local/lib/python3.7/dist-packages/cv2/.libs/libQtCore-bbdab771.so.4.8.7\n","    /usr/local/lib/python3.7/dist-packages/cv2/.libs/libQtGui-903938cd.so.4.8.7\n","    /usr/local/lib/python3.7/dist-packages/cv2/.libs/libQtTest-1183da5d.so.4.8.7\n","Proceed (y/n)? y\n","  Successfully uninstalled opencv-python-headless-4.1.2.30\n","Collecting opencv-python-headless==4.1.2.30\n","  Using cached opencv_python_headless-4.1.2.30-cp37-cp37m-manylinux1_x86_64.whl (21.8 MB)\n","Requirement already satisfied: numpy>=1.14.5 in /usr/local/lib/python3.7/dist-packages (from opencv-python-headless==4.1.2.30) (1.21.6)\n","Installing collected packages: opencv-python-headless\n","Successfully installed opencv-python-headless-4.1.2.30\n","Requirement already satisfied: albumentations in /usr/local/lib/python3.7/dist-packages (1.1.0)\n","Requirement already satisfied: numpy>=1.11.1 in /usr/local/lib/python3.7/dist-packages (from albumentations) (1.21.6)\n","Requirement already satisfied: opencv-python-headless>=4.1.1 in /usr/local/lib/python3.7/dist-packages (from albumentations) (4.1.2.30)\n","Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from albumentations) (1.4.1)\n","Requirement already satisfied: scikit-image>=0.16.1 in /usr/local/lib/python3.7/dist-packages (from albumentations) (0.18.3)\n","Requirement already satisfied: qudida>=0.0.4 in /usr/local/lib/python3.7/dist-packages (from albumentations) (0.0.4)\n","Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from albumentations) (3.13)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from qudida>=0.0.4->albumentations) (4.1.1)\n","Requirement already satisfied: scikit-learn>=0.19.1 in /usr/local/lib/python3.7/dist-packages (from qudida>=0.0.4->albumentations) (1.0.2)\n","Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (2021.11.2)\n","Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (2.4.1)\n","Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (2.6.3)\n","Requirement already satisfied: pillow!=7.1.0,!=7.1.1,>=4.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (7.1.2)\n","Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (1.3.0)\n","Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image>=0.16.1->albumentations) (3.2.2)\n","Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (3.0.8)\n","Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (1.4.2)\n","Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (2.8.2)\n","Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (0.11.0)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib!=3.0.0,>=2.0.0->scikit-image>=0.16.1->albumentations) (1.15.0)\n","Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.19.1->qudida>=0.0.4->albumentations) (1.1.0)\n","Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.19.1->qudida>=0.0.4->albumentations) (3.1.0)\n","Requirement already satisfied: watermark in /usr/local/lib/python3.7/dist-packages (2.3.0)\n","Requirement already satisfied: importlib-metadata<3.0 in /usr/local/lib/python3.7/dist-packages (from watermark) (2.1.3)\n","Requirement already satisfied: ipython in /usr/local/lib/python3.7/dist-packages (from watermark) (5.5.0)\n","Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata<3.0->watermark) (3.8.0)\n","Requirement already satisfied: pygments in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (2.6.1)\n","Requirement already satisfied: pexpect in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (4.8.0)\n","Requirement already satisfied: simplegeneric>0.8 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (0.8.1)\n","Requirement already satisfied: decorator in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (4.4.2)\n","Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (5.1.1)\n","Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (1.0.18)\n","Requirement already satisfied: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (0.7.5)\n","Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.7/dist-packages (from ipython->watermark) (57.4.0)\n","Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->watermark) (1.15.0)\n","Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->watermark) (0.2.5)\n","Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.7/dist-packages (from pexpect->ipython->watermark) (0.7.0)\n"]}]},{"cell_type":"code","source":["# Restarting post installing libraries -if needed\n","import os\n","os.kill(os.getpid(), 9)"],"metadata":{"id":"jOtBnGtu1AOs"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# importing libraries\n","import albumentations as A\n","import cv2\n","import os\n","import shutil\n","from zipfile import ZipFile\n","import random"],"metadata":{"id":"qvCQ2LwZIsNx"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# Set Directory path\n","\n","path=\"/content/drive/MyDrive/CAPSTONE/CAPSTONE_FINAL/Module2_Object_Detection_Yolov4_tiny\"  ## UPDATE\n","os.chdir(path) # Change Path"],"metadata":{"id":"XYdCmaAEBPSR"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## Ensure the following folder structure in the working directory where this notebooks is stored -->\n","\n","-Module2_Object_Detection_Yolov4_tiny\n","\n","--images_labels\n","\n","--INPUT_to_detection\n","\n","--OUTPUT_from_detection\n","\n","--SAVED_performance_charts\n","\n","--SAVED_training_weights\n","\n","--training\n","\n","--ZIPPED_images_labels"],"metadata":{"id":"nAkbm7bLjBrp"}},{"cell_type":"markdown","source":["## STEP3: Loading the Image Folder, cleaning the class and bounding box text file if needed"],"metadata":{"id":"Bpg0LQl_RknW"}},{"cell_type":"markdown","source":["Manually copy the *obj_proam_256* folder (output of manual object labelling on selected extracted video frames from Module1_Step2 using labelimg ) from the BIG_Files_Folders folder to images_labels google drive folder link (https://drive.google.com/drive/folders/1-skl0_iiKYZDJPM6hhFxCqmW-cgqGh3q?usp=sharing) in Module2_Object_Detection_Yolov4_tiny folder."],"metadata":{"id":"PwxQvTSgJ3Z8"}},{"cell_type":"code","source":["# Copy initial image folder\n","source_fldr = r\"images_labels/obj_proam_256\" ## UPDATE\n","copy_folder=r\"images_labels/obj_proam_256_copy\"  ## UPDATE\n","\n","try:\n","  shutil.copytree(source_fldr, copy_folder)\n","except:\n","  # delete previous folder if present\n","  shutil.rmtree(copy_folder)\n","  shutil.copytree(source_fldr, copy_folder)\n","\n","\n","# Open all files in the image folder\n","folder1=copy_folder\n","file_list_yolo=os.listdir(folder1)\n","\n","print(len(file_list_yolo))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ICz09XeAq10_","executionInfo":{"status":"ok","timestamp":1650416164050,"user_tz":-330,"elapsed":9782,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"e2345914-2221-49d3-acdd-e9cbec64f5d4"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["513\n"]}]},{"cell_type":"code","source":["# Print available images and text files\n","image_file_list=[]\n","bbox_file_list=[]\n","\n","for fil in file_list_yolo[:]:\n","  if str(fil).split('.')[1]==\"jpg\":\n","    image_file_list.append(fil)\n","\n","  elif str(fil).split('.')[1]==\"txt\":\n","    bbox_file_list.append(fil)\n","\n","print('image_file_list')\n","print(len(image_file_list))\n","print(image_file_list[:5])\n","\n","print('-'*150)\n","print('-'*150)\n","print('bbox_file_list')\n","print(len(bbox_file_list))\n","print(bbox_file_list[:5])\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"VB7d8Gs91cvR","executionInfo":{"status":"ok","timestamp":1650416164051,"user_tz":-330,"elapsed":34,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"outputId":"fcbfd4ae-52d9-40db-ab11-c652653efa07"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["image_file_list\n","256\n","['vid_am_2_frame_1484.jpg', 'vid_am_2_frame_1442.jpg', 'vid_am_2_frame_1428.jpg', 'vid_am_2_frame_1988.jpg', 'vid_am_2_frame_1974.jpg']\n","------------------------------------------------------------------------------------------------------------------------------------------------------\n","------------------------------------------------------------------------------------------------------------------------------------------------------\n","bbox_file_list\n","257\n","['vid_am_2_frame_1512.txt', 'vid_am_2_frame_1498.txt', 'vid_am_2_frame_1442.txt', 'vid_am_2_frame_1428.txt', 'vid_am_2_frame_1400.txt']\n"]}]},{"cell_type":"code","source":["## Modify classes file and bbox file contents\n","\n","# Modifying classes text file\n","for fil in bbox_file_list:\n","  fil_path=folder1+\"/\"+str(fil)\n","  if str(fil).split('.')[0]==\"classes\":\n","    with open(fil_path,'r') as f:\n","      contents = f.readlines()\n","    contents='ball'\n","       \n","    with open(fil_path,'w') as f:\n","      f.writelines(contents)\n","\n","    \n","\n","\n","# Modifying bbox file\n","for fil in bbox_file_list[:]:\n","  fil_path=\"images_labels/obj_proam_256_copy\"+\"/\"+str(fil) ## UPDATE\n","  if str(fil).split('.')[0]!=\"classes\":\n","\n","    with open(fil_path,'r') as f:\n","      contents = f.readlines()\n","\n","    # # Update bbox file\n","    contents_2=[]\n","    for c1 in contents:\n","      if c1.split(' ')[0]=='15': # If the class is not updated while using labelimg, the additional class of ball becomes class 15\n","        c2=c1.split(' ')[1:]\n","        c2='0'+' '+ ' '.join(c2)\n","        contents_2.append(c2)\n","\n","      elif c1.split(' ')[0]=='0': # If the class is updated while using labelimg, the additional class of ball becomes class 15\n","        c2=c1.split(' ')[1:]\n","        c2='0'+' '+ ' '.join(c2)\n","        contents_2.append(c2)\n","\n","    #   # Update bbox file\n","    with open(fil_path,'w') as f:\n","      f.writelines(contents_2)\n","\n"],"metadata":{"id":"AsHO7jfx3aKh"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["## STEP4. Zipping the original image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process"],"metadata":{"id":"LRVLZ6MR7vGu"}},{"cell_type":"code","source":["# Zipping the folder of original images\n","\n","# Zip the folder\n","full_folder=\"images_labels/obj_proam_256\" ## UPDATE\n","zip_folder=\"ZIPPED_images_labels/obj_proam_256_original_zip\" ## UPDATE\n","shutil.make_archive(zip_folder, 'zip',full_folder)\n","\n","# Original folder count\n","original_len=len(os.listdir(full_folder))\n","\n","\n","# Count contents in the zipped folder\n","with ZipFile(\"ZIPPED_images_labels/obj_proam_256_original_zip.zip\", 'r') as zipObj: ## UPDATE\n","   # Get list of files names in zip\n","  listOfiles = zipObj.namelist()\n","  zip_len=len(listOfiles)\n","\n","# Are the number of files pre and post zipping same\n","print ('Difference in file count pre and post zipping',original_len-zip_len)"],"metadata":{"id":"Z6t4217M5f1c","executionInfo":{"status":"ok","timestamp":1650416172582,"user_tz":-330,"elapsed":6381,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"ebf53d51-4e48-4350-dfb1-a16c97706fda"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Difference in file count pre and post zipping 0\n"]}]},{"cell_type":"markdown","source":["## STEP5. Augmenting the original images using 3 albumenations based augmentations. Only labeled augmented images selected.\n"],"metadata":{"id":"cWsSm3Yr8wpg"}},{"cell_type":"code","source":["## Setting random seed to ensure reproducibility of augmentation\n","random.seed(42)\n","\n","transform1 = A.Compose([\n","    A.RandomCrop(width=500, height=500),\n","   \n","], bbox_params=A.BboxParams(format='yolo', ))\n","\n","transform2 = A.Compose([\n","    \n","    A.HorizontalFlip(p=1),\n","    \n","], bbox_params=A.BboxParams(format='yolo', ))\n","\n","transform3 = A.Compose([\n","    \n","    A.RandomBrightnessContrast(p=1),\n","\n","], bbox_params=A.BboxParams(format='yolo',  ))\n","\n","\n"],"metadata":{"id":"90vttqKZqTHe"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["%%time\n","# Save a set of Augmentations and BBoxes for Input Images\n","\n","# Create Folder and copy contents\n","source_fldr=\"images_labels/obj_proam_256\"  ## UPDATE\n","destination_fldr = r\"images_labels/obj_proam_256_aug\" ## UPDATE\n","try:\n","  shutil.copytree(source_fldr, destination_fldr)\n","except:\n","  # delete previous folder if present\n","  shutil.rmtree(destination_fldr)\n","  shutil.copytree(source_fldr, destination_fldr)\n","\n","\n","folder1=destination_fldr\n","\n","aug_lst=[transform1,transform2,transform3,\n","                 ]\n","\n","aug_lst_name=['transform1','transform2','transform3',\n","                      ]\n","\n","for img in image_file_list[:]: \n","  bbox_file=img.split('.')[0]+'.txt'\n","  \n","  image = cv2.imread(folder1+\"/\"+img) \n","  image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n","\n"," \n","\n","  # Read Bbox\n","  bbox_file_path=folder1+\"/\"+bbox_file \n","  \n","  lst_of_lst=[[]]\n","  fil = open(bbox_file_path, 'r')\n","\n","  for line in fil:\n","    stripped_line = line.strip()\n","    line_list = stripped_line.split()\n","    lst_of_lst.append(line_list)\n","  fil.close()\n","\n","  # Pop 1st element in list of list\n","  lst_of_lst.pop(0)\n","\n","  # For each list in list of list pop 1st item\n","  lst_of_lst=[a[1:] for a in lst_of_lst]\n","\n","  # converting values to float\n","  lst_of_lst=[[float(a) for a in b] for b in lst_of_lst]\n"," \n","  # For each list in list of list append 'balls'\n","  abc=[]\n","\n","  for a in lst_of_lst:\n","    a.append('ball')\n","    abc.append(a) \n","\n","  lst_of_lst=abc\n","  bboxes=lst_of_lst\n","\n","\n","  for idx,val in enumerate(aug_lst[:]): # Update to blank\n","    transformed=val(image=image,bboxes=bboxes)\n","    transformed_image = transformed['image']\n","    transformed_bboxes = transformed['bboxes']\n","\n","\n","    aug_nam=aug_lst_name[idx]\n","\n","    # Save Transformed txt file\n","    save_txt_path=folder1+\"/\"+img.split('.')[0]+\"_\"+aug_nam+\".txt\"\n","  \n","    txt=''\n","    for s in transformed_bboxes:\n","      # Pop last item from set\n","      s=s[:-1]\n","\n","      s=' '.join(map(str,s))\n","      s='0 '+s\n","      txt+=s+'\\n'\n","\n","    # Only capturing the augmented images which have annotation, as augmentation could end up removing the part of the image with detected ball\n","    if len(transformed_bboxes)>0: \n","      # print('len(transformed_bboxes)>0')\n","       # Save Transformed image and text\n","      save_image_path=folder1+\"/\"+img.split('.')[0]+\"_\"+aug_nam+\".jpg\"\n","      cv2.imwrite(save_image_path, transformed_image)\n","      with open(save_txt_path, \"w\") as f:\n","        f.write(txt)\n","\n","\n","    \n","\n","\n","    \n","\n","  \n","\n","\n","\n","\n","  \n","\n","\n"],"metadata":{"id":"yoQyFHw1qUrr","executionInfo":{"status":"ok","timestamp":1650416225820,"user_tz":-330,"elapsed":53274,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"065a80d5-f8c8-41fb-ed28-7685e3454e12"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["CPU times: user 24.5 s, sys: 2.16 s, total: 26.7 s\n","Wall time: 53.3 s\n"]}]},{"cell_type":"code","source":["import os\n","\n","folder_len=len(os.listdir(folder1))\n","\n","print('folder_len',folder_len)\n","\n"],"metadata":{"id":"ACi2Sa6eLr0V","executionInfo":{"status":"ok","timestamp":1650416225821,"user_tz":-330,"elapsed":28,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"dcb0382f-f284-430a-9183-760ee63bbb35"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["folder_len 1775\n"]}]},{"cell_type":"markdown","source":["## STEP6. Zipping the Augmented image folder, as this zipped file will be used in the YOLOv4-tiny Object Detection process"],"metadata":{"id":"z_j87e_o-Wo0"}},{"cell_type":"code","source":["# Zip the folder\n","full_folder=\"images_labels/obj_proam_256_aug\"  ## UPDATE\n","zip_folder=\"ZIPPED_images_labels/obj_proam_256_aug_zip\" ## UPDATE\n","shutil.make_archive(zip_folder, 'zip',full_folder)"],"metadata":{"id":"MGdKv0Q64DwZ","executionInfo":{"status":"ok","timestamp":1650416247241,"user_tz":-330,"elapsed":21444,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/","height":35},"outputId":"c6cb01e1-81b8-46a7-9e38-747d626a1a2a"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["'/content/drive/MyDrive/CAPSTONE/CAPSTONE_FINAL/Module2_Object_Detection_Yolov4_tiny/ZIPPED_images_labels/obj_proam_256_aug_zip.zip'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":72}]},{"cell_type":"code","source":["# Count contents in the zipped folder\n","from zipfile import ZipFile\n","\n","with ZipFile(\"ZIPPED_images_labels/obj_proam_256_aug_zip.zip\", 'r') as zipObj: ## UPDATE\n","   # Get list of files names in zip\n","  listOfiles = zipObj.namelist()\n","  zip_len=len(listOfiles)\n","print(zip_len)"],"metadata":{"id":"TS8N7OgbZk-Q","executionInfo":{"status":"ok","timestamp":1650416247241,"user_tz":-330,"elapsed":31,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"54d2312b-d824-46f6-eddb-2c9eb510c301"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["1775\n"]}]},{"cell_type":"code","source":["# Are the number of files pre and post zipping same\n","print ('Difference in file count pre and post zipping',folder_len-zip_len)"],"metadata":{"id":"LXtVN4MJaqcw","executionInfo":{"status":"ok","timestamp":1650416247242,"user_tz":-330,"elapsed":27,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"060c87a4-f74a-490d-8369-dfb96da428ad"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Difference in file count pre and post zipping 0\n"]}]},{"cell_type":"code","source":["print('End of Augmentation for 256 proam Images')"],"metadata":{"id":"jVIjZYhnuiJ9","executionInfo":{"status":"ok","timestamp":1650416247243,"user_tz":-330,"elapsed":21,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"47ae4a76-1426-4e10-e5d5-601577fc27c2"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["End of Augmentation for 256 proam Images\n"]}]},{"cell_type":"markdown","source":["## STEP7. Dependencies"],"metadata":{"id":"uImzHwh62-kv"}},{"cell_type":"code","source":["# Dependencies\n","%reload_ext watermark\n","%watermark\n","%watermark --iversions"],"metadata":{"id":"sLkChGuk29hK","executionInfo":{"status":"ok","timestamp":1650416247243,"user_tz":-330,"elapsed":16,"user":{"displayName":"Sharadwata Ganguli","userId":"11154011829512635121"}},"colab":{"base_uri":"https://localhost:8080/"},"outputId":"4732d11e-8edb-4f69-fd5c-b28dfef2fdc1"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Last updated: 2022-04-20T00:57:25.957546+00:00\n","\n","Python implementation: CPython\n","Python version       : 3.7.13\n","IPython version      : 5.5.0\n","\n","Compiler    : GCC 7.5.0\n","OS          : Linux\n","Release     : 5.4.144+\n","Machine     : x86_64\n","Processor   : x86_64\n","CPU cores   : 2\n","Architecture: 64bit\n","\n","albumentations: 1.1.0\n","cv2           : 4.1.2\n","IPython       : 5.5.0\n","\n"]}]}]}