Face Detection & Recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "33fb2a2c",
   "metadata": {},
   "source": [
    "## Face Detection \n",
    "- Verifies that the image consisting a face or not?\n",
    "- Face Detection task can be broken down into two steps\n",
    "    * Classification task - whether there are any face present in the image\n",
    "    * Localization task - finding the location of the face in the image(x,y,width,height)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a0baccd",
   "metadata": {},
   "source": [
    "### Factors that affect the Facedetection accuracy\n",
    "- One of the most important parts of face detection is the algorithm used for detecting the facial features.\n",
    "- The quality of the input image or video. Poor lighting, low resolution, and occlusions can make it difficult for the face detection algorithm to accurately detect facial features.\n",
    "- The size and orientation of the face can also affect the accuracy of face detection. If the face is too small or too large in the image or if the face is tilted or rotated,it can be difficult for the algorithm to detect the facial features. \n",
    "\n",
    "\n",
    "### Improvements\n",
    "- One common technique is to use machine learning algorithms, such as convolutional neural networks (CNNs), to improve the accuracy of facial feature detection. Additionally, techniques such as image pre-processing, data augmentation, and ensemble learning can also be used to improve the accuracy of face detection.\n",
    "- Another important technique for improving face detection is to use multiple algorithms in combination, such as a combination of Haar cascades and deep learning models\n",
    "- Finally, it is important to continuously evaluate and test the performance of the face detection system under different conditions to identify areas for improvement and refine the algorithm and system design."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9f23b9a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "f897cf41",
   "metadata": {},
   "source": [
    "## Using Viola-Jones Classifier\n",
    "- In OpenCV, we have several trained Haar Cascade models which are saved as XML files. Instead of creating and training the model from scratch, we use this file. \n",
    "- We are going to use “haarcascade_frontalface_alt2.xml” file in this project\n",
    "- works on grayscale images"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3b852fb",
   "metadata": {},
   "source": [
    "#### 1. find the path to the “haarcascade_frontalface_alt2.xml” and “haarcascade_eye_tree_eyeglasses.xml” files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "35d71820",
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0d4e8356",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\DELL\\\\anaconda3\\\\envs\\\\CV\\\\lib\\\\site-packages\\\\cv2\\\\__init__.py'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv2.__file__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "93ac42e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "cascPathface = os.path.dirname(\n",
    "    cv2.__file__) + \"/data/haarcascade_frontalface_alt2.xml\"\n",
    "cascPatheyes = os.path.dirname(\n",
    "    cv2.__file__) + \"/data/haarcascade_eye_tree_eyeglasses.xml\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf08369c",
   "metadata": {},
   "source": [
    "#### 2.  load the classifier\n",
    "- We are using two classifiers, one for detecting the face and others for detection eyes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "540dbf5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "faceCascade = cv2.CascadeClassifier(cascPathface)\n",
    "eyeCascade = cv2.CascadeClassifier(cascPatheyes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "352a4ba3",
   "metadata": {},
   "outputs": [],
   "source": [
    "path = r\"C:\\Users\\DELL\\Jupyter Notebook\\Images\\Group photos\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cb97615f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(720, 881, 3)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img = cv2.imread(path+\"\\grey-student.jpg\")\n",
    "img.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "11f114aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(720, 881)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
    "gray.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "dc8ce649",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cv2.namedWindow('Original',cv2.WINDOW_NORMAL)\n",
    "# cv2.resizeWindow('Original', 600,400)\n",
    "# cv2.imshow('Original', img) \n",
    "\n",
    "# cv2.namedWindow('gray',cv2.WINDOW_NORMAL)\n",
    "# cv2.resizeWindow('gray', 600,400)\n",
    "# cv2.imshow('gray', gray) \n",
    "\n",
    "# cv2.waitKey(0) \n",
    "# cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "6d6e3085",
   "metadata": {},
   "outputs": [],
   "source": [
    "faces = faceCascade.detectMultiScale(gray,scaleFactor=1.3,\n",
    "                                        minNeighbors=5)\n",
    "\n",
    "# Draw rectangles around the faces and eyes\n",
    "for (x, y, w, h) in faces:\n",
    "    cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 2)\n",
    "    \n",
    "    \n",
    "cv2.namedWindow('Original',cv2.WINDOW_NORMAL)\n",
    "cv2.resizeWindow('Original', 600,400)\n",
    "cv2.imshow('Original', img) \n",
    "\n",
    "cv2.waitKey(0) \n",
    "cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "19b6d109",
   "metadata": {},
   "outputs": [],
   "source": [
    "def detect_face_img(img_name):\n",
    "    path = r\"C:\\Users\\DELL\\Jupyter Notebook\\Images\\Group photos\\\\\"\n",
    "    \n",
    "    src = path+img_name\n",
    "    \n",
    "    cascPathface = os.path.dirname(\n",
    "        cv2.__file__) + \"/data/haarcascade_frontalface_alt2.xml\"\n",
    "    cascPatheyes = os.path.dirname(\n",
    "        cv2.__file__) + \"/data/haarcascade_eye_tree_eyeglasses.xml\"\n",
    "    \n",
    "    img = cv2.imread(src)\n",
    "    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n",
    "    \n",
    "    faces = faceCascade.detectMultiScale(gray,scaleFactor=1.3,\n",
    "                                        minNeighbors=5)\n",
    "\n",
    "    # Draw rectangles around the faces and eyes\n",
    "    for (x, y, w, h) in faces:\n",
    "        cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 2)\n",
    "\n",
    "\n",
    "    cv2.namedWindow('Original',cv2.WINDOW_NORMAL)\n",
    "    cv2.resizeWindow('Original', 600,400)\n",
    "    cv2.imshow('Original', img) \n",
    "\n",
    "    cv2.waitKey(0) \n",
    "    cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9e6b4a4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "detect_face_img(\"grey-student.jpg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "9d7edcb5",
   "metadata": {},
   "outputs": [],
   "source": [
    "detect_face_img(\"gvt-child.jpg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "00d8b592",
   "metadata": {},
   "outputs": [],
   "source": [
    "detect_face_img(\"leavers.jpg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "0d7b01a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "detect_face_img(\"leavers-group-photo.jpg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "949b30db",
   "metadata": {},
   "outputs": [],
   "source": [
    "detect_face_img(\"school-twenty-seven-april_d.webp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48133753",
   "metadata": {},
   "source": [
    "### Not detecting \n",
    "- africans faces\n",
    "- face with sunglasses\n",
    "- face with hair covers forehead\n",
    "- "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b81d72d",
   "metadata": {},
   "source": [
    "#### 3. open the webcam using this simple OpenCV one-liner code\n",
    "- The read() function returns:\n",
    "- The actual video frame read (one frame on each loop)\n",
    "- A return code\n",
    "- The return code tells us if we have run out of frames, which will happen if we are reading from a file. This doesn’t matter when reading from the webcam since we can record forever, so we will ignore it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "c6ced282",
   "metadata": {},
   "outputs": [],
   "source": [
    "video_capture = cv2.VideoCapture(0)\n",
    "while True:\n",
    "    # Capture frame-by-frame\n",
    "    ret, frame = video_capture.read()\n",
    "    frame = cv2.flip(frame,1)\n",
    "    #For this specific classifier to work, we need to convert the frame into greyscale.\n",
    "    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n",
    "    \n",
    "    #to run the classifier cascade over the image\n",
    "    faces = faceCascade.detectMultiScale(gray,scaleFactor=1.1,\n",
    "                                        minNeighbors=5,\n",
    "                                        minSize=(60,60),\n",
    "                                        flags=cv2.CASCADE_SCALE_IMAGE)\n",
    "    for (x,y,w,h) in faces:\n",
    "        cv2.rectangle(frame, (x, y), (x + w, y + h),(0,255,0), 2)\n",
    "        faceROI = frame[y:y+h,x:x+w]\n",
    "        eyes = eyeCascade.detectMultiScale(faceROI)\n",
    "        for (x2, y2, w2, h2) in eyes:\n",
    "            eye_center = (x + x2 + w2 // 2, y + y2 + h2 // 2)\n",
    "            radius = int(round((w2 + h2) * 0.25))\n",
    "            frame = cv2.circle(frame, eye_center, radius, (255, 0, 0), 4)\n",
    "    \n",
    "    #The variable faces now contain all the detections for the target image.\n",
    "    #To show the detected face, we will draw a rectangle over it using face cordinates\n",
    "    cv2.imshow(\"Face Detection\",frame)\n",
    "    if cv2.waitKey(1) & 0xFF==ord('q'):\n",
    "        break\n",
    "    \n",
    "video_capture.release()\n",
    "cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef1a0eae",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "dc9bc6ef",
   "metadata": {},
   "source": [
    "## Face Recognition\n",
    "- we are going to describe face recognition using deep learning.\n",
    "- We make use of face embedding in which each face is converted into a vector and this technique is called deep metric learning.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3419ad2e",
   "metadata": {},
   "source": [
    "#### 1. Face Detection\n",
    "- The very first task we perform is detecting faces in the image or video stream. Now that we know the exact location/coordinates of face, we extract this face for further processing ahead.\n",
    "\n",
    "#### 2. Feature Extraction\n",
    "-  Now that we have cropped the face out of the image, we extract features from it.\n",
    "- A neural network takes an image of the person’s face as input and outputs a vector which represents the most important features of a face. In machine learning, this vector is called embedding and thus we call this vector as face embedding.\n",
    "- While training the neural network, the network learns to output similar vectors for faces that look similar\n",
    "- Now after training the network, the network learns to output vectors that are closer to each other(similar) for faces of the same pers in a fileo\n",
    "- We recognise the face if the generated embedding is closer or similar to any other embeddingn(looking similar)\n",
    "- We will use a pre-trained network trained by Davis King on a dataset of ~3 million images. The network outputs a vector of 128 numbers which represent the most important features of a face.\n",
    "- We pass all the images in our data to this pre-trained network to get the respective embeddings and save these embeddings in a file for the next step.\n",
    "\n",
    "#### 3. Comparing Faces\n",
    "- here we recognise a new image that is not in our data.\n",
    "-  compute the face embedding for the image using the same network we used above and then compare this embedding with the rest of the embeddings we have in a file.\n",
    "- We recognise the face if the generated embedding is closer or similar to any other embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c6f8140",
   "metadata": {},
   "source": [
    "### Libraries\n",
    "- The dlib library, maintained by Davis King, contains our implementation of “deep metric learning” which is used to construct our face embeddings used for the actual recognition process.\n",
    "- The face_recognition  library, created by Adam Geitgey, wraps around dlib’s facial recognition functionality, and this library is super easy to work with and we will be using this in our code. Remember to install dlib library first before you install face_recognition."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7115c06",
   "metadata": {},
   "source": [
    "### Extracting features from Face"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "23ae966f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting imutils\n",
      "  Downloading imutils-0.5.4.tar.gz (17 kB)\n",
      "  Preparing metadata (setup.py): started\n",
      "  Preparing metadata (setup.py): finished with status 'done'\n",
      "Building wheels for collected packages: imutils\n",
      "  Building wheel for imutils (setup.py): started\n",
      "  Building wheel for imutils (setup.py): finished with status 'done'\n",
      "  Created wheel for imutils: filename=imutils-0.5.4-py3-none-any.whl size=25854 sha256=7b252d0d82c98457d439a73136cdffefa5b2b9ded0d924881d31486a2bec075d\n",
      "  Stored in directory: c:\\users\\dell\\appdata\\local\\pip\\cache\\wheels\\e2\\73\\ca\\f8ea71e39a18de34c287a665e8e821f19816dfe98996118a25\n",
      "Successfully built imutils\n",
      "Installing collected packages: imutils\n",
      "Successfully installed imutils-0.5.4\n"
     ]
    }
   ],
   "source": [
    "!pip install imutils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e5d29a42",
   "metadata": {},
   "outputs": [],
   "source": [
    "from imutils import paths\n",
    "import face_recognition\n",
    "import pickle\n",
    "import cv2\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "83765ee2",
   "metadata": {},
   "outputs": [],
   "source": [
    "#get paths of each file in folder named Images\n",
    "#Images here contains my data(folders of various persons)\n",
    "imagePaths = list(paths.list_images('Images'))\n",
    "knownEncodings = []\n",
    "knownNames = []\n",
    "\n",
    "# loop over the image paths\n",
    "for (i, imagePath) in enumerate(imagePaths):\n",
    "    # extract the person name from the image path\n",
    "    name = imagePath.split(os.path.sep)[-2]\n",
    "    # load the input image and convert it from BGR (OpenCV ordering)\n",
    "    # to dlib ordering (RGB)\n",
    "    image = cv2.imread(imagePath)\n",
    "    rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
    "    #Use Face_recognition to locate faces\n",
    "    boxes = face_recognition.face_locations(rgb,model='hog')\n",
    "    # compute the facial embedding for the face\n",
    "    encodings = face_recognition.face_encodings(rgb, boxes)\n",
    "    # loop over the encodings\n",
    "    for encoding in encodings:\n",
    "        knownEncodings.append(encoding)\n",
    "        knownNames.append(name)\n",
    "\n",
    "#save emcodings along with their names in dictionary data\n",
    "data = {\"encodings\": knownEncodings, \"names\": knownNames}\n",
    "#use pickle to save data into a file for later use\n",
    "f = open(\"face_enc\", \"wb\")\n",
    "f.write(pickle.dumps(data))\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "a4b8d961",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_encoding(img):\n",
    "    img = face_recognition.load_image_file(img)\n",
    "#     rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
    "    boxes = face_recognition.face_locations(img,model='hog')\n",
    "    enc = face_recognition.face_encodings(rgb,boxes)\n",
    "    return enc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "3827c53f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "e  = get_encoding(r\"C:\\Users\\DELL\\Jupyter Notebook\\Images\\Dipendra\\dip5.jpg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "5d6973f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "cfc40ec3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([-0.04244663,  0.088769  ,  0.05407013, -0.05314859, -0.09558126,\n",
       "        -0.03008778, -0.05468063, -0.08068252,  0.14236121, -0.06268652,\n",
       "         0.24435277, -0.01577381, -0.27382144,  0.02249621, -0.02079305,\n",
       "         0.14156501, -0.13798869, -0.05442272, -0.16349526, -0.10314798,\n",
       "        -0.00265061,  0.06218552,  0.02002058, -0.02206636, -0.06264982,\n",
       "        -0.26298031, -0.07937493, -0.09183647,  0.0860993 , -0.12760018,\n",
       "         0.03296038,  0.03192517, -0.1331813 ,  0.0072698 ,  0.03899386,\n",
       "         0.01524134, -0.02672438, -0.10770703,  0.13513732, -0.05833907,\n",
       "        -0.18521146, -0.02475039,  0.02809101,  0.20679039,  0.19525495,\n",
       "         0.01432962,  0.00716137, -0.12094369,  0.06193394, -0.27590922,\n",
       "         0.03955603,  0.12927254, -0.02298553,  0.09455177,  0.08733436,\n",
       "        -0.18320122,  0.02923244,  0.12889469, -0.12786748,  0.02048741,\n",
       "         0.10552447, -0.06819022, -0.059429  , -0.11837339,  0.19719313,\n",
       "         0.09564072, -0.11827008, -0.15199099,  0.1388972 , -0.17373057,\n",
       "        -0.04121886,  0.04946079, -0.12846182, -0.16105478, -0.24665195,\n",
       "         0.06130427,  0.38960904,  0.16357277, -0.12996249,  0.05936012,\n",
       "        -0.05791864,  0.00705532,  0.02411477,  0.03300194, -0.08451941,\n",
       "        -0.03629592, -0.05941492,  0.04385881,  0.20278497, -0.08665462,\n",
       "         0.04776636,  0.22606088, -0.06693959, -0.01761848, -0.00177701,\n",
       "         0.0297639 , -0.08123592, -0.01389912, -0.07636663,  0.00454357,\n",
       "         0.08951706, -0.14989577,  0.03235759,  0.12592649, -0.18058103,\n",
       "         0.14295724, -0.0325354 , -0.00347316,  0.07771447,  0.02340882,\n",
       "        -0.0223624 , -0.04377473,  0.22059308, -0.20952055,  0.26507935,\n",
       "         0.19764769,  0.07839454,  0.1150683 ,  0.03457483,  0.16549857,\n",
       "        -0.03505833,  0.01213329, -0.16257586, -0.05964639, -0.00340519,\n",
       "        -0.02458359, -0.05750001,  0.03121555])]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "19b5381d",
   "metadata": {},
   "outputs": [],
   "source": [
    "j_s = json.dumps(e[0].tolist())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "c8482a9a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-3.88793461e-02,  4.80120778e-02,  3.05329375e-02,  2.76116543e-02,\n",
       "       -1.38187766e-01, -6.79458380e-02,  3.84580344e-05, -8.84625018e-02,\n",
       "        8.13399181e-02, -6.20967522e-02,  1.74074993e-01, -5.71883023e-02,\n",
       "       -2.17736661e-01, -5.09011596e-02,  2.73617711e-02,  1.02648467e-01,\n",
       "       -1.03422657e-01, -1.42723471e-02, -8.51526260e-02, -1.05434969e-01,\n",
       "        2.66750902e-02,  7.28340447e-02,  3.98029275e-02,  8.87039304e-03,\n",
       "       -7.88770691e-02, -2.52398938e-01, -9.60173532e-02, -6.58060387e-02,\n",
       "        8.21518153e-02, -5.44406027e-02,  1.30003262e-02,  5.66741005e-02,\n",
       "       -2.65963942e-01, -4.23976146e-02, -2.23812349e-02,  8.51448923e-02,\n",
       "       -5.03989831e-02, -7.46729895e-02,  1.94131389e-01,  2.60139536e-03,\n",
       "       -1.50824413e-01,  3.74736786e-02,  5.49237542e-02,  1.60275787e-01,\n",
       "        1.76750898e-01, -4.28585187e-02, -1.62193589e-02, -2.45095138e-02,\n",
       "        6.21440075e-02, -1.89621300e-01,  2.35786103e-02,  1.66055515e-01,\n",
       "        1.35678738e-01,  7.20168278e-02,  1.74455009e-02, -1.08025640e-01,\n",
       "        1.64383613e-02,  1.25756860e-01, -1.42715946e-01,  4.85113785e-02,\n",
       "        7.64658600e-02, -7.94223845e-02, -4.00033332e-02, -4.22112197e-02,\n",
       "        1.55134484e-01,  4.77852523e-02, -6.33070916e-02, -1.21561408e-01,\n",
       "        1.47317588e-01, -1.93023175e-01, -5.62020503e-02,  9.85483229e-02,\n",
       "       -1.00630671e-01, -1.40027300e-01, -3.18748146e-01,  9.69543010e-02,\n",
       "        3.62043023e-01,  1.59411058e-01, -1.60615131e-01,  5.20532578e-03,\n",
       "       -1.00994088e-01, -3.32211368e-02,  3.38139455e-03,  5.08244671e-02,\n",
       "       -3.51771675e-02, -4.09128629e-02, -1.15675710e-01,  7.45182484e-02,\n",
       "        1.67010367e-01, -5.63334487e-02,  1.66829862e-02,  2.54203230e-01,\n",
       "        2.79843397e-02,  3.35872471e-02, -3.73520330e-02,  1.22062936e-02,\n",
       "       -7.85856619e-02, -7.38678426e-02, -4.52183597e-02, -3.21298875e-02,\n",
       "        1.01905972e-01, -1.85300261e-01,  1.60654373e-02,  1.78385824e-01,\n",
       "       -2.18470097e-01,  1.48950770e-01, -3.91924493e-02, -4.57032211e-02,\n",
       "        4.96496633e-02,  4.62638140e-02, -4.82482426e-02, -1.62595678e-02,\n",
       "        1.55746505e-01, -1.93770200e-01,  2.14741603e-01,  1.51956633e-01,\n",
       "       -6.92922994e-02,  1.00968622e-01,  3.35257985e-02,  1.78875506e-01,\n",
       "       -7.78699219e-02, -4.18809801e-02, -1.50662512e-01, -1.24168590e-01,\n",
       "       -2.54439935e-02,  9.43982042e-04,  8.46303534e-03,  6.94413409e-02])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array(json.loads(j_s))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "f7dc3c07",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(128,)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"encodings\"][0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0346a7ec",
   "metadata": {},
   "source": [
    "### Face Recognition in Live webcam Feed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "e0785f91",
   "metadata": {},
   "outputs": [],
   "source": [
    "import imutils\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "67fa630d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Streaming started\n"
     ]
    }
   ],
   "source": [
    "#find path of xml file containing haarcascade file \n",
    "cascPathface = os.path.dirname(\n",
    " cv2.__file__) + \"/data/haarcascade_frontalface_alt2.xml\"\n",
    "# load the harcaascade in the cascade classifier\n",
    "faceCascade = cv2.CascadeClassifier(cascPathface)\n",
    "# load the known faces and embeddings saved in last file\n",
    "data = pickle.loads(open('face_enc', \"rb\").read())\n",
    " \n",
    "print(\"Streaming started\")\n",
    "video_capture = cv2.VideoCapture(0)\n",
    "\n",
    "# loop over frames from the video file stream\n",
    "while True:\n",
    "    # grab the frame from the threaded video stream\n",
    "    ret, frame = video_capture.read()\n",
    "    frame = cv2.flip(frame,1)\n",
    "    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n",
    "    faces = faceCascade.detectMultiScale(gray,\n",
    "                                         scaleFactor=1.1,\n",
    "                                         minNeighbors=5,\n",
    "                                         minSize=(60, 60),\n",
    "                                         flags=cv2.CASCADE_SCALE_IMAGE)\n",
    " \n",
    "    # convert the input frame from BGR to RGB \n",
    "    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)\n",
    "    # the facial embeddings for face in input\n",
    "    encodings = face_recognition.face_encodings(rgb)\n",
    "    \n",
    "    names = []\n",
    "    # loop over the facial embeddings incase\n",
    "    # we have multiple embeddings for multiple fcaes\n",
    "    for encoding in encodings:\n",
    "       #Compare encodings with encodings in data[\"encodings\"]\n",
    "       #Matches contain array with boolean values and True for the embeddings it matches closely\n",
    "       #and False for rest\n",
    "        matches = face_recognition.compare_faces(data[\"encodings\"],\n",
    "         encoding)\n",
    "\n",
    "        #set name =inknown if no encoding matches\n",
    "        name = \"Unknown\"\n",
    "        # check to see if we have found a match\n",
    "        if True in matches:\n",
    "            #Find positions at which we get True and store them\n",
    "            matchedIdxs = [i for (i, b) in enumerate(matches) if b]\n",
    "            counts = {}\n",
    "            # loop over the matched indexes and maintain a count for\n",
    "            # each recognized face\n",
    "            for i in matchedIdxs:\n",
    "                #Check the names at respective indexes we stored in matchedIdxs\n",
    "                name = data[\"names\"][i]\n",
    "                #increase count for the name we got\n",
    "                counts[name] = counts.get(name, 0) + 1\n",
    "            #set name which has highest count\n",
    "            name = max(counts, key=counts.get)\n",
    "            \n",
    "         # update the list of names\n",
    "        names.append(name)\n",
    "        # loop over the recognized faces\n",
    "        for ((x, y, w, h), name) in zip(faces, names):\n",
    "            # rescale the face coordinates\n",
    "            # draw the predicted face name on the image\n",
    "            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)\n",
    "            cv2.putText(frame, name, (x, y), cv2.FONT_HERSHEY_SIMPLEX,\n",
    "             0.75, (0, 255, 0), 2)\n",
    "    cv2.imshow(\"Frame\", frame)\n",
    "    if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "        break\n",
    "video_capture.release()\n",
    "cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "2a14c8e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Empty\n"
     ]
    }
   ],
   "source": [
    "if not []:\n",
    "    print(\"Empty\")\n",
    "    \n",
    "else:\n",
    "    print(\"Not Empty!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c859debc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "CV",
   "language": "python",
   "name": "cv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}