Algorithm-action-recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [Sensifai](https://sensifai.com) Action Recognition\n",
    "Sensifai offers one of the most accurate Deep Learning training platform to train your sport recognition system and incorporate it into your application. This product lets you access Sensifai's advanced sport recognition algorithm and train it with our own data and validate it with any set you would like. Our sport action recognition system let you recognize actions and activities in the short and long videos.\n",
    "\n",
    "## Step 1: preparing your data\n",
    "Before starting the training job you have to prepare your data and copy the files to S3. generally, it's better to have 2 separated folders for training and validation files. but if you do not provide a separate folder for validation the algorithm split the data based on the *val_percent* and *train_percent* parameters.\n",
    "\n",
    "in train(and also validation) folder, you have to create a subfolder for each class and copy the video files there. it's recommended to use videos that satisfy the following  conditions:\n",
    "- the algorithm supports most of the common video formats (mp4,avi,...) and we do not set any limitation on the video type, however, if the algorithm cannot detect the video format, skips the file.  also if the file is corrupted, the algorithm skips it. if the number of files that algorithm skip exceeds 50% of all files in the folder the training procedure exited with the failure code.\n",
    "- we do not have strict conditions on the resolution. However, very low-resolution videos (lower than 280p)may have a bad effect on training accuracy. also, very high-resolution videos take longer time for transferring and preprocessing. 360p and 480p videos are the ideal \n",
    "- depends on the type of the action you want to train, the duration of the video can vary.  the algorithm chooses *num_samples_per_video* (can set in training parameters) random samples from each video file. please keep in mind that if the duration of a video is too long, it may that some irrelevant data is feed to the algorithm and it can mislead the algorithm. videos between 10 seconds to 1 minute can be ideal.\n",
    "- It's better to have a folder named **other** that contains all videos that don't belong to any class.\n",
    "- It's important to have a balanced dataset in order to reach outstanding results.\n",
    "\n",
    "## uploading files to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker as sage\n",
    "import boto3\n",
    "import time\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "\n",
    "bucket = \"your bucket here\"\n",
    "prefix = \"prefix on s3 that the test files are stored\"\n",
    "\n",
    "sess = sage.Session()\n",
    "s3_train=\"s3://{}/{}/new_data_171113/\".format(bucket,prefix)\n",
    "s3_validation=\"s3://{}/{}/validation/\".format(bucket,prefix)\n",
    "\n",
    "#we already have transfer the data to s3, if you want to copy the files uncomment below code  \n",
    "#s3_train = sess.upload_data(train_data_dir, bucket, \"{}/train\".format(prefix))\n",
    "#s3_validation = sess.upload_data(validation_data_dir, bucket, \"{}/validation\".format(prefix))\n",
    "\n",
    "print(\"uploaded training data file to {}\".format(s3_train))\n",
    "print(\"uploaded validation data file to {}\".format(s3_validation))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### important note :\n",
    "- currently, we support just \"FileMode\" for input mode. In order to insure that there is enough space for transferring and preprocessing files, please set *ValumeSizeinGB* parameter of the *ResourceConfig* section to 2*size_of_dateset-inGB*)\n",
    "- Generally, the algorithm needs to work for at least for 20 epochs to achieve sufficient accuracy.\n",
    "\n",
    "\n",
    "## Step 2: Create a model \n",
    "__Training Parameters__\n",
    "\n",
    "\n",
    "\n",
    "| Name                  | Description                                                                                       | Type       | min Value | Max Value | IsTunable | IsRequired | DefaultValue |\n",
    "|-----------------------|---------------------------------------------------------------------------------------------------|------------|-----------|-----------|-----------|------------|--------------|\n",
    "| train_percent         | data percentage for training                                                                      | Continuous | 0         | 1         | False     | False      | 0.8          |\n",
    "| val_percent           | data percentage for validation                                                                    | Continuous | 0         | 1         | False     | False      | 0.2          |\n",
    "| learning_rate         | Initial learning rate                                                                             | Continuous | 1E-06     | 0.1       | False     | False      | 0.001        |\n",
    "| momentum              | Momentum                                                                                          | Continuous | 0         | 0.9       | False     | False      | 0.9          |\n",
    "| batch_size            | batch size(if set to 0, will automatically set batch size considering GPU memories)               | Integer    | 0         | 500       | False     | True       | 0            |\n",
    "| lr_patience           | Patience of LR scheduler                                                                          | Integer    | 1         | 100       | False     | False      | 5            |\n",
    "| max_patience          | Terminate training after validation loss become greater than train loss for this number of epochs | Integer    | 1         | 500       | False     | False      | 10           |\n",
    "| num_epochs            | Total number of training epochs                                                                   | Integer    | 1         | 1000      | False     | False      | 30           |\n",
    "| num_samples_per_video | Number of samples to get from each video for training                                             | Integer    | 1         | 100       | False     | False      | 3            |\n",
    "| score_result_threshold | Show the results that their score is greater than this threshold for each timestamp in Inference json file| Continuous    | 0        | 1       | False     | False      | 0.5            |\n",
    "| num_result_tags | Number of tags(Top n tags) to show in each timestamp of Inference json file                             | Integer    | 1         | #classes       | False     | False      | 5            |\n",
    "\n",
    "\n",
    "__Run a SageMaker training job__\n",
    "\n",
    "This code will start a training job, wait for it to be done, and report its status."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "alg_arn=\"Paste the algorithm ARN\"\n",
    "job_name_prefix = 'sensifai-video-action-rec-train'\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "job_name = job_name_prefix + timestamp\n",
    "\n",
    "create_training_params = \\\n",
    "{\n",
    "   \"AlgorithmSpecification\": {\n",
    "        \"AlgorithmName\":alg_arn,\n",
    "        \"TrainingInputMode\": \"File\",\n",
    "        \"MetricDefinitions\": []\n",
    "    },\n",
    "    \"RoleArn\": role,\n",
    "    \"EnableNetworkIsolation\":True,\n",
    "    \"OutputDataConfig\": {\n",
    "        \"S3OutputPath\": 's3://{}/{}/{}/output'.format(bucket,prefix, job_name_prefix)\n",
    "    },\n",
    "    \"ResourceConfig\": {\n",
    "        \"InstanceCount\": 1,\n",
    "        \"InstanceType\": \"ml.p2.8xlarge\",\n",
    "        \"VolumeSizeInGB\": 40\n",
    "    },\n",
    "    \"TrainingJobName\": job_name,\n",
    "    \"StoppingCondition\": {\n",
    "        \"MaxRuntimeInSeconds\": 14400\n",
    "    },\n",
    "    \"HyperParameters\": {\n",
    "   \n",
    "    },\n",
    "    \"InputDataConfig\": [\n",
    "        {\n",
    "            \"ChannelName\": \"train\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": s3_train,\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "                }\n",
    "            },\n",
    "            \"ContentType\": \"\",\n",
    "            \"CompressionType\": \"None\"\n",
    "        },\n",
    "         {\n",
    "            \"ChannelName\": \"validation\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": s3_validation,\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "                }\n",
    "            },\n",
    "            \"ContentType\": \"\",\n",
    "            \"CompressionType\": \"None\"\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "sagemaker = boto3.client(service_name='sagemaker')\n",
    "sagemaker.create_training_job(**create_training_params)\n",
    "status = sagemaker.describe_training_job(TrainingJobName=job_name)['TrainingJobStatus']\n",
    "print('Training job current status: {}'.format(status))\n",
    "\n",
    "try:\n",
    "    sagemaker.get_waiter('training_job_completed_or_stopped').wait(TrainingJobName=job_name)\n",
    "    job_info = sagemaker.describe_training_job(TrainingJobName=job_name)\n",
    "    status = job_info['TrainingJobStatus']\n",
    "    print(\"Training job ended with status: \" + status)\n",
    "except:\n",
    "    print('Training failed to start')\n",
    "    message = sagemaker.describe_training_job(TrainingJobName=job_name)['FailureReason']\n",
    "    print('Training failed with the following error: {}'.format(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step3 :Create a SageMaker model \n",
    "This will set up the model created during training within SageMaker to be used later for recognition.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "model_name=\"sensifai-sport-recognition\" + timestamp\n",
    "job_info = sagemaker.describe_training_job(TrainingJobName=job_name)\n",
    "model_data = job_info['ModelArtifacts']['S3ModelArtifacts']\n",
    "\n",
    "model_package_arn =  \"Paste the model ARN\"\n",
    "model_creation = {\n",
    "    \"ModelName\": model_name,\n",
    "    \"PrimaryContainer\": {\n",
    "        \"ModelPackageName\": model_package_arn\n",
    "    },\n",
    "    \"ExecutionRoleArn\":role\n",
    "} \n",
    "\n",
    "## For Marketplace products, Network isolation flag must be set to true\n",
    "model_creation['EnableNetworkIsolation'] = True\n",
    "\n",
    "model = sagemaker.create_model(**model_creation)\n",
    "sagemaker.describe_model(ModelName = model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## step 4: inference with the trained model  (Batch transform)\n",
    "finally the model is ready to serve and you can feed the videos to the model and save the results in output folder\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "s3_batch_input=\"s3://{}/{}/test/\".format(bucket,prefix)\n",
    "#we already have transfer the data to s3, if you want to copy the files uncomment below code  \n",
    "# s3_bath_input = sess.upload_data(batch_input_dir, bucket, \"{}/test\".format(prefix))\n",
    "print(\"uploaded batch data files to {}\".format(s3_batch_input))\n",
    "\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "batch_job_name = \"sensifai-sport-rec-bt\" + timestamp\n",
    "batch_output = 's3://{}/{}/{}/output'.format(bucket,prefix, batch_job_name)\n",
    "\n",
    "request = \\\n",
    "{\n",
    "  \"TransformJobName\": batch_job_name,\n",
    "  \"ModelName\": model_name,\n",
    "  \"TransformInput\": {\n",
    "    \"DataSource\": {\n",
    "      \"S3DataSource\": {\n",
    "        \"S3DataType\": \"S3Prefix\",\n",
    "        \"S3Uri\": s3_batch_input\n",
    "      }\n",
    "    },\n",
    "    \"ContentType\": \"video/mp4\",\n",
    "    \"CompressionType\": \"None\",\n",
    "    \"SplitType\": \"None\"\n",
    "  },\n",
    "  \"TransformOutput\": {\n",
    "    \"S3OutputPath\": batch_output,\n",
    "    \"Accept\": \"application/json\",\n",
    "    \"AssembleWith\": \"Line\"\n",
    "  },\n",
    "  \"TransformResources\": {\n",
    "    \"InstanceType\": \"ml.p2.xlarge\",\n",
    "    \"InstanceCount\": 1\n",
    "  }\n",
    "}\n",
    "\n",
    "sagemaker.create_transform_job(**request)\n",
    "\n",
    "print(\"Created Transform job with name: \", batch_job_name)\n",
    "\n",
    "while(True):\n",
    "    job_info = sagemaker.describe_transform_job(TransformJobName=batch_job_name)\n",
    "    status = job_info['TransformJobStatus']\n",
    "    if status == 'Completed':\n",
    "        print(\"Transform job ended with status: \" + status)\n",
    "        break\n",
    "    if status == 'Failed':\n",
    "        message = job_info['FailureReason']\n",
    "        print('Transform failed with the following error: {}'.format(message))\n",
    "        raise Exception('Transform job failed') \n",
    "    time.sleep(30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### download the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "from pprint import pprint\n",
    "\n",
    "output_path=\"./output\"\n",
    "\n",
    "if not os.path.exists(output_path):\n",
    "    os.makedirs(output_path)\n",
    "    \n",
    "!aws s3 cp $batch_output $output_path --recursive\n",
    "\n",
    "#do anything with json files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## step 5 : cleaning up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# optionally uncomment and run the code to clean everything up\n",
    "#sagemaker.delete_model(ModelName= model_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}