Skip to content

Commit

Permalink
Quickstart tutorial (#161)
Browse files Browse the repository at this point in the history 
Basic tutorial for using EVA
- [x] Need to support Unnest to enable useful queries. #152 
- [x] Disable concurrent queries for cursor correctness. #160 @xzdandy 
- [x] Upload fails because of missing permissions. #162 @pchunduri6 
- [x] Better response messages from the server. #158
  • Loading branch information
@gaurav274
gaurav274 committed Aug 4, 2021
1 parent bdb78d9 commit cd19b87
Showing 1 changed file with 341 additions and 0 deletions.
341 changes: 341 additions & 0 deletions tutorials/object_detection.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,341 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "b18c8d8c",
"metadata": {},
"source": [
"### Launch EVA DB\n",
"Run the command `python eva.py` in the server where you want to deploy EVA"
]
},
{
"cell_type": "markdown",
"id": "3c5b2a6f",
"metadata": {},
"source": [
"### Establish Connection With Eva DB"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "62e83a0b",
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"sys.path.insert(0,'..')\n",
"from src.server.db_api import connect"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "b78a43e3",
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"nest_asyncio.apply()\n",
"connection = connect(host = '0.0.0.0', port = 5432) # hostname, port of the server where EVADB is running"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "1bc99e88",
"metadata": {},
"outputs": [],
"source": [
"cursor = connection.cursor()"
]
},
{
"cell_type": "markdown",
"id": "6d64d3b6",
"metadata": {},
"source": [
"### Upload the survellience video to analyse"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "130b8561",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Response Object:\n",
"@status: 0\n",
"@batch: Batch Object:\n",
"@dataframe: Empty DataFrame\n",
"Columns: []\n",
"Index: []\n",
"@batch_size: 0\n",
"@identifier_column: id\n",
"@metrics: None\n"
]
}
],
"source": [
"cursor.execute('UPLOAD INFILE \"../data/ua_detrac/ua_detrac.mp4\" PATH \"video.mp4\";')\n",
"response = cursor.fetch_all()\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"id": "6450f505",
"metadata": {},
"source": [
"### Visualize Video"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "bef7026f",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "464b168f81d44edba13db5eb9143f304",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Video(value=b'\\x00\\x00\\x00 ftypisom\\x00\\x00\\x02\\x00isomiso2avc1mp41\\x00\\x00\\x00\\x08free\\x00\\x19K[mdat\\x00\\x00\\…"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from ipywidgets import Video\n",
"Video.from_file(\"../data/ua_detrac/ua_detrac.mp4\", embed=True)"
]
},
{
"cell_type": "markdown",
"id": "686adeb5",
"metadata": {},
"source": [
"### Load video into EVA"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "b18cfe2a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Response Object:\n",
"@status: 0\n",
"@batch: Batch Object:\n",
"@dataframe: Num Loaded Frames Video\n",
"0 252 video.mp4\n",
"@batch_size: 1\n",
"@identifier_column: id\n",
"@metrics: None\n"
]
}
],
"source": [
"cursor.execute('LOAD DATA INFILE \"video.mp4\" INTO MyVideo;')\n",
"response = cursor.fetch_all()\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"id": "c09a40cb",
"metadata": {},
"source": [
"### Optional - Register FasterRCNN (object detection) model into EVA \n",
"\n",
"#### Syntax\n",
" \n",
" `CREATE UDF [ IF NOT EXISTS ] <name> \n",
" INPUT ( [ <arg_name> <arg_data_type> ] [ , ... ] )\n",
" OUTPUT ( [ <result_name> <result_data_type> ] [ , ... ] )\n",
" TYPE <udf_type_name>\n",
" IMPL '<path_to_implementation>'`\n",
"\n",
"#### Required Parameters\n",
"`<name>` - specifies the unique identifier for the UDF.\n",
"\n",
"`[ <arg_name> <arg_data_type> ] [ , ... ]` - specifies the name and data type of the udf input arguments. Name is kept for consistency (ignored by eva right now), arguments data type is required. `ANYDIM` means the shape is inferred at runtime.\n",
"\n",
"`[ <result_name> <result_data_type> ] [ , ... ]` - specifies the name and data type of the udf output arguments. \n",
"Users can access a specific output of the UDF similar to access a column of a table. Eg. `<name>.<result_name>` \n",
"\n",
"`<udf_type_name>` - specifies the identifier for the type of the UDF. UDFs of the same type are assumed to be interchangeable. They should all have identical input and output arguments. For example, object classification can be one type. \n",
"\n",
"`<path_to_implementation>` - specifies the path to the implementation class for the UDF"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "e83e5a44",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Response Object:\n",
"@status: 0\n",
"@batch: Batch Object:\n",
"@dataframe: Empty DataFrame\n",
"Columns: []\n",
"Index: []\n",
"@batch_size: 0\n",
"@identifier_column: id\n",
"@metrics: None\n"
]
}
],
"source": [
"cursor.execute(\"\"\"CREATE UDF IF NOT EXISTS FastRCNNObjectDetector\n",
" INPUT (frame NDARRAY UINT8(3, ANYDIM, ANYDIM))\n",
" OUTPUT (labels NDARRAY STR(ANYDIM), bboxes NDARRAY FLOAT32(ANYDIM, 4),\n",
" scores NDARRAY FLOAT32(ANYDIM))\n",
" TYPE Classification\n",
" IMPL 'src/udfs/fastrcnn_object_detector.py';\n",
" \"\"\")\n",
"response = cursor.fetch_all()\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"id": "7bbd789e",
"metadata": {},
"source": [
"### Run Object detector on the video"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "91bdcaca",
"metadata": {},
"outputs": [],
"source": [
"cursor.execute(\"\"\"SELECT id, Unnest(FastRCNNObjectDetector(data)) FROM MyVideo\"\"\")\n",
"response = cursor.fetch_all()"
]
},
{
"cell_type": "markdown",
"id": "d81ed233",
"metadata": {},
"source": [
"### Visualize output of Object detector on the video"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "ecc977d8",
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"def annotate_video(detections, input_video_path, output_video_path):\n",
" color=(0,255,0)\n",
" thickness=3\n",
"\n",
" vcap = cv2.VideoCapture(input_path)\n",
" width = int(vcap.get(3))\n",
" height = int(vcap.get(4))\n",
" fps = vcap.get(5)\n",
" fourcc = cv2.VideoWriter_fourcc(*'H264') #codec\n",
" video=cv2.VideoWriter(output_path, fourcc, fps, (width,height))\n",
"\n",
" frame_id = 0\n",
" # Capture frame-by-frame\n",
" ret, frame = vcap.read() # ret = 1 if the video is captured; frame is the image\n",
"\n",
" while ret:\n",
" df = detections\n",
" df = df[['boxes', 'labels']][df.id == frame_id]\n",
" if df.size:\n",
" for bbox, label in df.values:\n",
" x1, y1 = bbox[0]\n",
" x2, y2 = bbox[1]\n",
" x1, y1, x2, y2 = int(x1), int(y1), int(x2), int(y2)\n",
" img=cv2.rectangle(frame, (x1, y1), (x2, y2), color, thickness) # object bbox\n",
" cv2.putText(img, label, (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, color, thickness-1) # object label\n",
" video.write(img)\n",
"\n",
" frame_id+=1\n",
" ret, frame = vcap.read()\n",
"\n",
" video.release()\n",
" vcap.release()"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "7a2dee29",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "0c02b702167e4100a705b927a480cd3a",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Video(value=b'\\x00\\x00\\x00\\x1cftypisom\\x00\\x00\\x02\\x00isomiso2mp41\\x00\\x00\\x00\\x08free\\x00\\xdai\\\\mdat\\x82I\\x83…"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from ipywidgets import Video\n",
"input_path = '../data/ua_detrac/ua_detrac.mp4'\n",
"output_path = 'video.mp4'\n",
"annotate_video(response.batch.frames, input_path, output_path)\n",
"Video.from_file(output_path)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

0 comments on commit cd19b87

Please sign in to comment.