DL models as serverless functions

[nmanovic]

Fix #796, fix #792, fix #743, fix #297, fix #296, fix #197, fix #196, fix #896, fix #910, fix #1028, fix #1832, fix #1846, fix #1551

Motivation and context

Before the PR CVAT has all "automatic annotation" features inside one cvat container. CUDA, OpenVINO, extra python packages. Also each "DL model" (such as Mask-RCNN, DEXTR, Faster RCNN) was implemented as a Django application with own REST API. It was very difficult to support new models.

The PR solves most of these issues. All "automatic annotation" features are serverless functions. Current implementation uses nuclio framework (https://github.com/nuclio/nuclio) to deploy and invoke them. Each such serverless function is a separate docker container which can be accessed by HTTP. lambda_manager is Django app which provides convenient
REST API to work with serverless functions:

It is possible to call a function directly using POST /api/v1/lambda/functions/<name> or send a request POST /api/v1/lambda/requests.

How has this been tested?

It was tested manually.

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG file
• I have updated the documentation accordingly
• I have added tests to cover my changes
• I have linked related issues (read github docs)
• I have increased versions of npm packages if it is necessary (cvat-canvas,
  cvat-core, cvat-data and cvat-ui)

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

• Permissions (only users with task.change permissions can run DL models)
• Mask RCNN via TensorFlow
• ReID serverless function
• Code duplication (python3, model_loader.py) for OpenVINO functions
• Semantic segmentation for ADAS function
• Text detection function

Next PR:

• Images by URL in serverless functions
• Tracker serverless function
• Optimize serverless function invocation (measure overhead of nuclio serverless platform and submitting images as json strings, indirect call using dashboard)
• Fix swagger documentation for lambda_manager REST API

[bsekachev]

Excluding comments above, the PR looks good to me.

[azhavoro]

LGTM

[nmanovic]

I had two tasks based on the same video:

The first consists of 1 job
The second consists of 3 job (segment size 50) with enabled ZOrder
Stop frame in both cases is the same: 100

When I run openvino.omz.semantic-segmentation-adas-0001 on the first task, it works correctly
When I run the same model on the second task, progress goes to 100% and then process fails with: ZeroDivisionError: float division by zero

Can share the video with you

UPD. Checked Faster RCNN on the multi-job task and it works well

Found a typo in dataset_manager (probably old one). Fixed. Another problem was with a polygon with many points but at the same line. The area was 0.

[bsekachev]

Found a typo in dataset_manager (probably old one). Fixed. Another problem was with a polygon with many points but at the same line. The area was 0.

Confirm. It has been fixed.

[rushtehrani]

I ran into an issue where I had to pass --platform local to nuctl, example:

./nuctl deploy --project-name cvat \
  --path serverless/openvino/dextr/nuclio \
  --volume `pwd`/serverless/openvino/common:/opt/nuclio/common \
  --platform local

Otherwise, I would get this error:

Error - the server could not find the requested resource (post nuclioprojects.nuclio.io)
    /nuclio/pkg/platform/kube/platform.go:393

This may only be an issue with the latest nuctl release, but I can create a PR and update the docs accordingly if it makes sense.

[rushtehrani]

Also, I'm not seeing a deployed model, even though the model is showing in the API response:

It's also showing up in Nuclio's dashboard:

[nmanovic]

@rushtehrani , dextr isn't showed in the CVAT models list. I don't think that it is right and probably we need to fix it in the future. Now it is filtered explicitly.

[nmanovic]

@rushtehrani , I got the same advice about --platform local from nuclio maintainers. Definitely I will fix that ASAP. Thanks!

[rushtehrani]

dextr isn't showed in the CVAT models list. I don't think that it is right and probably we need to fix it in the future. Now it is filtered explicitly.

Got it. If that's the case, would it make sense to use another example command here that will show up in CVAT?