Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support automatic tag annotation #3896

Merged

Conversation

subdavis
Copy link
Contributor

@subdavis subdavis commented Nov 10, 2021

Motivation and context

resolves #3358

How has this been tested?

I have a full-frame classifier nuclio lambda and needed a way to save these tags.

  • I ran the code in development using docker-compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
  • Invoke my nuclio classifier from the task summary page (full video) and from the annotator (single frame)
  • See new tags appear in the UI.

I did see that the author of #3358 has included confidence values, which I do not believe are supported by tags. If I should support this somewhere, please LMK.

Checklist

  • I submit my changes into the develop branch
  • I have added description of my changes into CHANGELOG file
  • I have updated the documentation accordingly I do not see any documentation about what response types are supported for nuclio functions, so I don't think there's any documentation to update. New documentation could be added about the general capabilities of this feature, but it seems out of scope for this PR.
  • I have added tests to cover my changes I do not see tests related to this section of the code. If they exist, please LMK and I'll add coverage.
  • I have linked related issues (read github docs)
  • I have increased versions of npm packages if it is necessary (cvat-canvas,
    cvat-core, cvat-data and cvat-ui)
    Not applicable

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below) No new files added
# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

CHANGELOG.md Outdated
@@ -40,6 +40,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add a tutorial on attaching cloud storage AWS-S3 (<https://github.com/openvinotoolkit/cvat/pull/3745>)
and Azure Blob Container (<https://github.com/openvinotoolkit/cvat/pull/3778>)
- The feature to remove annotations in a specified range of frames (<https://github.com/openvinotoolkit/cvat/pull/3617>)
- Add "tag" return type for automatic annotation in Nuclio (<https://github.com/openvinotoolkit/cvat/pull/3896>)
Copy link
Contributor

@nmanovic nmanovic Nov 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@subdavis , Could you please update the changelog? After we released 1.7.0 it isn't valid anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I can do that.

if label_id is not None:
if label_id is None:
continue # Invalid label provided
if anno["type"].lower() == "tag":
Copy link
Contributor

@nmanovic nmanovic Nov 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@subdavis , Could you please contribute any serverless function for classification? Thus we can test the PR.

Copy link
Contributor Author

@subdavis subdavis Dec 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the introduction of a new serverless function is appropriately within scope for #3358

I actually don't even have one I can share: the code I have that generates full-frame detections is closed source. I can provide some instructions about how to temporarily modify an existing function, but I don't really have time to write a new one. I'm not an ML developer.

The problem here seems to be one of documentation: the only way to understand the API between CVAT and Nuclio functions is to read this views.py file, so I think documentation would be ideal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@subdavis , it will be great to hear how in your company (kitware?) you are using CVAT to annotate data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use CVAT occasionally at Kitware, but we're a consultancy and this PR was made for one of our customers who uses CVAT heavily in their own annotation processes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@subdavis , thanks for sharing the information.

@nmanovic nmanovic removed the python label Nov 20, 2021
@nmanovic
Copy link
Contributor

nmanovic commented Dec 1, 2021

@subdavis , please let me know if you have time to answer on my comments?

@nmanovic nmanovic changed the title Support automatic tag annotation [WIP] Support automatic tag annotation Dec 1, 2021
@subdavis subdavis changed the title [WIP] Support automatic tag annotation Support automatic tag annotation Dec 6, 2021
@subdavis subdavis force-pushed the 3358-add-tag-automatic-annotation branch from f5ffe8b to 1224416 Compare December 6, 2021 20:54
@subdavis subdavis force-pushed the 3358-add-tag-automatic-annotation branch from 1224416 to 491e3c9 Compare December 6, 2021 20:55
@subdavis
Copy link
Contributor Author

subdavis commented Dec 6, 2021

Using any of the existing functions, you should be able to test by swapping main with this. You'll need a dataset with an existing label called "example"

import json

def init_context(context):
    pass

def handler(context, event):
    results = [{
        'type': 'tag',
        'label': 'example',
    }]
    return context.Response(body=json.dumps(results), headers={},
        content_type='application/json', status_code=200)

Copy link
Contributor

@nmanovic nmanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nmanovic nmanovic merged commit b1c2689 into cvat-ai:develop Dec 7, 2021
@subdavis subdavis deleted the 3358-add-tag-automatic-annotation branch December 7, 2021 13:25
@nmanovic nmanovic mentioned this pull request Mar 4, 2022
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Tag from Automatic Annotation
3 participants