FusionModel + SVM as `FewShotSVMPredictor` #2850

yongxinw · 2023-02-06T22:29:53Z

Issue #, if available:

Description of changes:
Integrating FusionModel + SVM as a utility FewShotSVMPredictor class. The way to use this predictor is the same as in MultiModalPredictor. See below:

predictor = FewShotSVMPredictor(
                    label="label",
                    hyperparameters=hyperparameters,
                    problem_type="feature_extraction",
                    eval_metric="acc"
                )
predictor.fit(train_df)
preds = predictor.predict(data=test_df)
result = predictor.evaluate(test_df, metrics=["acc", "macro_f1"])

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…p all annotation xml files

…when labels.txt is not present.

� Conflicts: � multimodal/src/autogluon/multimodal/predictor.py

…s` to using `get_detection_classes`. Refactoring saving results to utils.

� Conflicts: � multimodal/src/autogluon/multimodal/predictor.py

…ing visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

…tion.

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py

.gitignore

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py

github-actions · 2023-02-07T02:18:01Z

Job PR-2850-f53113d is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2850/f53113d/index.html

yongxinw · 2023-02-07T06:22:06Z

Just verified for image classification. This implementation also works.

…fewshot_svm_predictor

github-actions · 2023-02-07T07:54:16Z

Job PR-2850-b3b059a is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2850/b3b059a/index.html

github-actions · 2023-02-07T07:56:26Z

Job PR-2850-10c5885 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2850/10c5885/index.html

.gitignore

zhiqiangdon

Do we want to expose FewShotSVMPredictor to users, i.e., letting from autogluon.multimodal import FewShotSVMPredictor ?

zhiqiangdon

PR #2827 seems to have a similar functionality. What is the consideration of having 2 PRs of doing the same thing?

zhiqiangdon · 2023-02-07T18:49:38Z

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py

+        self._automm_predictor = MultiModalPredictor(
+            label=label,
+            hyperparameters=hyperparameters,
+            problem_type=FEATURE_EXTRACTION,


FEATURE_EXTRACTION is for text only now. Do we also need to handle image data?

github-actions · 2023-02-07T23:14:51Z

Job PR-2850-76b063a is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2850/76b063a/index.html

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py

…ModalPredictor

…fewshot_svm_predictor

yongxinw · 2023-02-08T07:26:29Z

Do we want to expose FewShotSVMPredictor to users, i.e., letting from autogluon.multimodal import FewShotSVMPredictor ?

I think we'll leave it as a util function for now. We can design our learner after 0.7 and expose that to the users.

github-actions · 2023-02-08T08:14:05Z

Job PR-2850-ae55a94 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2850/ae55a94/index.html

sxjscience

LGTM overall. We can refactor the source code later

yongxinw added 30 commits November 4, 2022 14:51

Adding tutorial for converting data to COCO format.

90fcd09

adding python script to get all unique classes in voc format, and dum…

a5ca590

…p all annotation xml files

adding docs

3693cf9

adding docs

e95784e

refactor function name

6244f42

Reformat

dbc2f99

Reformat: single quote to double quote

d0c5f14

Added inference from pretrained. Added Visualization

ad01ece

Adding visualization. Now get_voc_format_classes dumps class_names …

103b67d

…when labels.txt is not present.

Reformat previous commit.

8a425d6

Addressing issues in PR#2298.

ce884db

Addressing usage of voc2coco.py

9322912

Reformat files to pass Lint Check

f539df8

Deleting as_pandas. Adding default saving path.

982d165

Reformatting to pass Lint Check

0c91e85

Reformatting to pass Lint Check

4bab308

Reformat imports

607ff31

Merge branch 'master' of https://github.com/awslabs/autogluon

30ccfdb

� Conflicts: � multimodal/src/autogluon/multimodal/predictor.py

Adding save results to data frame. Deprecating `get_voc_format_classe…

26e2bda

…s` to using `get_detection_classes`. Refactoring saving results to utils.

Reformat for Lint

73eb0e6

Reformat for Lint

1fa1d76

Finalizing saving results; Refactored visualization into predictor.py.

2177224

Merge branch 'master' of https://github.com/awslabs/autogluon

cffacae

� Conflicts: � multimodal/src/autogluon/multimodal/predictor.py

Add running instructions

ba5644f

Lint check

7402af3

Removing unused imports

d9a0a53

Merge branch 'master' of https://github.com/awslabs/autogluon

229e592

Addressing issues in code review: deleting unused functions, refactor…

51252cc

…ing visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

removing mypy

ca910e8

Adding tutorials for running inference, saving results, and visualiza…

052d805

…tion.

yongxinw requested a review from bryanyzhu February 6, 2023 22:29

yongxinw changed the title ~~Fewshot svm predictor~~ FusionModel + SVM as FewShotSVMPredictor Feb 6, 2023

sxjscience reviewed Feb 6, 2023

View reviewed changes

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py Outdated Show resolved Hide resolved

tonyhoo reviewed Feb 6, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py Show resolved Hide resolved

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py Outdated Show resolved Hide resolved

yongxinw added 3 commits February 6, 2023 15:35

Adding svm model save and load

18ee9a7

Fixing the pickle safety issue.

d15d25f

Formatting

f53113d

yongxinw requested review from sxjscience and tonyhoo February 7, 2023 02:55

sxjscience mentioned this pull request Feb 7, 2023

Add an example of few-shot memory bank model with MultiModal Feature Extraction #2822

Merged

yongxinw added 2 commits February 6, 2023 22:24

Removing experiments/ from .gitignore

b3b059a

Merge branch 'master' of https://github.com/autogluon/autogluon into …

10c5885

…fewshot_svm_predictor

zhiqiangdon reviewed Feb 7, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

zhiqiangdon reviewed Feb 7, 2023

View reviewed changes

reverting .gitignore

76b063a

zhiqiangdon reviewed Feb 7, 2023

View reviewed changes

multimodal/src/autogluon/multimodal/utils/few_shot_learning.py Outdated Show resolved Hide resolved

yongxinw added 3 commits February 7, 2023 17:55

Fixing for CI

5945027

Aligning calling convention of FewShotSVMPredictor .load() with Multi…

238698c

…ModalPredictor

Merge branch 'master' of https://github.com/autogluon/autogluon into …

ae55a94

…fewshot_svm_predictor

sxjscience approved these changes Feb 8, 2023

View reviewed changes

yongxinw merged commit c89491d into autogluon:master Feb 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FusionModel + SVM as `FewShotSVMPredictor` #2850

FusionModel + SVM as `FewShotSVMPredictor` #2850

yongxinw commented Feb 6, 2023 •

edited

github-actions bot commented Feb 7, 2023

yongxinw commented Feb 7, 2023

github-actions bot commented Feb 7, 2023

github-actions bot commented Feb 7, 2023

zhiqiangdon left a comment

zhiqiangdon left a comment

zhiqiangdon Feb 7, 2023

github-actions bot commented Feb 7, 2023

yongxinw commented Feb 8, 2023

github-actions bot commented Feb 8, 2023

sxjscience left a comment

FusionModel + SVM as FewShotSVMPredictor #2850

FusionModel + SVM as FewShotSVMPredictor #2850

Conversation

yongxinw commented Feb 6, 2023 • edited

github-actions bot commented Feb 7, 2023

yongxinw commented Feb 7, 2023

github-actions bot commented Feb 7, 2023

github-actions bot commented Feb 7, 2023

zhiqiangdon left a comment

Choose a reason for hiding this comment

zhiqiangdon left a comment

Choose a reason for hiding this comment

zhiqiangdon Feb 7, 2023

Choose a reason for hiding this comment

github-actions bot commented Feb 7, 2023

yongxinw commented Feb 8, 2023

github-actions bot commented Feb 8, 2023

sxjscience left a comment

Choose a reason for hiding this comment

FusionModel + SVM as `FewShotSVMPredictor` #2850

FusionModel + SVM as `FewShotSVMPredictor` #2850

yongxinw commented Feb 6, 2023 •

edited