Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding Joint IA3_LoRA as efficient finetuning strategy #2451

Merged
merged 6 commits into from
Nov 22, 2022
Merged

Adding Joint IA3_LoRA as efficient finetuning strategy #2451

merged 6 commits into from
Nov 22, 2022

Conversation

Raldir
Copy link
Collaborator

@Raldir Raldir commented Nov 21, 2022 •
edited

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Raldir Raldir changed the title Adding IA3_LoRA as efficient finetuning strategy Adding Joint IA3_LoRA as efficient finetuning strategy Nov 21, 2022
@github-actions
Copy link

Job PR-2451-5d84ead is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2451/5d84ead/index.html

@github-actions
Copy link

Job PR-2451-e9dc69a is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2451/e9dc69a/index.html

@github-actions
Copy link

Job PR-2451-d3059de is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-2451/d3059de/index.html

sxjscience
sxjscience approved these changes Nov 22, 2022
View reviewed changes
Copy link
Collaborator

@sxjscience sxjscience left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

zhiqiangdon
zhiqiangdon approved these changes Nov 22, 2022
View reviewed changes
Copy link
Contributor

@zhiqiangdon zhiqiangdon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sxjscience sxjscience merged commit cebe3b5 into autogluon:master Nov 22, 2022
yongxinw pushed a commit to yongxinw/autogluon that referenced this pull request Nov 23, 2022
@Raldir @yongxinw
Adding Joint IA3_LoRA as efficient finetuning strategy (autogluon#2451)
901cab0 
* adding ia3_lora peft and raft#1 preset

* fix

* dynamic max length template choice;

* fix trigger of PEFT

* add coment for 11B model in preset

Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>
yongxinw added a commit that referenced this pull request Nov 24, 2022
@yongxinw @zhiqiangdon
@Innixma @BingzhaoZhu @tonyhoo @canerturkmen @FANGAreNotGnu @bryanyzhu @shchur @cheungdaven @sxjscience @gradientsky @lvwerra @yinweisu @Raldir
Refactoring detection inference quickstart and bug fix on fit->predict (
dc3b4e2 
#2464)

* Adding tutorial for converting data to COCO format.

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* adding docs

* adding docs

* refactor function name

* Reformat

* Reformat: single quote to double quote

* Added inference from pretrained. Added Visualization

* Adding visualization. Now `get_voc_format_classes` dumps class_names when labels.txt is not present.

* Reformat previous commit.

* Addressing issues in PR#2298.

* Addressing usage of voc2coco.py

* Reformat files to pass Lint Check

* Deleting as_pandas. Adding default saving path.

* Reformatting to pass Lint Check

* Reformatting to pass Lint Check

* Reformat imports

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Reformat for Lint

* Reformat for Lint

* Finalizing saving results; Refactored visualization into predictor.py.

* Add running instructions

* Lint check

* Removing unused imports

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* removing mypy

* Adding tutorials for running inference, saving results, and visualization.

* fixing save_path issue

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Add mmdet, mmcv-full links

* Addressing issues in reviews

* Removing .input

* changing quick start coco_*.json to *_cocoformat.json. putting `mim install mmcv-full` before `pip install mmdet`

* Editing visualize_detection to take as input pd.DataFrame. Fixed a bug in save results. Update tutorials

* refactor text semantic search tutorial (#2375)

* Minor fix to infer_speed calc (#2374)

* Refactor the clip embedding tutorial (#2378)

* refactor the clip embedding tutorial

* address comments

* Auto-switch to additive attention for FTT (#2379)

* Remove build_all from platform tests (#2382)

* [timeseries] Fix use of dataframe.iteritems in timeseries (#2385)

* [timeseries] fix usage of pandas iteritems in timeseries

* fix black and isort

* [v0.6] Updated dependency versions (#2373)

* Updated dependency versions

* fix XGBoost slowdown by adding small min_delta to early stopping

* Modified default parameters for FTT (#2386)

* Modified default parameters for FTT

* Modified default parameters for FTT

* [Tutorial] Update AutoMM Detection Tutorial (#2380)

* Updated the logic to infer rois.

* remove tutorial and a minor fix

* add raw_feature for hf_text

* add onnx export

* black

* format and hp changes

* black

* fix get_column_features for other modality, need refactor

* add export_onnx, remove index vectorization

* minor fixes

* black

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* Update tutorial links (#2381)

* update to raw content

* trigger ci

* fix inconsistencies in AutoMM tutorials (#2388)

* Fix FTTransformer errors, add max_features (#2389)

* Fix FTTransformer errors, add max_features

* Hide progress bar if verbosity<=2 FTT

* Create save_path in init and remove warning in fit (#2392)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* [timeseries] Update tutorials and FAQ (#2355)

Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>

* [Tutorial] Fix AutoMM detection tutorial layouts and cli tool (#2391)

* minor fixes

* black

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* fix tuto and cli tool

* merge

* fix docs

* change max depth

* add all detection tuto in multimodal index

* restart ci

* fix max depth

* Remove dummy layer (#2394)

* Update constants.py

* Update predictor.py

* Update ner_text.py

* Update ner_text.py

* Update ner_text.py

* add standalone test

* fix

* Update test_ner_standalone.py

* Update test_ner.py

* Update test_ner.py

* fix

* Update process_ner.py

* Update test_ner_standalone.py

* Update process_ner.py

* Update fusion_mlp_image_text_tabular.yaml

* Update model.py

* Update process_ner.py

* Update process_ner.py

* Update ner_text.py

* start ci

Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-45.us-west-2.compute.internal>

* Fix leaderboard with static features (#2398)

* [Tutorial] Knowledge Distillation tutorial (#2397)

* Create model_distillation.md

* Update automm_distillation_glue.py

* Update automm_distillation_glue.py

* update

* Update tutorial

* update

* Update model_distillation.md

* update

* Fix seed

* fix tutorial

* Update model_distillation.md

* Fix installation error for pycocotools package (#2400)

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* fix tuto and cli tool

* merge

* fix docs

* change max depth

* add all detection tuto in multimodal index

* restart ci

* fix max depth

* fix installation error for pycocotools

* remove windows

* Fix hyperparameters in matcher fit (#2404)

* support hyperparameters in matcher fit

* fix

* fix

* [docs] Apple Silicon Instructions (#2403)

* [timeseries] make core and tabular dependencies explicit (#2405)

* Turn off AutoMM prediction progress bar in tabular (#2401)

* save and load enable_progress_bar

* turn off progress bar for automm in tabular

* Remove warnings (#2402)

* replace iteritems with items

* replace iteritems with items

* remove seqeval warning

* Update metric.py

* set num_workers to 0 by default

Some huggingface checkpoints such as deberta-v3, mdeberta-v3, roberta, flan-t5-xl do not work with ddp_spawn when setting num_workers=2

* update ner presets

* add fixme

* Update presets.py

* Improve text_prediction tutorial (#2414)

* fix multimodal tutorial index (#2407)

* Remove warnings and duplicate function (#2409)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* Fix cpu inference bug (#2413)

* fix cpu inference bug

* add test-case of cpu-only inference

* Fix NER best quality preset (#2412)

* Update presets.py

* Update presets.py

* Update presets.py

* Update Quick Start Tutorial and Add Installation Warnings (#2418)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* add warnings on import error, update quick start example

* update quick start doc

* fix wording

* add installation info

* fix

* Remove dependency on vision.imagedataset (#2411)

* reemove imagedataset

* add datasets

* move dataset download to utils

* lint

* lint

* lint

* Fixed model save and load path with uuid for multiple runs (#2415)

* FTT preset (#2410)

* Fix FTTransformer errors, add max_features

* Add FTT presets

* Added parallel bagging on CPU with FTTransformer

* remove ultra preset, add docs

* Set num_workers for image related presets and clean matcher config (#2416)

* clean matcher config

* num_workers

* Fix bad refs in detection tutorial (#2419)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* add warnings on import error, update quick start example

* update quick start doc

* fix wording

* add installation info

* fix

* fix bad ref

* [Tutorial] Shorten multilingual tutorial + Improve tutorial website (#2417)

* improve document

* shorten tutorial

* Update index.rst

* update

* revise tutorial.

* update tutorial

* Fix typo

* Adding inference quick start tutorial

* Fixing YOLOv3 description

* Moving mmdet mmcv to top of page

* Changing {.python .input} to python .input

* [0.6 Release][Object Detection][Tutorial] Remove mmdet output format (#2393)

* Adding tutorial for converting data to COCO format.

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* adding docs

* adding docs

* refactor function name

* Reformat

* Reformat: single quote to double quote

* Added inference from pretrained. Added Visualization

* Adding visualization. Now `get_voc_format_classes` dumps class_names when labels.txt is not present.

* Reformat previous commit.

* Addressing issues in PR#2298.

* Addressing usage of voc2coco.py

* Reformat files to pass Lint Check

* Deleting as_pandas. Adding default saving path.

* Reformatting to pass Lint Check

* Reformatting to pass Lint Check

* Reformat imports

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Reformat for Lint

* Reformat for Lint

* Finalizing saving results; Refactored visualization into predictor.py.

* Add running instructions

* Lint check

* Removing unused imports

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* removing mypy

* Adding tutorials for running inference, saving results, and visualization.

* fixing save_path issue

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Add mmdet, mmcv-full links

* Addressing issues in reviews

* Removing .input

* changing quick start coco_*.json to *_cocoformat.json. putting `mim install mmcv-full` before `pip install mmdet`

* Editing visualize_detection to take as input pd.DataFrame. Fixed a bug in save results. Update tutorials

* editting tutorials to reflect pd.DataFrame detection output.

* Update tutorials to print pred

* Update inference quick start tutorial to single image

* Update inference quick start tutorial to single image; Changing matplotlib to PIL for displaying visualization.

* Update inference quick start tutorial to single image; Changing matplotlib to PIL for displaying visualization.

* Fixing quick start import error with numpy.

* Fixed links in inference tutorials, moved inference quick start to quick_start, fixed a typo in detection_eval_fasterrcnn_coco

* fixing kernel died issue

* Removing quick start to another PR

* Remove detection load predictor

* Changing {.python} to python

* Removing quick start in index.rst

* Update to v0.6 (#2425)

* [Release 0.6] [Multimodal] Check if folder is empty before raise (#2426)

* fix

* fix

* lint

* fix

* fix

* Update to v0.6.1

* bump evaluate to `0.3.0` (#2433)

* [Cloud] Cloud Custom Image and Cleanup (#2408)

* checkpoint

* checkpoint

* api

* cleanup old version code

* fix to local

* fix

* fix

* fix

* additional volume size

* fix

* fix

* addressing comments

* fix

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* fix (#2437)

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* 0.6.0 Release notes (#2383)

* Add finetune/eval tests for AutoMM detection (#2441)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* add fintune/eval test for detection

* lint

* fix layout (#2450)

* [CI][Cloud] Nightly Build of AG Images for Cloud Testing (#2436)

* initial

fix

fix

fix

checkpoint

checkpoint

checkpoint

api

cleanup old version code

fix to local

fix

fix

fix

additional volume size

fix

fix

tests

* fix

* fix

* enable nightly for cloud CI

* minor fix entry point

* comments

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* Adding Joint IA3_LoRA as efficient finetuning strategy (#2451)

* adding ia3_lora peft and raft#1 preset

* fix

* dynamic max length template choice;

* fix trigger of PEFT

* add coment for 11B model in preset

Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>

* Fix AutoMM warnings about object detection (#2458)

* fix warnings

* ocr constant

* [CI][Fair] Enable CI for Fair Module (#2460)

* fair ci

* fix

* [Cloud][CI] Enable Cloud Lint (#2455)

* lint

* fix

* fix

* isort skip

* gox

* fix

* fix

* fix

* fix

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* Reformat previous commit.

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Deleting inference tutorials, refactoring inference quick start to quick_start_coco.md, fixing the bug for fit->predict with dummy df converter from dict, updating detection unit test

* Deleting inference tutorials, refactoring inference quick start to quick_start_coco.md, fixing the bug for fit->predict with dummy df converter from dict, updating detection unit test

* Addressing comments in reviews.

* Re-organizing tutorial, picking a different image to display

* Deleting inference cards in the object detection index page

Co-authored-by: Zhiqiang Tang <zhiqiang.tang@rutgers.edu>
Co-authored-by: Nick Erickson <neerick@amazon.com>
Co-authored-by: BingzhaoZhu <39958219+BingzhaoZhu@users.noreply.github.com>
Co-authored-by: tonyhu <tonyhoo@users.noreply.github.com>
Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>
Co-authored-by: Haoyang Fang <107515844+FANGAreNotGnu@users.noreply.github.com>
Co-authored-by: Yi Zhu <yizhu59@gmail.com>
Co-authored-by: Oleksandr Shchur <shchuro@amazon.com>
Co-authored-by: Shuai Zhang <cheungdaven@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-45.us-west-2.compute.internal>
Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>
Co-authored-by: Alexander Shirkov <10080307+gradientsky@users.noreply.github.com>
Co-authored-by: Weisu Yin <weisy@amazon.com>
Co-authored-by: Alexander Shirkov <ashyrkou@amazon.com>
Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
Co-authored-by: Weisu Yin <weisuyin96@gmail.com>
Co-authored-by: Rami <rami-aly@hotmail.de>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>
yongxinw added a commit that referenced this pull request Dec 6, 2022
@yongxinw @zhiqiangdon
@Innixma @BingzhaoZhu @tonyhoo @canerturkmen @FANGAreNotGnu @bryanyzhu @shchur @cheungdaven @sxjscience @gradientsky @lvwerra @yinweisu @Raldir
Use detectron2 visualizer and update quickstart (#2502)
81b8e9d 
* Adding tutorial for converting data to COCO format.

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* adding docs

* adding docs

* refactor function name

* Reformat

* Reformat: single quote to double quote

* Added inference from pretrained. Added Visualization

* Adding visualization. Now `get_voc_format_classes` dumps class_names when labels.txt is not present.

* Reformat previous commit.

* Addressing issues in PR#2298.

* Addressing usage of voc2coco.py

* Reformat files to pass Lint Check

* Deleting as_pandas. Adding default saving path.

* Reformatting to pass Lint Check

* Reformatting to pass Lint Check

* Reformat imports

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Reformat for Lint

* Reformat for Lint

* Finalizing saving results; Refactored visualization into predictor.py.

* Add running instructions

* Lint check

* Removing unused imports

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* removing mypy

* Adding tutorials for running inference, saving results, and visualization.

* fixing save_path issue

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Add mmdet, mmcv-full links

* Addressing issues in reviews

* Removing .input

* changing quick start coco_*.json to *_cocoformat.json. putting `mim install mmcv-full` before `pip install mmdet`

* Editing visualize_detection to take as input pd.DataFrame. Fixed a bug in save results. Update tutorials

* refactor text semantic search tutorial (#2375)

* Minor fix to infer_speed calc (#2374)

* Refactor the clip embedding tutorial (#2378)

* refactor the clip embedding tutorial

* address comments

* Auto-switch to additive attention for FTT (#2379)

* Remove build_all from platform tests (#2382)

* [timeseries] Fix use of dataframe.iteritems in timeseries (#2385)

* [timeseries] fix usage of pandas iteritems in timeseries

* fix black and isort

* [v0.6] Updated dependency versions (#2373)

* Updated dependency versions

* fix XGBoost slowdown by adding small min_delta to early stopping

* Modified default parameters for FTT (#2386)

* Modified default parameters for FTT

* Modified default parameters for FTT

* [Tutorial] Update AutoMM Detection Tutorial (#2380)

* Updated the logic to infer rois.

* remove tutorial and a minor fix

* add raw_feature for hf_text

* add onnx export

* black

* format and hp changes

* black

* fix get_column_features for other modality, need refactor

* add export_onnx, remove index vectorization

* minor fixes

* black

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* Update tutorial links (#2381)

* update to raw content

* trigger ci

* fix inconsistencies in AutoMM tutorials (#2388)

* Fix FTTransformer errors, add max_features (#2389)

* Fix FTTransformer errors, add max_features

* Hide progress bar if verbosity<=2 FTT

* Create save_path in init and remove warning in fit (#2392)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* [timeseries] Update tutorials and FAQ (#2355)

Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>

* [Tutorial] Fix AutoMM detection tutorial layouts and cli tool (#2391)

* minor fixes

* black

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* fix tuto and cli tool

* merge

* fix docs

* change max depth

* add all detection tuto in multimodal index

* restart ci

* fix max depth

* Remove dummy layer (#2394)

* Update constants.py

* Update predictor.py

* Update ner_text.py

* Update ner_text.py

* Update ner_text.py

* add standalone test

* fix

* Update test_ner_standalone.py

* Update test_ner.py

* Update test_ner.py

* fix

* Update process_ner.py

* Update test_ner_standalone.py

* Update process_ner.py

* Update fusion_mlp_image_text_tabular.yaml

* Update model.py

* Update process_ner.py

* Update process_ner.py

* Update ner_text.py

* start ci

Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-45.us-west-2.compute.internal>

* Fix leaderboard with static features (#2398)

* [Tutorial] Knowledge Distillation tutorial (#2397)

* Create model_distillation.md

* Update automm_distillation_glue.py

* Update automm_distillation_glue.py

* update

* Update tutorial

* update

* Update model_distillation.md

* update

* Fix seed

* fix tutorial

* Update model_distillation.md

* Fix installation error for pycocotools package (#2400)

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* fix tuto and cli tool

* merge

* fix docs

* change max depth

* add all detection tuto in multimodal index

* restart ci

* fix max depth

* fix installation error for pycocotools

* remove windows

* Fix hyperparameters in matcher fit (#2404)

* support hyperparameters in matcher fit

* fix

* fix

* [docs] Apple Silicon Instructions (#2403)

* [timeseries] make core and tabular dependencies explicit (#2405)

* Turn off AutoMM prediction progress bar in tabular (#2401)

* save and load enable_progress_bar

* turn off progress bar for automm in tabular

* Remove warnings (#2402)

* replace iteritems with items

* replace iteritems with items

* remove seqeval warning

* Update metric.py

* set num_workers to 0 by default

Some huggingface checkpoints such as deberta-v3, mdeberta-v3, roberta, flan-t5-xl do not work with ddp_spawn when setting num_workers=2

* update ner presets

* add fixme

* Update presets.py

* Improve text_prediction tutorial (#2414)

* fix multimodal tutorial index (#2407)

* Remove warnings and duplicate function (#2409)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* Fix cpu inference bug (#2413)

* fix cpu inference bug

* add test-case of cpu-only inference

* Fix NER best quality preset (#2412)

* Update presets.py

* Update presets.py

* Update presets.py

* Update Quick Start Tutorial and Add Installation Warnings (#2418)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* add warnings on import error, update quick start example

* update quick start doc

* fix wording

* add installation info

* fix

* Remove dependency on vision.imagedataset (#2411)

* reemove imagedataset

* add datasets

* move dataset download to utils

* lint

* lint

* lint

* Fixed model save and load path with uuid for multiple runs (#2415)

* FTT preset (#2410)

* Fix FTTransformer errors, add max_features

* Add FTT presets

* Added parallel bagging on CPU with FTTransformer

* remove ultra preset, add docs

* Set num_workers for image related presets and clean matcher config (#2416)

* clean matcher config

* num_workers

* Fix bad refs in detection tutorial (#2419)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* add warnings on import error, update quick start example

* update quick start doc

* fix wording

* add installation info

* fix

* fix bad ref

* [Tutorial] Shorten multilingual tutorial + Improve tutorial website (#2417)

* improve document

* shorten tutorial

* Update index.rst

* update

* revise tutorial.

* update tutorial

* Fix typo

* Adding inference quick start tutorial

* Fixing YOLOv3 description

* Moving mmdet mmcv to top of page

* Changing {.python .input} to python .input

* [0.6 Release][Object Detection][Tutorial] Remove mmdet output format (#2393)

* Adding tutorial for converting data to COCO format.

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* adding docs

* adding docs

* refactor function name

* Reformat

* Reformat: single quote to double quote

* Added inference from pretrained. Added Visualization

* Adding visualization. Now `get_voc_format_classes` dumps class_names when labels.txt is not present.

* Reformat previous commit.

* Addressing issues in PR#2298.

* Addressing usage of voc2coco.py

* Reformat files to pass Lint Check

* Deleting as_pandas. Adding default saving path.

* Reformatting to pass Lint Check

* Reformatting to pass Lint Check

* Reformat imports

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Reformat for Lint

* Reformat for Lint

* Finalizing saving results; Refactored visualization into predictor.py.

* Add running instructions

* Lint check

* Removing unused imports

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* removing mypy

* Adding tutorials for running inference, saving results, and visualization.

* fixing save_path issue

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Add mmdet, mmcv-full links

* Addressing issues in reviews

* Removing .input

* changing quick start coco_*.json to *_cocoformat.json. putting `mim install mmcv-full` before `pip install mmdet`

* Editing visualize_detection to take as input pd.DataFrame. Fixed a bug in save results. Update tutorials

* editting tutorials to reflect pd.DataFrame detection output.

* Update tutorials to print pred

* Update inference quick start tutorial to single image

* Update inference quick start tutorial to single image; Changing matplotlib to PIL for displaying visualization.

* Update inference quick start tutorial to single image; Changing matplotlib to PIL for displaying visualization.

* Fixing quick start import error with numpy.

* Fixed links in inference tutorials, moved inference quick start to quick_start, fixed a typo in detection_eval_fasterrcnn_coco

* fixing kernel died issue

* Removing quick start to another PR

* Remove detection load predictor

* Changing {.python} to python

* Removing quick start in index.rst

* Update to v0.6 (#2425)

* [Release 0.6] [Multimodal] Check if folder is empty before raise (#2426)

* fix

* fix

* lint

* fix

* fix

* Update to v0.6.1

* bump evaluate to `0.3.0` (#2433)

* [Cloud] Cloud Custom Image and Cleanup (#2408)

* checkpoint

* checkpoint

* api

* cleanup old version code

* fix to local

* fix

* fix

* fix

* additional volume size

* fix

* fix

* addressing comments

* fix

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* fix (#2437)

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* 0.6.0 Release notes (#2383)

* Add finetune/eval tests for AutoMM detection (#2441)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* add fintune/eval test for detection

* lint

* fix layout (#2450)

* [CI][Cloud] Nightly Build of AG Images for Cloud Testing (#2436)

* initial

fix

fix

fix

checkpoint

checkpoint

checkpoint

api

cleanup old version code

fix to local

fix

fix

fix

additional volume size

fix

fix

tests

* fix

* fix

* enable nightly for cloud CI

* minor fix entry point

* comments

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* Adding Joint IA3_LoRA as efficient finetuning strategy (#2451)

* adding ia3_lora peft and raft#1 preset

* fix

* dynamic max length template choice;

* fix trigger of PEFT

* add coment for 11B model in preset

Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>

* Fix AutoMM warnings about object detection (#2458)

* fix warnings

* ocr constant

* [CI][Fair] Enable CI for Fair Module (#2460)

* fair ci

* fix

* [Cloud][CI] Enable Cloud Lint (#2455)

* lint

* fix

* fix

* isort skip

* gox

* fix

* fix

* fix

* fix

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* Reformat previous commit.

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Deleting inference tutorials, refactoring inference quick start to quick_start_coco.md, fixing the bug for fit->predict with dummy df converter from dict, updating detection unit test

* Deleting inference tutorials, refactoring inference quick start to quick_start_coco.md, fixing the bug for fit->predict with dummy df converter from dict, updating detection unit test

* Addressing comments in reviews.

* Re-organizing tutorial, picking a different image to display

* Deleting inference cards in the object detection index page

* Adding visualizer from detectron2

* Adding visualizer from detectron2 - Complete. Updating quick_start_coco.md

* Editing docs for colormap.py

* Deleting unused classes. Reformatting for lint.

* Reformatting for lint.

* Adding new line

* add back newline under cv2

* removing __main__ in colormap as it is not necessary

Co-authored-by: Zhiqiang Tang <zhiqiang.tang@rutgers.edu>
Co-authored-by: Nick Erickson <neerick@amazon.com>
Co-authored-by: BingzhaoZhu <39958219+BingzhaoZhu@users.noreply.github.com>
Co-authored-by: tonyhu <tonyhoo@users.noreply.github.com>
Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>
Co-authored-by: Haoyang Fang <107515844+FANGAreNotGnu@users.noreply.github.com>
Co-authored-by: Yi Zhu <yizhu59@gmail.com>
Co-authored-by: Oleksandr Shchur <shchuro@amazon.com>
Co-authored-by: Shuai Zhang <cheungdaven@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-45.us-west-2.compute.internal>
Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>
Co-authored-by: Alexander Shirkov <10080307+gradientsky@users.noreply.github.com>
Co-authored-by: Weisu Yin <weisy@amazon.com>
Co-authored-by: Alexander Shirkov <ashyrkou@amazon.com>
Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
Co-authored-by: Weisu Yin <weisuyin96@gmail.com>
Co-authored-by: Rami <rami-aly@hotmail.de>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>
yongxinw added a commit that referenced this pull request Jan 11, 2023
@yongxinw @zhiqiangdon
@Innixma @BingzhaoZhu @tonyhoo @canerturkmen @FANGAreNotGnu @bryanyzhu @shchur @cheungdaven @sxjscience @gradientsky @lvwerra @yinweisu @Raldir
[Merged after 0.6.1] Predictor support filename filelist input (#2520)
5296127 
* Adding tutorial for converting data to COCO format.

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* adding docs

* adding docs

* refactor function name

* Reformat

* Reformat: single quote to double quote

* Added inference from pretrained. Added Visualization

* Adding visualization. Now `get_voc_format_classes` dumps class_names when labels.txt is not present.

* Reformat previous commit.

* Addressing issues in PR#2298.

* Addressing usage of voc2coco.py

* Reformat files to pass Lint Check

* Deleting as_pandas. Adding default saving path.

* Reformatting to pass Lint Check

* Reformatting to pass Lint Check

* Reformat imports

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Reformat for Lint

* Reformat for Lint

* Finalizing saving results; Refactored visualization into predictor.py.

* Add running instructions

* Lint check

* Removing unused imports

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* removing mypy

* Adding tutorials for running inference, saving results, and visualization.

* fixing save_path issue

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Add mmdet, mmcv-full links

* Addressing issues in reviews

* Removing .input

* changing quick start coco_*.json to *_cocoformat.json. putting `mim install mmcv-full` before `pip install mmdet`

* Editing visualize_detection to take as input pd.DataFrame. Fixed a bug in save results. Update tutorials

* refactor text semantic search tutorial (#2375)

* Minor fix to infer_speed calc (#2374)

* Refactor the clip embedding tutorial (#2378)

* refactor the clip embedding tutorial

* address comments

* Auto-switch to additive attention for FTT (#2379)

* Remove build_all from platform tests (#2382)

* [timeseries] Fix use of dataframe.iteritems in timeseries (#2385)

* [timeseries] fix usage of pandas iteritems in timeseries

* fix black and isort

* [v0.6] Updated dependency versions (#2373)

* Updated dependency versions

* fix XGBoost slowdown by adding small min_delta to early stopping

* Modified default parameters for FTT (#2386)

* Modified default parameters for FTT

* Modified default parameters for FTT

* [Tutorial] Update AutoMM Detection Tutorial (#2380)

* Updated the logic to infer rois.

* remove tutorial and a minor fix

* add raw_feature for hf_text

* add onnx export

* black

* format and hp changes

* black

* fix get_column_features for other modality, need refactor

* add export_onnx, remove index vectorization

* minor fixes

* black

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* Update tutorial links (#2381)

* update to raw content

* trigger ci

* fix inconsistencies in AutoMM tutorials (#2388)

* Fix FTTransformer errors, add max_features (#2389)

* Fix FTTransformer errors, add max_features

* Hide progress bar if verbosity<=2 FTT

* Create save_path in init and remove warning in fit (#2392)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* [timeseries] Update tutorials and FAQ (#2355)

Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>

* [Tutorial] Fix AutoMM detection tutorial layouts and cli tool (#2391)

* minor fixes

* black

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* fix tuto and cli tool

* merge

* fix docs

* change max depth

* add all detection tuto in multimodal index

* restart ci

* fix max depth

* Remove dummy layer (#2394)

* Update constants.py

* Update predictor.py

* Update ner_text.py

* Update ner_text.py

* Update ner_text.py

* add standalone test

* fix

* Update test_ner_standalone.py

* Update test_ner.py

* Update test_ner.py

* fix

* Update process_ner.py

* Update test_ner_standalone.py

* Update process_ner.py

* Update fusion_mlp_image_text_tabular.yaml

* Update model.py

* Update process_ner.py

* Update process_ner.py

* Update ner_text.py

* start ci

Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-45.us-west-2.compute.internal>

* Fix leaderboard with static features (#2398)

* [Tutorial] Knowledge Distillation tutorial (#2397)

* Create model_distillation.md

* Update automm_distillation_glue.py

* Update automm_distillation_glue.py

* update

* Update tutorial

* update

* Update model_distillation.md

* update

* Fix seed

* fix tutorial

* Update model_distillation.md

* Fix installation error for pycocotools package (#2400)

* add eval

* remove hard code in get_processed_batch

* remove hardcoded batch size

* fix bug and refactor

* black

* add descriptions for functions

* lint

* update example and fix comments

* remove extra inpit

* change back config

* fix label column infer in _predict

* update feature extraction example

* black

* add test add fix several comments

* black

* black

* lint

* add onnxruntime in setup

* fix

* remove code for test

* black

* remove print in test

* remove default batch test

* skip onnx test for version conflict

* edit setup

* minor fix

* change eval to evaluate

* fix merge

* fix merge

* change cascade_rcnn_x101 to a smaller centernet in detection test

* increase timeout from 15 to 120 for mim install

* update centernet version

* fix

* change to maskrcnn

* finished ft, testing val metric

* complete val metric integration for obj detection

* fix val bug

* fix val_metric error

* yolo voc finetune success

* refactors

* remove COLLECTION

* refactors

* refactor

* refactor

* black

* small fic

* small fix

* black

* lint

* lint

* lint

* fix pred without label

* fix ocr

* add comments

* add comments

* fix merge

* revert detection inference example change

* fix a conflict in matcher and update a dependency for mmocr

* decrease mim timeout to 60 second

* import mmcv only when use

* import mmcv only when use

* black

* black

* bug fix

* bug fix

* update detection eval example

* add todo mark

* cleaner code and black

* resolve several comments

* change infer_label_column_type_by's name

* fix typo

* add todo mark

* change constant map to mAP

* change mmcv_model constant to mmlab_models

* lint

* add todos and add if det to reset_index

* multi gpu training

* black

* add val_metric option and fix some TODOs

* small refactors

* black

* lint

* fix circular import

* black

* small fix

* fix collate and better integrate with lightning for detection

* remove unnecessary save/load

* remove train_to_val and val_to_train in litmmdet

* fix map

* temp fix for MMOCR, need to update

* enable multigpu inference

* merge

* lint

* fix single gpu inference

* add doc

* skip ocr test for now

* black

* black

* rename output_shape to num_classes

* fix

* add a TODO

* add train from scratch for detection model

* fix hardcoding for voc and coco while load coco format data

* black

* small fix and add voc2coco tool

* update voc2coco.py

* fix type

* fix voc2coco

* fix

* fix classes hardcode and add support for two stage lr

* fix single gpu eval and fix torchmetrics eval

* add support for voc format input

* fix a word

* add preset, fix eval bug for voc

* some refactors

* black and small refactor

* lint

* fix assets loading

* fix

* black

* add docs

* open PIL safely to avoid resource warning

* add fasterrcnn 2 stage lr and black

* add set_num_gpus

* add evaluation tutorial

* refine documents

* update ddetr

* fix typo

* change ref names

* merge

* add eval on voc and default two stage setting

* finish tutorial in datasets preparation

* small fix

* update tutorial and codes

* add EOF line and remove finetune script

* black

* fix typo in docs

* fix wordings and add get_detection_classes, fix voc2coco, fix predict

* fix type

* small update

* fix predict df error

* fix voc2coco

* add finetune tutorial and support vfnet, raise error for mask models

* add voc2coco tutorial

* add eof

* update doc

* minor

* support more models and skip image size if not in config

* black

* update docs

* add load predictor and change to problem type

* update fps information

* update load preditor example

* fix type

* update tutorial

* move val_metric

* add presets and fix val_metric in tuto and example

* black

* remove mypy in test_common

* fix

* change preset name

* restart ci

* index tutorial and fix save_path problem

* fix get_num_gpu

* fix save_path

* fix tuto and example typo and path error

* add a TODO and force detection not realtime in _predict

* fix voc2coco

* black

* bug bash + quick start

* merge

* merge

* fix save_path problem for DDP

* black

* update warning

* update tutorial, lint

* update tuto

* update voc to coco

* lint

* refine index

* lint

* update docs in multimodal

* update the multimodal index

* add doc depth

* fix save path init

* restart ci

* restart ci

* fix save_path in fit

* fix save_path

* fix save_path

* update quick start

* update cli in doc

* change prefix coco to postfix cocoformat for voc2coco

* fix s3 path and index error

* restart ci

* install necessary packages

* restart ci

* add build mmcv in .github

* fix mmlab installation

* fix

* fix mmcv install in tuto

* fix

* fix

* fix index

* update docs

* refine quick start

* remove unnecessary warnings

* restart ci

* replace voc2012 to 2007

* fix tuto and cli tool

* merge

* fix docs

* change max depth

* add all detection tuto in multimodal index

* restart ci

* fix max depth

* fix installation error for pycocotools

* remove windows

* Fix hyperparameters in matcher fit (#2404)

* support hyperparameters in matcher fit

* fix

* fix

* [docs] Apple Silicon Instructions (#2403)

* [timeseries] make core and tabular dependencies explicit (#2405)

* Turn off AutoMM prediction progress bar in tabular (#2401)

* save and load enable_progress_bar

* turn off progress bar for automm in tabular

* Remove warnings (#2402)

* replace iteritems with items

* replace iteritems with items

* remove seqeval warning

* Update metric.py

* set num_workers to 0 by default

Some huggingface checkpoints such as deberta-v3, mdeberta-v3, roberta, flan-t5-xl do not work with ddp_spawn when setting num_workers=2

* update ner presets

* add fixme

* Update presets.py

* Improve text_prediction tutorial (#2414)

* fix multimodal tutorial index (#2407)

* Remove warnings and duplicate function (#2409)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* Fix cpu inference bug (#2413)

* fix cpu inference bug

* add test-case of cpu-only inference

* Fix NER best quality preset (#2412)

* Update presets.py

* Update presets.py

* Update presets.py

* Update Quick Start Tutorial and Add Installation Warnings (#2418)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* add warnings on import error, update quick start example

* update quick start doc

* fix wording

* add installation info

* fix

* Remove dependency on vision.imagedataset (#2411)

* reemove imagedataset

* add datasets

* move dataset download to utils

* lint

* lint

* lint

* Fixed model save and load path with uuid for multiple runs (#2415)

* FTT preset (#2410)

* Fix FTTransformer errors, add max_features

* Add FTT presets

* Added parallel bagging on CPU with FTTransformer

* remove ultra preset, add docs

* Set num_workers for image related presets and clean matcher config (#2416)

* clean matcher config

* num_workers

* Fix bad refs in detection tutorial (#2419)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* remove duplicate function

* filter mmcv warning

* lint

* add warnings on import error, update quick start example

* update quick start doc

* fix wording

* add installation info

* fix

* fix bad ref

* [Tutorial] Shorten multilingual tutorial + Improve tutorial website (#2417)

* improve document

* shorten tutorial

* Update index.rst

* update

* revise tutorial.

* update tutorial

* Fix typo

* Adding inference quick start tutorial

* Fixing YOLOv3 description

* Moving mmdet mmcv to top of page

* Changing {.python .input} to python .input

* [0.6 Release][Object Detection][Tutorial] Remove mmdet output format (#2393)

* Adding tutorial for converting data to COCO format.

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* adding docs

* adding docs

* refactor function name

* Reformat

* Reformat: single quote to double quote

* Added inference from pretrained. Added Visualization

* Adding visualization. Now `get_voc_format_classes` dumps class_names when labels.txt is not present.

* Reformat previous commit.

* Addressing issues in PR#2298.

* Addressing usage of voc2coco.py

* Reformat files to pass Lint Check

* Deleting as_pandas. Adding default saving path.

* Reformatting to pass Lint Check

* Reformatting to pass Lint Check

* Reformat imports

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Reformat for Lint

* Reformat for Lint

* Finalizing saving results; Refactored visualization into predictor.py.

* Add running instructions

* Lint check

* Removing unused imports

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* removing mypy

* Adding tutorials for running inference, saving results, and visualization.

* fixing save_path issue

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Add mmdet, mmcv-full links

* Addressing issues in reviews

* Removing .input

* changing quick start coco_*.json to *_cocoformat.json. putting `mim install mmcv-full` before `pip install mmdet`

* Editing visualize_detection to take as input pd.DataFrame. Fixed a bug in save results. Update tutorials

* editting tutorials to reflect pd.DataFrame detection output.

* Update tutorials to print pred

* Update inference quick start tutorial to single image

* Update inference quick start tutorial to single image; Changing matplotlib to PIL for displaying visualization.

* Update inference quick start tutorial to single image; Changing matplotlib to PIL for displaying visualization.

* Fixing quick start import error with numpy.

* Fixed links in inference tutorials, moved inference quick start to quick_start, fixed a typo in detection_eval_fasterrcnn_coco

* fixing kernel died issue

* Removing quick start to another PR

* Remove detection load predictor

* Changing {.python} to python

* Removing quick start in index.rst

* Update to v0.6 (#2425)

* [Release 0.6] [Multimodal] Check if folder is empty before raise (#2426)

* fix

* fix

* lint

* fix

* fix

* Update to v0.6.1

* bump evaluate to `0.3.0` (#2433)

* [Cloud] Cloud Custom Image and Cleanup (#2408)

* checkpoint

* checkpoint

* api

* cleanup old version code

* fix to local

* fix

* fix

* fix

* additional volume size

* fix

* fix

* addressing comments

* fix

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* fix (#2437)

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* 0.6.0 Release notes (#2383)

* Add finetune/eval tests for AutoMM detection (#2441)

* create save_path in init and remove warning in fit

* remove unecessary line

* fix

* add fintune/eval test for detection

* lint

* fix layout (#2450)

* [CI][Cloud] Nightly Build of AG Images for Cloud Testing (#2436)

* initial

fix

fix

fix

checkpoint

checkpoint

checkpoint

api

cleanup old version code

fix to local

fix

fix

fix

additional volume size

fix

fix

tests

* fix

* fix

* enable nightly for cloud CI

* minor fix entry point

* comments

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* Adding Joint IA3_LoRA as efficient finetuning strategy (#2451)

* adding ia3_lora peft and raft#1 preset

* fix

* dynamic max length template choice;

* fix trigger of PEFT

* add coment for 11B model in preset

Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>

* Fix AutoMM warnings about object detection (#2458)

* fix warnings

* ocr constant

* [CI][Fair] Enable CI for Fair Module (#2460)

* fair ci

* fix

* [Cloud][CI] Enable Cloud Lint (#2455)

* lint

* fix

* fix

* isort skip

* gox

* fix

* fix

* fix

* fix

Co-authored-by: Weisu Yin <weisuyin96@gmail.com>

* adding python script to get all unique classes in voc format, and dump all annotation xml files

* Reformat previous commit.

* Adding save results to data frame. Deprecating `get_voc_format_classes` to using `get_detection_classes`. Refactoring saving results to utils.

* Addressing issues in code review: deleting unused functions, refactoring visualization to a separate function call outside predict, removing prints, modifying doc strings, adding TODO to use mmdet visualization later,

* Added in index.rst instructions to install mmdet, add return detection as df by setting as_pandas flag, added quick start tutorial with downloading tiny motorbike, added warning in predict init for mmdet and mmcv-full, reorganized tutorials,

* Deleting inference tutorials, refactoring inference quick start to quick_start_coco.md, fixing the bug for fit->predict with dummy df converter from dict, updating detection unit test

* Deleting inference tutorials, refactoring inference quick start to quick_start_coco.md, fixing the bug for fit->predict with dummy df converter from dict, updating detection unit test

* Addressing comments in reviews.

* Re-organizing tutorial, picking a different image to display

* Deleting inference cards in the object detection index page

* Adding visualizer from detectron2

* Adding visualizer from detectron2 - Complete. Updating quick_start_coco.md

* Editing docs for colormap.py

* Deleting unused classes. Reformatting for lint.

* Reformatting for lint.

* Adding new line

* Add support for filename/filelist input for predict

* add back newline under cv2

* removing imports

* Adding type description

Co-authored-by: Zhiqiang Tang <zhiqiang.tang@rutgers.edu>
Co-authored-by: Nick Erickson <neerick@amazon.com>
Co-authored-by: BingzhaoZhu <39958219+BingzhaoZhu@users.noreply.github.com>
Co-authored-by: tonyhu <tonyhoo@users.noreply.github.com>
Co-authored-by: Caner Turkmen <turkmen.ac@gmail.com>
Co-authored-by: Haoyang Fang <107515844+FANGAreNotGnu@users.noreply.github.com>
Co-authored-by: Yi Zhu <yizhu59@gmail.com>
Co-authored-by: Oleksandr Shchur <shchuro@amazon.com>
Co-authored-by: Shuai Zhang <cheungdaven@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-53-45.us-west-2.compute.internal>
Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>
Co-authored-by: Alexander Shirkov <10080307+gradientsky@users.noreply.github.com>
Co-authored-by: Weisu Yin <weisy@amazon.com>
Co-authored-by: Alexander Shirkov <ashyrkou@amazon.com>
Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
Co-authored-by: Weisu Yin <weisuyin96@gmail.com>
Co-authored-by: Rami <rami-aly@hotmail.de>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-44-212.eu-west-1.compute.internal>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sxjscience sxjscience sxjscience approved these changes

@zhiqiangdon zhiqiangdon zhiqiangdon approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@Raldir @sxjscience @zhiqiangdon