AssertionError when running examples from readme

[HeyangQin]

The example from https://github.com/microsoft/DeepSpeed-MII/blob/main/README.md:

import mii
mii_configs = {"tensor_parallel": 1, "dtype": "fp16"}
mii.deploy(task="text-generation",
           model="bigscience/bloom-560m",
           deployment_name="bloom560m_deployment",
           mii_config=mii_configs)

gives

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/vc_data/users/heyangqin/anaconda3/envs/mii_moe/lib/python3.10/site-packages/mii/deployment.py", line 82, in deploy
    mii.utils.check_if_task_and_model_is_valid(task, model)
  File "/vc_data/users/heyangqin/anaconda3/envs/mii_moe/lib/python3.10/site-packages/mii/utils.py", line 122, in check_if_task_and_model_is_valid
    model_name in valid_task_models
AssertionError: text-generation only supports ['distilgpt2', 'gpt2-large', 'gpt2-medium', 'gpt2-xl', 'gpt2', 'openai-gpt', 'transfo-xl-wt103', 'xlnet-base-cased', 'xlnet-large-cased', '13on/gpt2-wishes', 'AbderrahimRezki/HarryPotterBot', 'Akash7897/gpt2-wikitext2', 'Aleksandar1932/distilgpt2-rock', 'Aleksandar1932/gpt2-country', 'Aleksandar1932/gpt2-hip-hop', 'Aleksandar1932/gpt2-pop', 'Aleksandar1932/gpt2-rock-124439808', 'Aleksandar1932/gpt2-soul', 'Aleksandar1932/gpt2-spanish-classics', 'AlekseyKorshuk/comedy-scripts', 'AlekseyKorshuk/horror-scripts', 'Alerosae/SocratesGPT-2', 'Alireza1044/dwight_bert_lm', 'Alireza1044/michael_bert_lm', 'AntonClaesson/movie-plot-generator', 'Apisate/Discord-Ai-Bot', 'Atampy26/GPT-Glacier', 'Augustvember/wokka', 'Ayah/GPT2-DBpedia', 'Azaghast/GPT2-SCP-ContainmentProcedures', 'Azaghast/GPT2-SCP-Descriptions', 'Azaghast/GPT2-SCP-Miscellaneous', 'BenDavis71/GPT-2-Finetuning-AIRaid', 'BenWitter/DialoGPT-small-Tyrion', 'BigSalmon/GPT2HardArticleEasyArticle', 'BigSalmon/GPT2HardandEasy', 'BigSalmon/GPTHeHe', 'BigSalmon/GPTNeo350MInformalToFormalLincoln', 'BigSalmon/GPTNeo350MInformalToFormalLincoln2', 'BigSalmon/GPTNeo350MInformalToFormalLincoln3', 'BigSalmon/GPTNeo350MInformalToFormalLincoln4', 'BigSalmon/GPTNeo350MInformalToFormalLincoln5', 'BigSalmon/GPTNeo350MInformalToFormalLincoln6', 'BigSalmon/GPTT', 'BigSalmon/InfillFormalLincoln', 'BigSalmon/InformalToFormalLincoln14', 'BigSalmon/InformalToFormalLincoln15', 'BigSalmon/InformalToFormalLincoln16', 'BigSalmon/InformalToFormalLincoln17', 'BigSalmon/InformalToFormalLincoln18', 'BigSalmon/InformalToFormalLincoln19', 'BigSalmon/InformalToFormalLincoln20', 'BigSalmon/InformalToFormalLincoln21', 'BigSalmon/InformalToFormalLincoln22', 'BigSalmon/InformalToFormalLincoln23', 'BigSalmon/InformalToFormalLincoln24', 'BigSalmon/InformalToFormalLincoln25', 'BigSalmon/InformalToFormalLincolnDistilledGPT2', 'BigSalmon/Lincoln4', 'BigSalmon/MrLincoln', 'BigSalmon/MrLincoln10', 'BigSalmon/MrLincoln11', 'BigSalmon/MrLincoln12', 'BigSalmon/MrLincoln125MNeo', 'BigSalmon/MrLincoln13', 'BigSalmon/MrLincoln2', 'BigSalmon/MrLincoln3', 'BigSalmon/MrLincoln4', 'BigSalmon/MrLincoln5', 'BigSalmon/MrLincoln6', 'BigSalmon/MrLincoln8', 'BigSalmon/NEO125InformalToFormalLincoln', 'BigSalmon/Neo', 'BigSalmon/ParaphraseParentheses', 'BigSalmon/ParaphraseParentheses2.0', 'BigSalmon/Points', 'BigSalmon/Points2', 'BigSalmon/SimplifyText', 'Broadus20/DialoGPT-small-harrypotter', 'BumBelDumBel/TRUMP', 'BumBelDumBel/ZORK-AI-TEST', 'BumBelDumBel/ZORK_AI_SCIFI', 'CallumRai/HansardGPT2', 'Cedille/fr-boris', 'Chaewon/mmnt_decoder_en', 'Chaewon/mnmt_decoder_en', 'Chakita/gpt2_mwp', 'CheonggyeMountain-Sherpa/kogpt-trinity-poem', 'Chun/DialoGPT-large-dailydialog', 'Chun/DialoGPT-medium-dailydialog', 'Chun/DialoGPT-small-dailydialog', 'CoderEFE/DialoGPT-medium-marx', 'CoffeeAddict93/gpt1-call-of-the-wild', 'CoffeeAddict93/gpt1-modest-proposal', 'CoffeeAddict93/gpt2-call-of-the-wild', 'CoffeeAddict93/gpt2-medium-call-of-the-wild', 'CoffeeAddict93/gpt2-medium-modest-proposal', 'CoffeeAddict93/gpt2-modest-proposal', 'Coldestadam/Breakout_Mentors_SpongeBob_Model', 'cometrain/neurotitle-rugpt3-small', 'Connorvr/TeachingGen', 'CrisLeaf/generador-de-historias-de-tolkien', 'DHBaek/gpt2-stackoverflow-question-contents-generator', 'DecafNosebleed/scarabot-model', 'DeepESP/gpt2-spanish-medium', 'DeepESP/gpt2-spanish', 'Deniskin/emailer_medium_300', 'Deniskin/gpt3_medium', 'Duugu/alexia-bot-test', 'EasthShin/BTS_Lyrics_GPT-Neo-base', 'EasthShin/Chatbot-LisaSimpson-DialoGPT', 'EasthShin/Youth_Chatbot_Kogpt2-base', 'Ebtihal/AraDiaBERT', 'Ebtihal/AraDiaBERT_V3', 'EleutherAI/gpt-j-6B', 'EleutherAI/gpt-neo-1.3B', 'EleutherAI/gpt-neo-125M', 'EleutherAI/gpt-neo-2.7B', 'ErykWdowiak/GPTalian', 'Eunooeh/mnmt_gpt2', 'Felipehonorato/storIA', 'Ferch423/gpt2-small-portuguese-wikipediabio', 'Finnish-NLP/gpt2-finnish', 'Finnish-NLP/gpt2-large-finnish', 'Finnish-NLP/gpt2-medium-finnish', 'Forest/gpt2-fanfic', 'FutureFanatik/DialoGPT-small-rick', 'Galuh/id-journal-gpt2', 'GammaPTest/e_bot', 'Gantenbein/ADDI-CH-GPT2', 'Gantenbein/ADDI-DE-GPT2', 'Gantenbein/ADDI-FI-GPT2', 'Gantenbein/ADDI-FR-GPT2', 'Gantenbein/ADDI-IT-GPT2', 'GroNLP/gpt2-medium-dutch-embeddings', 'GroNLP/gpt2-medium-italian-embeddings', 'GroNLP/gpt2-small-dutch-embeddings', 'GroNLP/gpt2-small-dutch', 'GroNLP/gpt2-small-italian-embeddings', 'GroNLP/gpt2-small-italian', 'HJK/PickupLineGenerator', 'HScomcom/gpt2-MyLittlePony', 'HScomcom/gpt2-fairytales', 'HScomcom/gpt2-friends', 'HScomcom/gpt2-game-of-thrones', 'HScomcom/gpt2-lovecraft', 'HScomcom/gpt2-theoffice', 'HamidRezaAttar/gpt2-product-description-generator', 'Haotian/distilgpt2-finetuned-wikitext2', 'HeyLucasLeao/gpt-neo-small-emo-lyrics', 'HeyLucasLeao/gpt-neo-small-portuguese', 'HoeioUser/kod', 'HooshvareLab/gpt2-fa-comment', 'HooshvareLab/gpt2-fa-poetry', 'HooshvareLab/gpt2-fa', 'IDEA-CCNL/Wenzhong-GPT2-3.5B', 'IDEA-CCNL/Yuyuan-GPT2-3.5B', 'Iacopo/Shakespear-GPT2', 'Ifromspace/GRIEFSOFT-walr', 'Ifromspace/GRIEFSOFT', 'Inkdrop/gpt2-property-classifier', 'Irina/Fairytale', 'Irina/cyoa_GPT3Medium', 'Irina/fantasy_GPT3Medium', 'Irina/trans_GPT3Medium', 'Irina/trans_cyoa_GPT3Medium', 'Irina/trans_cyoa_rollouted', 'ItelAi/Chatbot', 'JerryQu/v2-distilgpt2', 'Jipski/Flos_gpt-2_erw-02', 'Jipski/Flos_gpt-2_erw', 'Jipski/MegStuart_gpt-2', 'Jipski/gpt2-Flo-BasBoettcher-Chefkoch', 'Jipski/gpt2-Flo-BasBoettcher', 'Jipski/gpt2-FloSolo', 'KK/DialoGPT-small-Rick', 'Kirili4ik/ruDialoGpt3-medium-finetuned-telegram-6ep', 'KoboldAI/GPT-J-6B-Adventure', 'KoboldAI/GPT-J-6B-Janeway', 'KoboldAI/GPT-J-6B-Shinen', 'KoboldAI/GPT-J-6B-Skein', 'KoboldAI/GPT-Neo-125M-AID', 'KoboldAI/GPT-Neo-2.7B-AID', 'KoboldAI/GPT-Neo-2.7B-Horni-LN', 'KoboldAI/GPT-Neo-2.7B-Horni', 'KoboldAI/GPT-Neo-2.7B-Janeway', 'KoboldAI/GPT-Neo-2.7B-Picard', 'KoboldAI/GPT-Neo-2.7B-Shinen', 'KoboldAI/fairseq-dense-1.3B', 'KoboldAI/fairseq-dense-125M', 'KoboldAI/fairseq-dense-13B', 'KoboldAI/fairseq-dense-2.7B-Janeway', 'KoboldAI/fairseq-dense-2.7B', 'KoboldAI/fairseq-dense-355M', 'KoboldAI/fairseq-dense-6.7B', 'KrishParikh/gpt2_imdb_movie_plots', 'Laeyoung/BTS-comments-generator', 'Langame/convai-gpt-j-6B-8bit', 'Langame/distilgpt2-starter', 'Langame/gpt2-starter-2', 'Langame/gpt2-starter', 'Langame/gpt2-waiting', 'LenaT/distilgpt2-finetuned-wikitext2', 'LeoCordoba/beto2beto', 'Littlemilk/autobiography-generator', 'LorenzoDeMattei/GePpeTto', 'Luciano/gpt2-small-portuguese-finetuned-peticoes', 'Luciano/gpt2-small-portuguese-finetuned-tcu-acordaos', 'ML-ass/english_decoder', 'MM98/ft-bz', 'MYX4567/distilgpt2-finetuned-wikitext2', 'MYX4567/gpt2-wikitext2', 'MaalK/DialoGPT-small-Petyr', 'Madhour/gpt2-eli5', 'MaiaMaiaMaia/DialoGPT-medium-PeterParkerBot', 'Mamatha/agri-gpt2', 'Martian/Neo-GPT-Title-Generation-Electric-Car', 'Mary222/GPT2_RU_GAME', 'Mary222/GPT2_Vit', 'Mary222/MADE_AI_Dungeon_model_RUS', 'Mary222/Models_testing_ai', 'Mary222/SBERBANK_RUS', 'Mary222/made-ai-dungeon', 'Meli/GPT2-Prompt', 'Michael711/feinschwarz', 'MicroTurtle/DialoGPT-medium-shawn', 'MilaBromm/TNGMain', 'Milos/slovak-gpt-j-1.4B', 'Milos/slovak-gpt-j-162M', 'Milos/slovak-gpt-j-405M', 'MisterFavourite/Genesis_KJV_fine_tuned', 'MisterFavourite/Sherlock_Holmes_fine_tuned', 'MoeZilla/Chatbot', 'Momerio/meigen_generate_Japanese', 'Motty/DialogGPT', 'MrBananaHuman/kogpt_6b_fp16', 'NYTK/text-generation-news-gpt2-small-hungarian', 'NYTK/text-generation-poem-petofi-gpt2-small-hungarian', 'Narrativa/spanish-gpt2-finetuned-rap-lyrics', 'Narsil/gpt2', 'Narsil/pretrained2', 'NbAiLab/nb-gpt-j-6B', 'Nehc/adpatres', 'Nehc/gpt2_lovecraft_ru', 'Nehc/gpt2_priest_ru', 'Nicki/citadel', 'Nicki/gpt3-base', 'Nicki/scarlet-choir', 'NlpHUST/gpt-neo-vi-small', 'Nokia/nlgp-docstring', 'Nokia/nlgp-natural', 'Norimoji/DialoGPT-medium-FF7', 'Norod78/distilgpt2-base-pretrained-he', 'Norod78/english-sienfeld-distilgpt2', 'Norod78/hebrew-bad_wiki-gpt_neo-tiny', 'Norod78/hebrew-gpt_neo-small', 'Norod78/hebrew-gpt_neo-tiny', 'Norod78/hebrew-gpt_neo-xl-poetry', 'Norod78/hebrew-gpt_neo-xl', 'Norod78/hebrew_poetry-gpt_neo-small', 'Norod78/hebrew_stories-gpt_neo-small', 'Norod78/hewiki-articles-distilGPT2py-il', 'NovelAI/genji-jp', 'NovelAI/genji-python-6B', 'Ochiroo/tiny_mn_gpt', 'OscarNav/dialoGPT_translate', 'Peter/in_g_2', 'Phantomhive/Noelle-bot', 'Phiion/DialoGPT-large-dilucbot', 'PhilSad/GPT-J6B-Guided-SCP', 'PhilSad/gpt-scp-neo-125M', 'PlanTL-GOB-ES/gpt2-base-bne', 'PlanTL-GOB-ES/gpt2-large-bne', 'PolyakovMaxim/ModelGptTS', 'Pyjay/gpt2-medium-dutch-finetuned-text-generation', 'QianWeiTech/GPT2-News', 'QianWeiTech/GPT2-Titles', 'Rai220/test1', 'Ranger/Dial0GPT-small-harrypotter', 'Rocketknight1/codeparrot-ds', 'Rocketknight1/distilgpt2-finetuned-wikitext2', 'Rocketknight1/gpt2-finetuned-wikitext2', 'Rocketknight1/gpt2-wikitext2', 'RollingMuffin/scripts_ru', 'RuRI/Talkmodel01', 'SIC98/GPT2-first-model', 'SIC98/GPT2-python-code-generator', 'SJSui/AstroBot', 'SJSui/NekuBot', 'Sentdex/GPyT', 'Sid51/CB', 'Sid51/Chan', 'Sid51/ChanBot', 'SilentMyuth/stable-jenny', 'SilentMyuth/stablejen', 'SimonThormeyer/movie-plot-generator-longer-plots', 'SimonThormeyer/movie-plot-generator', 'Simovod/simRU', 'Simovod/testSIM', 'SkolkovoInstitute/gpt2-base-gedi-detoxification', 'SouvikGhosh/DialoGPT-Souvik', 'SteveC/sdc_bot_15K', 'SteveC/sdc_bot_medium', 'SteveC/sdc_bot_small', 'SteveC/sdc_bot_two_step', 'StevenShoemakerNLP/pitchfork', 'SuperDoge/DialoGPT-small-harrypotter', 'THUMT/mGPT', 'TajMahaladeen/pokemon_gptj', 'Tanhim/gpt2-model-de', 'Tereveni-AI/gpt2-124M-uk-fiction', 'ThaiUWA/gpt-2-josh-uwa', 'ThaiUWA/gpt2test', 'TheBakerCat/2chan_ruGPT3_small', 'TheGeeKing/DialoGPT-small-Rick', 'TrLOX/gpt2-tdk', 'TristanBehrens/js-fakes-4bars', 'TsinghuaAI/CPM-Generate', 'Ulto/avengeeers', 'Ulto/avengers2', 'Ulto/pythonCoPilot', 'Ulto/pythonCoPilot2', 'Ulto/pythonCoPilot3', 'Vamsi/T5_Paraphrase_Paws', 'VietAI/gpt-j-6B-vietnamese-news', 'VietAI/gpt-neo-1.3B-vietnamese-news', 'Vivek/GPT2_GSM8k', 'Wasabi42/Joker_Model', 'Wintermute/Wintermute', 'Wintermute/Wintermute_extended', 'YYJ/KunquChat', 'Yankee/TEST21', 'Yoshisaur/kono-chat', 'YusufSahin99/IFIS_ZORK_AI_FANTASY', 'YusufSahin99/IFIS_ZORK_AI_HORROR', 'YusufSahin99/IFIS_ZORK_AI_MODERN', 'YusufSahin99/IFIS_ZORK_AI_SCIFI', 'YusufSahin99/Zork_AI_SciFi', 'Zohar/distilgpt2-finetuned-restaurant-reviews', 'aadelucia/GPT2_medium_narrative_finetuned_large', 'aadelucia/GPT2_medium_narrative_finetuned_medium', 'aadelucia/GPT2_small_narrative_finetuned_medium', 'aadilhassan/DialoGPT-small-chandler', 'abbas/gpt2-horror-stories', 'abhinema/distillgpt2', 'abhinema/gpt-medium', 'abhinema/gpt', 'abhinema/testauto', 'abhiramtirumala/DialoGPT-sarcastic-medium', 'abinayam/gpt-2-tamil', 'accelotron/rugpt3-ficbook-bts', 'adamlin/100perc', 'adamlin/topicalchat-multiturn', 'addy88/gpt-j-8bit', 'addy88/gpt-neo-netflix', 'addy88/gptj8', 'adresgezgini/Turkish-GPT-2-Finetuned_digital_ads', 'adresgezgini/turkish-gpt-2', 'ainize/GPT2-futurama-script', 'ainize/gpt2-mcu-script-large', 'ainize/gpt2-rnm-with-only-rick', 'ainize/gpt2-rnm-with-season-1', 'ainize/gpt2-rnm-with-spongebob', 'ainize/gpt2-simpsons-script-large', 'ainize/gpt2-spongebob-script-large', 'akahana/gpt2-indonesia', 'akhooli/gpt2-ar-poetry-aub', 'akhooli/gpt2-ar-poetry-aub_m', 'akhooli/gpt2-ar-poetry', 'akhooli/gpt2-small-arabic-poetry', 'akhooli/gpt2-small-arabic', 'akivo4ka/ruGPT3medium_psy', 'akozlo/con_bal60k', 'akozlo/conserv_fulltext_1_18_22', 'albertbn/gpt2-medium-finetuned-ads-fp16-blocksz512', 'alenusch/rugpt2-paraphraser', 'alenusch/rugpt3-paraphraser', 'alexLopatin/alex-ai', 'alexcg1/models', 'alexcg1/trekbot', 'microsoft/bloom-deepspeed-inference-fp16', 'microsoft/bloom-deepspeed-inference-int8', 'EleutherAI/gpt-neox-20b']

[mrwyattii]

@HeyangQin this is likely because we use huggingface_hub.HfApi to get a list of supported models and you have an older version installed. Can you try updating that package with pip install -U huggingface_hub and try again?

[HeyangQin]

@mrwyattii It works after pip install -U huggingface_hub. Many thanks😃!

[LaniakeaS]

it's that mean this tool need to connect to HuggingFace hub to check the model is valid or not whatsoever, even the model has already downloaded into local machine?