Fix requirements.txt #84
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kudep
approved these changes
Jan 24, 2022
dilyararimovna
added a commit
that referenced
this pull request
Jan 27, 2022
* refactor: agent container volumes and paths * fix: utils/verify_compose.py paths * Fix requirements.txt (#84) * add -pl to agent.run, add argparser to compose verification util * fix line length for codestyle check * blacked utils/verify_compose.py * blacked -ll=120 utils/verify_compose.py * fix compose verification in tests Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Feb 21, 2022
* Fix requirements.txt (#84) * feat: first version files for midas prediction annotator * feat: midas prediction annotator * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * feat: midas predictor new code * feat: requirements * feat: midas dataset * feat: variables and dockerfile * feat: dockerfile * fix: codestyle * feat: midas prediction * feat: formatter * feat: correct context * feat: correct requirements * feat: itsdangerous requirements * feat: model data path * feat: model data path * feat: gunicorn run * feat: dockerfile running * feat: dockerfile running * feat: volumes * feat: reqs * feat: reqs * feat: tests updates * feat: tests * black style * fix: codestyle * fix: upd seq preproc Co-authored-by: Denis Kapelyushnik <8568400+deniskapel@users.noreply.github.com> * fix: docstring Co-authored-by: Denis Kapelyushnik <8568400+deniskapel@users.noreply.github.com> * fix: fixes from review * codestyle * codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Denis Kapelyushnik <8568400+deniskapel@users.noreply.github.com>
dilyararimovna
added a commit
that referenced
this pull request
Feb 21, 2022
* Fix requirements.txt (#84) * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix deployment config * add formatters, fix pipeline * update speech function * sources * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Mar 3, 2022
* Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com>
dilyararimovna
added a commit
that referenced
this pull request
Apr 29, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * factoid qa fixes * update * reformat * update kbqa * codestyle * codestyle * fix tests * codestyle * update requests version * fixes * fixes * fixes * fixes * fixes * add get_factoid function * codestyle * use get_factoid * codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Apr 29, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * update entity detection * rm redundant files * reformat * reformat * codestyle * update requirements * codestyle * add requirements * fixes * fix: analyze_downloads handling incorrect path for agent container * refactor: black analyze_downloads * refactor: ambiguous variable name in analyze_downloads * fixes * codestyle * fixes * Update requirements.txt * fixes * fix pyopenssl requirement * fixes * fixes * fixes * Update test_entity_detection.py * handle case with empty input string * resolve * Update analyze_downloads.py * codestyle * input fixes * logs Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 4, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * entity detection hotfix Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 5, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * dff art skill fixes * dff art skill fixes * codestyle * codestyle * dff art skill fixes * dff art skill fixes * codestyle * codestyle * fix: remove speech funct * fix: use entities func * fix: tests for art skill * fix: no extra logs * move files * codestyle * updtea * fixes Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 6, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix uppercase * fix confidence * fix lowercase * capitalize in ner * use get entities * fixes * fix capitalize Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 12, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 25, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 26, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Jun 28, 2022
* Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com>
dilyararimovna
added a commit
that referenced
this pull request
Jun 30, 2022
* feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com>
dilyararimovna
added a commit
that referenced
this pull request
Aug 24, 2022
* docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dariamitciuk
added a commit
to dariamitciuk/dream_mini
that referenced
this pull request
Aug 31, 2022
* fix/slot extr conf (deeppavlov#156) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (deeppavlov#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (deeppavlov#160) * Feat/transformers intent catcher (deeppavlov#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (deeppavlov#164) * Dream mini (deeppavlov#161) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (deeppavlov#163) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (deeppavlov#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (deeppavlov#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (deeppavlov#172) * Fix/protobuf version (deeppavlov#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (deeppavlov#176) * feat: docker compose with main components * Feat/tests russian (deeppavlov#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (deeppavlov#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (deeppavlov#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (deeppavlov#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (deeppavlov#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (deeppavlov#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (deeppavlov#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (deeppavlov#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (deeppavlov#97) * Fix requirements.txt (deeppavlov#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (deeppavlov#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (deeppavlov#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (deeppavlov#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (deeppavlov#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (deeppavlov#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (deeppavlov#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (deeppavlov#127) * Feat/spacy lemmatizer (deeppavlov#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (deeppavlov#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (deeppavlov#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (deeppavlov#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (deeppavlov#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (deeppavlov#183) * Feat/multilingual ner (deeppavlov#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (deeppavlov#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (deeppavlov#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (deeppavlov#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (deeppavlov#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (deeppavlov#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com>
mtalimanchuk
added a commit
that referenced
this pull request
Sep 15, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * update entity linking * update * update entity ranking * add spelling check * fix tests * codestyle * update memory requirements * codestyle * fix kbqa * codestyle * fix download paths * fixes * update * add dbpedia types * fixes * fixes * fixes * fixes * codestyle * reduce batch size * entity linking on cpu * fixes * update * fixes * Update dockerfile_agent * fix/deeppavlov-url: upd old_dff url * fix/deeppavlov-url: replace all urls * fixes * dockerfile fix * fix: change deeppavlov origin in containers * fix: conveval fixes * fix: midas fixes * fix: requirements and dockerfile * fix: reqs for fact_retrieval * fix: reqs for fact_retrieval * fix: reqs for el * fix: reqs * fix: reqs * fix: reqs * fix: ner * fix: kbqa * fix * codestyle * el update Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Denis Kuznetsov <kuznetsov.den.p@gmail.com>
dariamitciuk
added a commit
to dariamitciuk/dream_mini
that referenced
this pull request
Sep 16, 2022
* fix/slot extr conf (deeppavlov#156) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (deeppavlov#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (deeppavlov#160) * Feat/transformers intent catcher (deeppavlov#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (deeppavlov#164) * Dream mini (deeppavlov#161) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (deeppavlov#163) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (deeppavlov#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (deeppavlov#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (deeppavlov#172) * Fix/protobuf version (deeppavlov#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (deeppavlov#176) * feat: docker compose with main components * Feat/tests russian (deeppavlov#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (deeppavlov#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (deeppavlov#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (deeppavlov#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (deeppavlov#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (deeppavlov#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (deeppavlov#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (deeppavlov#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (deeppavlov#97) * Fix requirements.txt (deeppavlov#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (deeppavlov#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (deeppavlov#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (deeppavlov#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (deeppavlov#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (deeppavlov#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (deeppavlov#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (deeppavlov#127) * Feat/spacy lemmatizer (deeppavlov#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (deeppavlov#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (deeppavlov#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (deeppavlov#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (deeppavlov#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (deeppavlov#183) * Feat/multilingual ner (deeppavlov#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (deeppavlov#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (deeppavlov#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (deeppavlov#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (deeppavlov#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (deeppavlov#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (deeppavlov#193) * Feat/sentence ranker as a service (deeppavlov#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (deeppavlov#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com>
dilyararimovna
added a commit
to dariamitciuk/dream_mini
that referenced
this pull request
Sep 27, 2022
* fix/slot extr conf (deeppavlov#156) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (deeppavlov#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (deeppavlov#160) * Feat/transformers intent catcher (deeppavlov#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (deeppavlov#164) * Dream mini (deeppavlov#161) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (deeppavlov#163) * Fix requirements.txt (deeppavlov#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (deeppavlov#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (deeppavlov#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (deeppavlov#172) * Fix/protobuf version (deeppavlov#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (deeppavlov#176) * feat: docker compose with main components * Feat/tests russian (deeppavlov#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (deeppavlov#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (deeppavlov#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (deeppavlov#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (deeppavlov#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (deeppavlov#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (deeppavlov#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (deeppavlov#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (deeppavlov#97) * Fix requirements.txt (deeppavlov#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (deeppavlov#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (deeppavlov#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (deeppavlov#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (deeppavlov#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (deeppavlov#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (deeppavlov#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (deeppavlov#127) * Feat/spacy lemmatizer (deeppavlov#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (deeppavlov#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (deeppavlov#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (deeppavlov#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (deeppavlov#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (deeppavlov#183) * Feat/multilingual ner (deeppavlov#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (deeppavlov#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (deeppavlov#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (deeppavlov#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (deeppavlov#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (deeppavlov#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (deeppavlov#193) * Feat/sentence ranker as a service (deeppavlov#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (deeppavlov#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com>
dilyararimovna
added a commit
that referenced
this pull request
Sep 27, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * copy * fix short story skill * fix short story skill * fix short story skill * fix short story skill * fix short story skill * fix short story skill * delete from pipeline change service ports * add tests; fixes * change timeouts * black * fix story skill * change storyskill device * black codestyle * exception log * prompt storygpt fixes * story skill fixes * add alternative choosing phrases * codestyle * fix short story skill * fix storygpt model url * change test * fix prompt storygpt * add prompt-storygpt to cpu.yml * change story skill conditions * fix codestyle * fix exceptions * fix condition & noun search pattern * get prev node fix * get prev node fix * change generation params * change generation params * change generation params * fixes * max len * change annotator funcs * change annotator funcs * fixes * black codestyle * split generation in two steps * split generation in two steps * split generation in two steps * split generation in two steps * timeout * timeout * split generation in two parts * fix confidence * fix confidence * fix confidence * check timeout message * fix confidence * black codestyle * change timeout * update service readme * add info in readme * add info in readme & extend scenario * black * extend scenario * extend scenario * black codestyle * fix: docs * feat: upd gpus * fix: gpt distr * fix: add rake-keywords to proxy Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Федор Игнатов <ignatov@gpu1.ipavlov.mipt.ru> Co-authored-by: root <root@LAPTOP-IFDLOTJU.localdomain> Co-authored-by: Яна Шишкина <yashkens@gpu2.ipavlov.mipt.ru>
dilyararimovna
added a commit
that referenced
this pull request
Sep 28, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * update (#2) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (#183) * Feat/multilingual ner (#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com> * image captioning * image captioning * Image captioning (#4) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna and thresholds intent * fix: more obscene words * fix: Russian documentation * fix: image in docs * fix: questions * fix: bad words * feat: ru toxic classifier * fix: toxic check batch hypotheses too * fix: intent responder uses lang * fix * fix: correct usage of human bot utterances * fix: return 5 hypotheses * fix: more hyps, fix reqs * fix: black codestyle * fix: codestyle * fix: codestyle * feat: response selector uses params * fix: requirements * fix: requirements * fix: revmoe dialogpt prev ru * fix: requirements * fix: add dialogrpt again * fix: add dialogrpt * fix: add dialogpt ru * fix: requirements for dialogpt and dialogrpt * fix: return pymorphy to reqs * Feat/ru intent catcher transformers (#171) * fix: itnent catcher ru transformers * fix: ru itnent catcher * fix: intent catcher updated * fix: INTENT_PHRASES_PATH as a main variable * fix: dockerfile updates * fix: test gpu * fix: black style * fix: add tests files * fix: tests * fix: rights on file * fix: regiths on file * fix: regiths * fix: numb hyps * fix: remove without threads * fix: документация * fix: add LET_ME_ASK_YOU_PHRASES * fix: black style * fix: revert extra files * fix: dream mini uses the same params * fix: generative default response * fix: incase of no gpu * fix: resources and gpus consumption * fix: new image * fix: add prompt ones * fix: ru and en version sentsegs Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: proxy usage command (#183) * Feat/multilingual ner (#186) * feat: ner multilingual case_agnostic * fix: ner config * fix: ner dockerfile * fix: upd config * fix: config for ner multilingual * feat: updated config * feat: working ner multilingual * fix: codfestyle * feat: upd spellcheck * fix: add cuda visible devices * fix: cuda visible devices * update fact-retrieval and text-qa (#168) * update fact retrieval * update squad * add answer sentence * update * fixes * update formatter * fixes * add logit ranker * codestyle * codestyle * fixes * codestyle * fix tests Co-authored-by: Дмитрий Евсеев <dmitrij.euseew@yandex.ru> * feat: upd dp-ner with extended version (#189) * feat: upd dp-ner with extended version * fix: upd tests * fix: working for tags * fix: codestyle * fix: user new model * feat: woking * fix: config * fix: upd ner dockerfile * fix: revert format list * fix: change ner for all dists * fix: upd dialogpt en params (#190) * fix: upd dialogpt en params * fix: black style * fix: upd params * fix: context format * fix: context format * fix: codestyle * docker fixes for hydra configuration poc (#34) * docker fixes hydra configuration poc * fix agent installation * fix dp-agent commit in dockerfile_agent * Fix requirements.txt (#84) * update pr against the new main branch * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * minimal reproducible example for new dream * add pem files to gitignore, small agent docker fix * change commit, remove copy settings * fix agent command in base compose file * fix agent installation * fix agent command in other dists * fix commands in readme, add telegram section * update en and ru readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: prompts from dummy skill (#193) * Feat/sentence ranker as a service (#191) * feat: sentence ranker almost * feat: sentence ranker * fix: tests * fix: get scores * fix: codestyle * fix: reqs * fix: flask jsonify * fix: flask jsonify * fix: json types * fix: logs * fix: usage of single scores calcution * fix: codestyle * fix: codestyle * fix: refactor * fix: scores and curr_single_scores * fix: codestyle * feat: log * fix: model and test * fix: upd reqs for kg service (#195) * image captioning * update image captioning server.py * update image captioning server.py * add dream_multimodal * updates * updated pipeline Co-authored-by: dmitrijeuseew <dmitrij.euseew@yandex.ru> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Olga Sofronova <60696748+olkaso@users.noreply.github.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> Co-authored-by: zucchini-nlp <100715397+zucchini-nlp@users.noreply.github.com> * add: files * fix: docs line * fix: codestyule * fix: run command * fix: run command * fix itsdangerous requirements * image captioning * image captioning * Image captioning (#4) * fix/slot extr conf (#156) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix slot extraction confidence Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/simplier skill selection (#159) * feat: simplier skill selector * fix: refactor skill selector * fix: codestyle * fix: get detected_topics * fixes (#160) * Feat/transformers intent catcher (#135) * feat: train intent catcher * feat: intrnt_catcher transformers train and use * feat: intrnt_catcher transformers train and use * feat: intent_catcher server and test * fix: model to cuda * fix: cuda is available * fix: paths * fix: ignore index * fix: black style * fix: paths * fix: paths * fix:model modes * fix: load dataset * fix: no extra info * fix: no extra info * fix: params * fix: paths * fix: paths * fix: paths * fix: dockerfile and downloading * fix: black style * fix: extra im ports * fix: dockerfile * fix: paths and reqs * fix: paths * fix: lineterminator * fix: paths to save model * fix: paths to save model * fix: paths * fix: sentences * fix: test file * fix: working version * fix: working * fix info * files * fix: gpu for tests * fix: gpu distr * fix: codestyle * fix: codestyle * fix: friendship fallback * fix: codestyle * fix: book skill start if lets chat about books * fix: no repeat * fix book turn on * fix: remove extra comments * fix: some changes * fix: use dp based model * fix: random prhases tto * fix: dockerfile * fix: after review * fix: codestyle * fix: yml configs * fix: add files for intent catcher * fix: codestyle * fix: nvidia error fix * fix: upd config * fix nvidia keyring using wget * fix: working version * fix: add files * file path in dockerfile * fix: order of init * fix: codestyle * fix: upd intent catcher version v2 * fix: revert to prev version * fix: tests for upd version * correct scores Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> * fix: proxy dialogpt (#164) * Dream mini (#161) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Add mini version of Dream * Update cpu.yml * Update pipeline_conf.json * Update proxy.yml * fix: cpu only for existing components * fix: cpu only itnent catcher Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/infilling: not used yet (#163) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * infilling added (not tested) * files moved, some paths fixed * [DGM-49] path to model fixed, test added, seems working * takes a batch, bigger test added * assert added to test * assert added to test * minor changes * fix: codestyle * fix: proxy pass * fix: yml configs * fix: refactor infilling and usage * fix: paths * fix: dockerfile * fix: upd files * fix: working version * fix: codestyle * fix: codestyle * fix: works on gpu * gix: readme Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Feat/update dialogpt (#170) * feat: update dialogpt * fix: codestyle * fix: book skill false start * Feat/parametrize response selector (#169) * feat: parameterization in progress * fix: some progress * fix: parametrised * fix: parametrised resp selector * feat: confs for intent responder * working version * fix: black style * fix: codestyle * fix: resp selector * fix: dialogpt params * fix: one more param * fix: codestyle * fix: params * fix: revert dialogpt * feat: change params of dialogpt (#172) * Fix/protobuf version (#173) * fix: fix protobuf version for sentseg * fix: ner protobuf * feat: first russian dream (#176) * feat: docker compose with main components * Feat/tests russian (#90) * feat: runtests russian * fix: test file and elements * feat: tests russian in jenkinsfile * feat: files for tests * fix; program-y name * fix: change to dff-intent-responder-skill * fix: sync with agent updates * fix: cleanup for both runtests * fix: fix path to pipeline conf * fix: remove ner from tests * fix: unbuild english bot before russian tests * fix: codestyle Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> Co-authored-by: Fedor Ignatov <ignatov.fedor@gmail.com> * fix: no sentrewrite needed * Feat/ru program y (#88) * feat: ru program-y version * fix: variable name * fix: russian tests * feat: test files * fix: dff program-y skill for russian * fix: dff program-y skill for russian docker compose * fix: dff program-y skip eng tests * fix: logs * fix: add variable env * fix: revert dangerous skil * fix: type * fix: imports types * fix: line buffering * fix: default value * fix: tests * fix: program-y patterns * Feat/spellchecker levenstein ru (#89) * feat: add files with correct rights * feat: spell check ru * fix: add dockerfile path * fix: add commit * feat: new files * fix: config name * fix: config address * fix: config as a file * fix: config title * fix: consider list not sample * fix: test fix * fix: test codestyle * fix: levenstein tests * fix: levenstein limit memroy * fix: levenshtein spelling * fix: mapping for spelling * feat: batch processing * Feat/ru badwords (#93) * feat: russian obscene words * fix: badlist ru named as en * fix: badlist language * fix: badlist tests passing * Feat/dummy skill ru (#94) * fix: russian dummy responses for russian letters in human utterance * fix: codestyle * fix: black * Feat/ner russian (#92) * feat: ner config * feat: files for ner ru * feat: ner model * feat: ner integration * fix: format yml config * fix: format dockerfile * fix: path to data * fix: tests for ner russian * fix: codestyle * fix: update ner version * add russian entity detection * add russian entity linking * Update requirements.txt * Update ner_chunker.py * fix: rus entity detection tests (#96) * fix: rus entity detection tests * black codestyle * fix codestyle * fix codestyle * fix bug * codestyle * codestyle * codestyle Co-authored-by: dmitry <dmitrij.euseew@yandex.ru> * Feat/intent catcher Ru based on multi lingual USE (#98) * fix: intent catcher params and paths * fix: paths in dockerfile * fix: pintent ru phrases without random ones * fix: random intent phrases * fix: intent training params * fix: intent requirements * fix: intent requirements * fix: download model * fix: model which to download * fix: imports for correct work * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: corrected phrases * fix: ccorrect path to save json threshold * feat: intent data ru json * fix: ccorrect path to save tests * fix: existing var * fix: regular phrases * fix: next test * fix: trianing logs and new threshold * fix: trianing logs and new thre change prhases * fix: change regexps * fix: change thresholds * fix: new template for itnent phrases * fix: etsts ru * feat: upd model * fix: upd logs of training, upd conf value * fix: punctuation * fix: punctuation * est: upd model * fix: training logs * fix: tests * fix: phrases for opinion * feat: upd model * feat: training logs * feat: upd model * fix: tests * fix: remove opinion request intent * feat: upd model * feat: upd model * fix: new train logs * fix: new phrases * fix: min prcision for intent * fix: lower boundary * fix: usage of lib * fix: codestyle * feat: add itsdangerous requirements * fix: spelling preproc endpoint * Feat/dialogpt ru and dff-generative-skill (#97) * Fix requirements.txt (#84) * feat: initialize dialogpt_RU * feat: files init * feat: basic integration of dialogpt_RU * fix: rename dialogpt * fix: dialogpt to device * fix: dialogpt final version * fix: dialogpt test * fix: dialogpt test * fix: dialogpt resources consumption * fix: dialogpt to tests * feat: dff generative skill * feat: dff generative skill * fix: remove extra files * fix: input to dialogpt * fix: input to dialogpt * fix: logging * fix: turn on tests * fix: get dialog from context * fix: get uttrs from context * fix: geempty uttrs * fix: return empty resp * fix: test file * fix: tests * fix: test ratio * add speech_function_* dist * add speech_function_* dist readme * added sf_functions * fix ports * fix:m codestyle * fix deployment config * fix: tests for generative skill * fix: codestyle * add formatters, fix pipeline * update speech function * sources * fix: check if dialogpt is ready * fix: wait services * rename book skill * remove old book skill, update usages * fix readme * fix codestyle * fix codestyle * fix codestyle * fix codestyle line length * move res_cor.json to shared files * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix cpu.yml, dockerfiles and test for sfc, sfp * fix codestyle issues * blacked with -l 120 * following Dilya's holy orders * following Dilya's not so holy orders * fix formatters * fix pipeline * fix pipeline and formatters * Adding timeouts + mapping of book skill * removed old & irrelevant tests * we've set confidence to super level * feat: midas cls sent tokenize only if needed (#101) * feat: midas cls sent tokenize only if needed * feat: take into account tokenized uttrs by bot * fix: codestyle * fix: itsdangerous reqs * fix: docker reqs * fix: check another container * fix: rights for file * fix: coestyle * fix: return tests for intent responder * fix: revert intent responder * fix: review fixes * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Daniel Kornev <daniel@zetuniverse.com> * fix: remove convert and sentseg for now * Feat/dff-intent-responder-skill ru (#99) * feat: prepare new intent responder * fix: responses for intent responder ru * fix: test based on language * fix: path to intent response phrases * fix: remove convert and sentseg * fix: another gpus * fix: file path and logs * fix: env and logs for intent responder * fix: exit response * fix: choose_topic to low prioity intents * feat: tests for ru * fix: tests for exit ru * fix: black codestyle * fix: tests for itnent catcher en * fix: куьщму сщтмуке фтв ыутеыуп акщь еуыеы * feat: turn on generative skill * Feat/wiki parser RU (#114) * update * codestyle * add language parameter * fix: language arg * fix: language arg and revert generative in dockercompose * fix tests * codestyle * fix: tests for ru * fix: language value * fix: ru test results * fix: test pipe * fix: sort types_2hop * fix: black codestyle * fix: tests for en wiki * fix: quotes * fix: codestyle * fix: sort objects * fix: test for wiki parser * fix: codestyle Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Feat/ru friendship skill (#120) * feat: add language parameters * fix: black codestyle * fix: codestyle * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru * fix: dff friendship ru уокротила реплики * fix: dff friendship tests * fix: dff friendship tests * fix: languAGE FOR WIKI * fix: language default value * fix: language default value * fix: language env var * fix: use templates by language * fix: ru templates * fix: no lang env var in common * fix: lang to ackn * fizx: codestyle * feat: default lang value * fix: dummy for russian * fix: no en acks * fix: how are you ru * fix: logs for response functions * fix: logs for condition functions * fix: ru version if what to talk about * feat: ru tests * fix: codestyle * fix: ru condition to resp selector * fix: ru condition to resp selector * fix: logging level and configuration * fix: ascii in tests * fix: add 'user' to dff input * fix: add language env variable everywhere * Feat/dialogrpt ru (#121) * fix: file drafts * feat: files for dialogrpt * feat: dialogrpt pipeline and scores * feat: dialogrpt pipeline and scores * feat: dialogrpt readme * fix: small readme * fix: sno healthcheck * feat: add dialogrpt to pipeline * fix: codestyle * fix: test files * feat: upd packages in dockerfile * fix: path to file * fix: shared file * fix: codestyle * fix: imports * fix: option consider * fix: option consider * fix: codestyle * fix: vars * fix: test file * fix: convert to list predictions * fix: tests * fix: codestyle * fix: codestyle * fix: codestyle * fix: readme * fix: dialogrpt to tests * feat: no extra files, add tokenizer as parameter * fix: codestyle * fix: var name * fix: batch prediction * fix: batch prediction parameter * fix: test choice * fix: format values * fix: codestyle * fix: upd deeppavlov download * fix: dialogrpt container name * fix: dialogrpt as hyp annotator * fix: dialogrpt test * Feat/ru personal info (#125) * fix: ignorecase and no text in code * fix: russian in patterns * fix: language env var * fix: russian patterns and responses * fix: russian patterns and responses * fix: path to file * fix: test for new version * fix: test for en * fix: codestyle * fix: f palceholders * fix: format usage * fix: codestyle * fix: logs * fix: my name is not * fix: homeland pattern fixes * fix: me name is not function * fix: more logs * fix: fix my name is not function * fix: my name is not * fix: do you know my name * fix: test format * fix: test format * fix: test format and mroe tests * fix: test format * fix: more tests * fix: more tests * fix: mtest format prints * fix: black * fix: en tests * fix: en tests * fix: en tests * fix: en tests * fix entity detection (#127) * Feat/spacy lemmatizer (#129) * fix: add spacy annotator * fix: usage of spacy attributes * fix: test spacy annotator * fix: add params * fix: add params * fix: fix test * fix: rights on file * fix: codestyle * fix: extra f string * Feat/russian sentseg (#128) * feat: basic config (with no changes) * feat: data preproc * feat: data processing * fix: codestyle * fix: sentseg ru like dp.ner_rus config * fix: rename config * fix: fpath * fix: readme * fix: custom sentseg reader * fix: custom sentseg config * feat: sent segmentation * feat: sent segmentation tests * fix: rights on file * fix: codestyle * fix: data preproc in sentseg_ru too * fix: metric values for sentseg trained on ru subtitiles * fix: path to sentseg to download * fix: usae sentseg ru model * fux: rights for file * fix: newer spacy version * fix: newer depepavlov version * fix: reqs * fix: server * feat: new config for bert model * fix: upd sentseg config * fix: upd sentseg config * fix: remove old config * fix: config path * fix: deeppavlov 17 2 * fix: remove extra import * fix: new docker image base * fix: reinstall spacy * fix: resentseg tests * fix: codestyle * fix: docs * fix: add sentseg to tests * fix: dockerfile * fix: model path * fix: add dialogrpt o wait hosts * fix: more complicated test for badwords annotator * Fix/upd badlisted words (#130) * fix: more complicated test for badwords annotator * fix: revert badlisted en words * fix: russian badlisted words * fix: give tokenized sents after spacy * fix: ru badlisted words * fix: ru badlisted words folder * fix: ru badlisted words get data * fix: test file * fix: ru badlisted words tokenized sent * fix: ru badlisted words tokens * fix: codestyle * fix: revert badlisted to dev * fix: pipeline conf post_skill_selector_annotators * fix: sleep before re try to connect to dialogpt * fix: formatter format * fix: more russian badwords * fix: correct endpoint for spacy annotator * Feat/ru random questions (#131) * feat: random russian questions * feat: dummy provides random russian questions * fix: refactor questions * fix: add pre-dummy phrase * fix: add pre-dummy phrase * fix: codestyle * fix: path to file * fix: strip russian questions * fix: last chance response * fix: documentation * fix: more confident generative skill * fix: dummy response always available * fix: intent responder check if exist * fix: most dummy responses language based * fix: remove punctuation if present * fix: documentation * fix: documentation * fix: new limits for russian baseline * fix: dialogrpt scores as conveval * fix: sentseg ru remove commas * fix: no wiki-skill yet * fix: ner no threads * fix: can add prompt * fix: prompt with conf * fix: remove bad questions * fix: add punctuation to generated hyp * fix: remove quotes * fix: re-choose hyp only for en version * fix: dff-generative is aka script * fix: increase intent conf thresholds * fix: store only tokens for hyps * fix: consider only special intents * fix: codestyle * fix: final fixes, resp selectiorna …
dilyararimovna
added a commit
that referenced
this pull request
Oct 9, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * To make mistakes is human, to forgive is divine. * Fix * more fixes for God of fixes * more fixes * more fixes for God of fixes * test.yml * fix: running everythin * end? * feat: readme about multilingual version * fix: working mgpt * fix: logs * fix: classes * fix: params for containers * fix: set to cuda * fix: set to cuda * fix: change format of the service * fix: change params of mgpt generation * fix: add all files * fix: ports * fix: remove proxy * fix: remove emotion multilingual * fix: codestyle * fix: codestyle * fix: change to multilingual sentence ranker * fix: remove prev sentence ranker * fix: add params * fix: add params * fix: change json file name * fix: run command * fix: final fixes * feat: test multilingual distr * fix: param from env * fix: args and defaults * fix: pipeline * fix: change ports * fix: test for toxic * fix: test values Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Oct 10, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * increase timeout to 5s * add logs * increase to 100 * add first working version gpt_persona * add sentence_ranking(not working) * fix wrong endpoint * create sentecnce ranker annotator * rewrite text generation logic * add comments to code * fix gpt_persona fallback * clean code, add train script, write tests * add get_intents, remove dataset, add hyperparams * add batch support * fix: move files * fix: merge * fix: codestyle * fix: remove sentence ranker * feat: new distribution and rename skill * feat: new annotator * feat: relative persona extractor * fix: codestyle * fix: proxy * fix: params * fix: volumes * fix: reqs * fix: tests * fix: tests relative sents extr * fix: batching * fix: codestyle * fix: persona extractor tests * fix: persona get * fix: tests * fix: imports * fix: logs * fix: docs * fix: add midas * fix: command * feat: add to main dist and tests * feat: remove infilling, add dialogpt persona based, docs * fix: gpus * fix: params * fix: param * fix: indent * fix: remove infilling from tests Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Dec 2, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * change fallback; add conditions; re-set can continue; * black codestyle * fix second part condition * fix: codestyle * fix: codestyle revert extras * fix fallback node * black codestyle * prohibit stories after questions * fix None confidence * fix: fallback node correctly * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Dec 16, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix index error * add responce variable * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Dec 30, 2022
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix dialogpt-persona-based ports Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: ilyazat <ilyazovazat@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Jan 19, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * add base code for bart_persona_based * change name in configs * misterious cuda error * increase limits, delete int8 mode * update port bart_persona_based * fix codestyle * fix black * add port to test * add proxy, transfer code, change deps * format code * fix batch processing * remove docker buildkit feature * fix persona field * change service name, generation parameters json * add proxy, format files * remove docker build kit * add description * rename constants, add generation config file * fix naming * feat: add seq2seq_persona_based to random skills * fix: rights and dialogpt persona based turn on * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Jan 23, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * first attempt at goal gptj skill * for Dilya to look * put back no_grad * working gptj FAQ skill * gtpj FAQ distributive * increased max_length for generation * added some skills & moved prompt files * changed a lot of names * added my skill to skill_selectors * added telegram.yml * prompt selector annotator added * changed context length but need to check still * removed intent-related skills * separate containers.py & faster prompt-selector * updated README for my skill, still need to calculate requirements * workaround for generation bug with punctuation * fixed length of gen response bug * test files * modified tests for prompt-selector * test for gptj * test for gptj upd * updated example prompts * updated readme * code style * code style acc to black * fixed str length for test * fixed style * code style * style yet again * fixed for Dilya * fixes for Dilya * changed back dream_mini * smaller timeout for prompt selector * some more fixes * small fix * style * changed links * updated pipeline_conf * timeout for is_container_running as an arg * default max_length for generation * added set of skills to skill_selector * compile regex * timeout * confidence update * removed comment * increased confidence * got rid of comments * some minor info fixes * renamed gptj to transformers_lm * one more name change * one more name change * service name fix * another rename * fixed punctuation bug * removed extra * removed extra * working version for telegram * style * style black * tiny fix * updated formatting and names * tiny fix * telegram fix attempt * telegram fix attempt * readme fix * lower confidence * constants * timeout increase for skill * STYLE * Feat/template prompt based skill (#288) * feat: template prompt based skill * fix: change port and separate yml configs * fix: move to env vars * fix: rename template to dream-persona * fix: turn on prompts skill * fix: rename files * fix: copy in dockerfile * fix: copy in dockerfile * fix: formatter * fix: tests for dff * fix: tests for dff * fix: formatter * fix: gpu file * fix: dropping responses * fix: consider some prompts only * fix: readme * fix: persona * feat: readme * feat: readme to the creation * fix: constrict context * fix: add types utterances * fix: upd directories * feat: prompt-based to prompted * feat: prompt-based to prompted * feat: prompt-based to prompted * fix: endpoint for spelling * fix: tests * fix: docs * feat: dream persona prompted * feat: readmes * fix: codestyle * fix: proxy * fix: tests for transformers lm * fix: test * fix: test prompt selector * fix: codestyle * fix: correct var name * fix: tests * fix: codestyle Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Nika Smilga <sunny-august@mail.ru>
dilyararimovna
added a commit
that referenced
this pull request
Jan 25, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * dff image skill * dff image skill * Update docker-compose.yml * dff_image_skill * dff image skill * dff image skill fixed * dff image skill * dff image skill fixed * dff image skill * dff image skill * dff image skill * dff image skill * dff image skill * dff img skill * dff img skill * fix: refactor * fix: refactoring and confs * fix: codestyle * fix: tests Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Nellya D <ndzhubaeva@gmail.com> Co-authored-by: Nellya Dzhubaeva <dzhubaeva@gpu7.ipavlov.mipt.ru>
dilyararimovna
added a commit
that referenced
this pull request
Feb 2, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * add wikidata parsing files * codestyle * update readme --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Feb 15, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * update entity detection * fix * fix * codestyle * fix a typo * add: use related tags * fix test entity linking --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Apr 30, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * add compose definitions to component cards * add host, port, endpoint to pipeline cards * fix authors, add missing pipelines, add template * add prompt-selector template, fix templates * fix template keys * remove unused comet-atomic component * rework service, component cards * move build args and env vars to environment.yml file * fix empty proxy values * fix pipeline confs * fix component connector.annotations * fix wait hosts in response_annotator_selectors service * fix wait hosts in other services, environments * fix response selectors, pipelines * fix ports in response selector * fix generative components * fix: remove spelling and badlists from dream_persona_prompted * fix: ignore all env_secret * fix: new formatters for dff * fix: universal dist does not utilize env_secret * fix: multiskill components and timeouts * fix: remove template_template distribution * fix: deeppavlov assistant * fix: formatters in components * fix: volumes for resp selectors * fix: correct filenames for multiskill * fix: rullama7bru distribution * fix pipelines after merge * fix sentseg annotator source in pipelines * fix agent services, components * fix: sentseg and prompt selector do not wait for spelling * fix: response_annotator_selectors card * fix: timeouts * fix: build args * create services, components for prompt selector * fix prompt selectors environment * fix: flask server * fix: path to prompt selector * fix: required groups * fix: required group skill_selectors * fix: required elements * fix: previous services * fix: correct link to sentseg components card * fix: correct link to sentseg components card * remove unused prompt selector component * remove old configs * fix: rename files without dashes --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 1, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * add compose definitions to component cards * add host, port, endpoint to pipeline cards * fix authors, add missing pipelines, add template * add prompt-selector template, fix templates * fix template keys * remove unused comet-atomic component * rework service, component cards * move build args and env vars to environment.yml file * fix empty proxy values * fix pipeline confs * fix component connector.annotations * fix wait hosts in response_annotator_selectors service * fix wait hosts in other services, environments * fix response selectors, pipelines * fix ports in response selector * fix generative components * fix: remove spelling and badlists from dream_persona_prompted * fix: ignore all env_secret * fix: new formatters for dff * fix: universal dist does not utilize env_secret * fix: multiskill components and timeouts * fix: remove template_template distribution * fix: deeppavlov assistant * fix: formatters in components * fix: volumes for resp selectors * fix: correct filenames for multiskill * fix: rullama7bru distribution * fix pipelines after merge * fix sentseg annotator source in pipelines * fix agent services, components * fix: sentseg and prompt selector do not wait for spelling * fix: response_annotator_selectors card * fix: timeouts * fix: build args * create services, components for prompt selector * fix prompt selectors environment * fix: flask server * fix: path to prompt selector * fix: required groups * fix: required group skill_selectors * fix: required elements * fix: previous services * fix: correct link to sentseg components card * fix: correct link to sentseg components card * remove unused prompt selector component * remove old configs * fix: rename files without dashes --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
May 12, 2023
* feat: update oasst model * fix: model descriptions (#410) * fix: sync dream_persona with demos (#413) * Feat/llm-based and ranking-based dm (#405) * feat: create files for llm based response selection * feat: tests * feat: algorithm * fix: filter bad or toxic uttrs * feat: ranking based response selector * feat: model cards * feat: ranking and refactoring * feat: component files * feat: component description * feat: component description * feat: use gpt-j resp selector * feat: context * feat: utilize hypotheses correctly * fix: configs * fix: configs * fix: mapping * fix: utilize ranker based selector * fix: decay dummy * fix: decay dummy * fix: response * fix: remove old selector * fix: logs * fix: getting annotations * fix: code style * fix: deepy uses resp selector * fix: no language * fix: ru usage * fix: usage of formatter * fix: codestyle * fix: logs * fix: after review * fix: sync * fix: add proxy usage (#416) * feat: new proxy address (#417) * prompt asks chatbot to be concise (otherwise problems with too long answers) (#418) * added gptj to proxy file (#424) * added gptj to proxy file * return newline * fix:dummy score decrease (#423) * Feat/api keys from attributes (#415) * fix: add attributes to formatter * fix: try to get env vars from user utterance * fix: none var values * fix: logs correct * fix: send lowercased keys * prompt to ask fewer questions (#429) * feat: universal dist requires api keys in request (#425) * feat: template_template (#414) * feat: template_template * feat: readme * fix: add chatgpt * fix: tempalte does not use secret env * fixed cutoff for AI utterance (#426) * fixed cutoff for AI utterance * fix for cases with extra ROBOT: etc * style * fix for newline * feat: list of public dists (#433) * fix: remove badlists from prompted dists (#431) * added info about payments (#432) * added info about payments * better descs + reworked fashion prompt * table update * typo fix * typo fix * typo * also increase fashion timeout in yml files * forgot one file * fix: no beauty in table --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * timeout 30 for fairytales (#427) * longer generation and timeout to reduce cutoffs (#420) * longer generation and timeout to reduce cutoffs * also updated in comp and pipe files * change prompts for ai faq and nutrition (#430) * reworked prompts to perform well with oasst * solved conflict * fix: tempalte dist and description (#435) * Feat/multiskill assistant (#434) * feat: multiskill assistant * fix: prompt for meeting notes * fix: waiting for * fix: formatters * fix: dot * fix: secrets * fix: add dream persona skill * fix: add dream persona skill * fix: add to public dists * fix: folder name * fix: description * fix: component cards * fix: component cards * feat: thresholds for toxic_cls (#437) * feat: thresholds for toxic_cls * fix: codestyle * fix: update pip (#439) * fix: urllib neuralcoref * fix: update pip * fix: update pip in spacy annotator * feat: smaller context for prompt selector (#438) * feat: smaller context for prompt selector * fix: index error * Add compose definitions to component cards (#384) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * add compose definitions to component cards * add host, port, endpoint to pipeline cards * fix authors, add missing pipelines, add template * add prompt-selector template, fix templates * fix template keys * remove unused comet-atomic component * rework service, component cards * move build args and env vars to environment.yml file * fix empty proxy values * fix pipeline confs * fix component connector.annotations * fix wait hosts in response_annotator_selectors service * fix wait hosts in other services, environments * fix response selectors, pipelines * fix ports in response selector * fix generative components * fix: remove spelling and badlists from dream_persona_prompted * fix: ignore all env_secret * fix: new formatters for dff * fix: universal dist does not utilize env_secret * fix: multiskill components and timeouts * fix: remove template_template distribution * fix: deeppavlov assistant * fix: formatters in components * fix: volumes for resp selectors * fix: correct filenames for multiskill * fix: rullama7bru distribution * fix pipelines after merge * fix sentseg annotator source in pipelines * fix agent services, components * fix: sentseg and prompt selector do not wait for spelling * fix: response_annotator_selectors card * fix: timeouts * fix: build args * create services, components for prompt selector * fix prompt selectors environment * fix: flask server * fix: path to prompt selector * fix: required groups * fix: required group skill_selectors * fix: required elements * fix: previous services * fix: correct link to sentseg components card * fix: correct link to sentseg components card * remove unused prompt selector component * remove old configs * fix: rename files without dashes --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * Fix/remove duplicates (#443) * fix: remove duplicating files * fix: remove duplicating files * Multiskill update (#440) * long gen, long timeout, more prompts * prompt update * timeouts * added longer context to persona * longer max_tokens * fix: increase to 20 sec * fix: used config * fix: dream persona openai * fix:multiskill components --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * Fix/increase all timeouts (#444) * fix: increase all timeouts * fix: increase all timeouts in pipelines * fix: 7sec to 20 also * fix: 5sec to 20 also for dream persona * fix: rebase and change to new model * fix: upd model * fix: display name --------- Co-authored-by: Nika Smilga <42929200+smilni@users.noreply.github.com> Co-authored-by: Maxim Talimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com>
dilyararimovna
added a commit
that referenced
this pull request
Jun 22, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * fix entity offsets * codestyle --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Aug 6, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * updated MODELS.md table: added info about models' licensing and commercial use + merged link+name cols to improve overall readability and decrease redundancy * Update MODELS.md fixed "is" for better consistency * fix: format table and add new models back * fix: sizes of models on gpu * updated table --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
smilni
pushed a commit
that referenced
this pull request
Aug 8, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * updated MODELS.md table: added info about models' licensing and commercial use + merged link+name cols to improve overall readability and decrease redundancy * Update MODELS.md fixed "is" for better consistency * fix: format table and add new models back * fix: sizes of models on gpu * updated table --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com>
dilyararimovna
added a commit
that referenced
this pull request
Aug 9, 2023
* robot first commit (no cards) * feat: do not use sentence ranker url from env (#535) * Feat/ruxglm prompted dist (#528) * feat: ignore env secret ru * feat: add access token * feat: distribution ruxglm * fix: ruxglm cards * fix: use use_auth_token * fix: eos tokens type * fix: stats cpu ram * fix: skills cards * fix: components cards * fix: eos tokens * fix: eos tokens * fix: eos tokens * fix: eos tokens * fix: cards and table * fix: ADDITIONAL_EOS_TOKENS * fix: ADDITIONAL_EOS_TOKENS * fix: codestyle * fix: universal * fix: dream persona ru skill name * fix: prompt selector ru * fix: replacement * fix: prompt selecrto * fix: use params * fix: timeout and history * fix: trye very short persona * fix: increase timeout * fix: sub replacement tokens correctly * fix: sub replacement tokens correctly * fix: use stopping criteria * fix: typing * fix: revert long persona * fix: duplicate spaces * fix: correct components for russian distribution * fix: proxy for russian distribution * feat: universal distr for ru * feat: universal distr for ru * fix: remove extra * fix: working configs * fix: configs * feat: ruxglm prompted dists * fix: component cards * fix: container name * fix: remove extra space after new line * fix: remove extra space after new line * feat: tests for dream ruxglm * fix: proxy and ru lang * fix: change port of universal ru * fix: rights on file * fix: tests skills * fix: test for resp selector * fix: tests for proxied components * fix: remove do sample true * fix: generative params * feat: used sentence ranker url * feat: utilized default llm * Feat/ru prompted dists (#532) * feat: ignore env secret ru * feat: add access token * feat: distribution ruxglm * fix: ruxglm cards * fix: use use_auth_token * fix: eos tokens type * fix: stats cpu ram * fix: skills cards * fix: components cards * fix: eos tokens * fix: eos tokens * fix: eos tokens * fix: eos tokens * fix: cards and table * fix: ADDITIONAL_EOS_TOKENS * fix: ADDITIONAL_EOS_TOKENS * fix: codestyle * fix: universal * fix: dream persona ru skill name * fix: prompt selector ru * fix: replacement * fix: prompt selecrto * fix: use params * fix: timeout and history * fix: trye very short persona * fix: increase timeout * fix: sub replacement tokens correctly * fix: sub replacement tokens correctly * fix: use stopping criteria * fix: typing * fix: revert long persona * fix: duplicate spaces * fix: correct components for russian distribution * fix: proxy for russian distribution * feat: universal distr for ru * feat: universal distr for ru * fix: remove extra * fix: working configs * fix: configs * feat: ruxglm prompted dists * fix: component cards * fix: container name * first dist (no cards) * fix: remove extra space after new line * fix: remove extra space after new line * feat: tests for dream ruxglm * fix: proxy and ru lang * fix: change port of universal ru * fix: rights on file * fix: tests skills * fix: test for resp selector * multiskill_ru_assistant * fix: tests for proxied components * fairytale and action stories dists * journalist helper dist * fairytale fixes * one more fix * action stories cards * add quotation marks * fairytale cards * storyteller cards * journalist helper cards * multiskill ru cards * agent services cards * minor fixes * fix: utilize sentence ranker url --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * update components.tsv (#537) * update components.tsv * tabulation * Feat/rugpt 3.5 distribution (#534) * feat: ignore env secret ru * feat: add access token * feat: distribution ruxglm * fix: ruxglm cards * fix: use use_auth_token * fix: eos tokens type * fix: stats cpu ram * fix: skills cards * fix: components cards * fix: eos tokens * fix: eos tokens * fix: eos tokens * fix: eos tokens * fix: cards and table * fix: ADDITIONAL_EOS_TOKENS * fix: ADDITIONAL_EOS_TOKENS * fix: codestyle * fix: universal * fix: dream persona ru skill name * fix: prompt selector ru * fix: replacement * fix: prompt selecrto * fix: use params * fix: timeout and history * fix: trye very short persona * fix: increase timeout * fix: sub replacement tokens correctly * fix: sub replacement tokens correctly * fix: use stopping criteria * fix: typing * fix: revert long persona * fix: duplicate spaces * fix: correct components for russian distribution * fix: proxy for russian distribution * feat: universal distr for ru * feat: universal distr for ru * fix: remove extra * fix: working configs * fix: configs * feat: ruxglm prompted dists * fix: component cards * fix: container name * fix: remove extra space after new line * fix: remove extra space after new line * feat: tests for dream ruxglm * fix: proxy and ru lang * fix: change port of universal ru * fix: rights on file * fix: tests skills * fix: test for resp selector * fix: tests for proxied components * feat: rugpt-3.5 by sber in universal russian distribution * fix; wait for it * fix: models card * fix: models card * fix: add to list * fix: change port * fix: change port * fix: change size to correct * feat: instruction how to add a new model * fix ru prompt selector, remove unused component (#538) * feat: replace oasst12b with gptjt (#541) * Feat/utilize rugpt35 (#540) * feat: utilize rugpt35 * feat: tests for jounrlist rugpt35 * feat: tests for jounrlist rugpt35 * fix: rights for tfile * feat: names * fix: ru_dists_names_and_prompts (#543) * rename ruxglm to u * more renaming * tabs * tabs * some more renaming * short prompt * many cards and name changes * fix typo * fixes for Dilya * tiny fix * tiny fix * huge name check * names * typo prompt * fix: no tests for non existing skills --------- Co-authored-by: dilyararimovna <dilyara.rimovna@gmail.com> * fix: cards for ru dists (#544) * fix: rugpt35 config and envs (#546) * Summarization models (#393) * Added abstractive summarization model for English texts * Added abstractive summarization model for Russian texts * Added summarization annotator * Moved rut5 summarizer to dream_russian * Changed endpoint * Added model path to Dockerfile * Updated test * Updated summarization annotator input * Updated test * Changed summarization service url * Changed test * Increased timeout * Updated ram_usage * Updated ports * Updated models cards * Added more info messages * Fixed path error * Added summarization output to bot attributes * Added timeout param to dockerfile * Updated model cards and ports * Fixed problem with incorrect batch processing * Updated summarization save format * Updated dialog summarization model * Updated tests * Minor formatting changes * Fixed black and flake8 codestyle * Fixed black codestyle * Updated models ports * Small fixes * Models table upd (#539) * Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * updated MODELS.md table: added info about models' licensing and commercial use + merged link+name cols to improve overall readability and decrease redundancy * Update MODELS.md fixed "is" for better consistency * fix: format table and add new models back * fix: sizes of models on gpu * updated table --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> * fix: anthropic model params (#547) * fix summarization annotator card (#549) * add cards for prompted robot * ports and n_utt * port * increase WAIT_HOSTS_TIMEOUT in cards --------- Co-authored-by: Dilyara Zharikova (Baymurzina) <dilyara.rimovna@gmail.com> Co-authored-by: Maxim Talimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Nikolay <99244955+Kolpnick@users.noreply.github.com> Co-authored-by: Anastásis <43078815+nstsj@users.noreply.github.com> Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com>
dilyararimovna
added a commit
that referenced
this pull request
Oct 10, 2023
* Fix requirements.txt (#84) * fix itsdangerous requirements * pin itsdangerous requirements for all flask==1.1.1 servers * Theoretically working version * f * j * j * h * Avoid circular imports * Update combined_classifier_ru.json * Removed circular imports * Removed circular imports * Test skill passed * j * Moved join_functions to separate file to avoid circular import * Moved join_functions to separate file to avoid circular import * passing codestyle * h * h * gg * gg * gg * gg * codestyle * Not to wait russian toxic_classification as it was replaced * j * Tests must pass now * Update service.yml * Update environment.yml * Create docker_compose_id.yml * Rename docker_compose_id.yml to 5eb485871be3.yml * Update pipeline_conf.json * Update pipeline_conf.json * Create 3dcc0944c3e0.yml * Update pipeline_conf.json * Update 5eb485871be3.yml * Update pipeline_conf.json * Update 3dcc0944c3e0.yml * Update 5eb485871be3.yml * Update pipeline_conf.json * was erroneous edit * Update 5eb485871be3.yml --------- Co-authored-by: Andrii.Hura <54397922+AndriiHura@users.noreply.github.com> Co-authored-by: mtalimanchuk <mtalimanchuk@gmail.com> Co-authored-by: Dilyara Baymurzina <dilyara.rimovna@gmail.com> Co-authored-by: Dmitry <dimakarp1996@github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Wrong repository. Commit with corresponding hash in different repo:
deeppavlov/dialog_flow_engine@3a2e3e5