-
Notifications
You must be signed in to change notification settings - Fork 345
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] CoNLL-2003 dataset -Ner, Chunk and POS tagger #170
Conversation
Oh wow, you have the hardest one. We'll try to help out. |
templates/conll2003/templates.yaml
Outdated
\n}) %}\nGenerate named entities from the following sentence. \n{{\"\"}}\n{%\ | ||
\ for i in tokens -%}\n {{- \" \" if not loop.last else \"\" -}}\n {{\ | ||
\ i }}\n{% endfor %} \n|||\n{% set flag = 0 %}\n{% set outer_cnt = namespace(value=0)\ | ||
\ -%}\n{% for tok in tokens -%}\n {% set inner_cnt = namespace(value=0) -%}\n\ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks very complex. If there a reason you can't use loop index?
https://stackoverflow.com/questions/1567291/get-loop-index-of-outer-loop
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
templates/conll2003/templates.yaml
Outdated
reference: Natural question | ||
87bb05ff-f6bf-4c0d-bbcc-56f75095f4a1: !Template | ||
id: 87bb05ff-f6bf-4c0d-bbcc-56f75095f4a1 | ||
jinja: "{% set _chunk_lable_dict = ({\n0:\"O\",\n1:\"B-ADJP\",\n2:\"I-ADJP\",\n\ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lable - > label
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
As suggested, changes are in this commit, 6539b83 New Prompt: 3 additional prompt added for ner, pos and chunks by this commit dc8d60e . The prompts intends to solve the following issue,
So I randomly TODO:
Issue:
For convenience, A template that selects label randomly looks like below. I think the test is not passing because of
Please also look into this issue before merge, #174 |
More details on random label selection by prompt We always include "O" tag for ner, pos and chunk labels. "O" lable indicated it is "other" (uncategorized) label. In case of parts-of-speech tags, there are few labels that are weird. For example see a prompt with all pos labels,
In my opinion, here first 9 labels should be normalized. However in the regular prompt I didn't change them. But in the random label selection prompt named as
This will convert the first 9 weird label as "O" tag. Note that, there is not "O" in dataset for pos labels. |
* first commit * respond to review * rephrase scinario to more reaonsble words
Co-authored-by: Jason Fries <jfries@mini.local>
* Add choice filter * Change random to choice in templates * Fix test
* Prompts for esnli * Update templates.yaml Changed neutral to neither entail nor contradict
* copying prompts to `winogrande` subsets * update templates according to the comments
* Add workaround for error in template highlighting. * Found that this bug can also raise a TypeError.
…l task. (#141) * add task template field * update docs * update docs * update
* Add templates for the MC-TACO dataset * Fixes
* Add template for snips_built_in_intents * add spaces in intent options; task template checkbox
* added opus_tedtalks translation templates * Added 5 templates for the mwsc dataset * tempotarily removed opus_tedtalks * Temporarily removed opus_tedtalks * Updated mwsc templates - replace for loops with joins since always 2 options * Added templates for the qasc dataset * added trims to options in qasc templates * added templates for selqa * modified variable declaration in template * modified variable declaration in template * modified variable declaration in template * using choice instead of random filter Co-authored-by: Rachel Bawden <rachel.bawden@inria.fr@users.noreply.github.com>
* templates for wiki_qa dataset * fixed typos * added squad v2 prompts * added two more templates and modified existing templates with variety of prompting sentences * factorized templates with conditionals
* Add MDD templates * Remove task4_reddit templates * Update mdd templates * Fix typo * Fix mdd templates * Fix typo * Fix qa_recs template * Fix qa_recs template * Fix qa_recs template * Fix qa_recs template * Fix qa_recs template * Replace random with choice * Remove choice_idx from recommend_movie_2 * Add choice_idx global * Fix globals
* feat: climate fever prompt templates. Signed-off-by: Matteo Manica <drugilsberg@gmail.com> * feat: updating labels and adding more templates - paired templates as suggested in the PR review. - a template considering al evidence statements. Signed-off-by: Matteo Manica <drugilsberg@gmail.com> * feat: adding templates for ncbi_disease. Signed-off-by: Matteo Manica <drugilsberg@gmail.com> * fix: setting variable before loop. Adding task template flag. Signed-off-by: Matteo Manica <drugilsberg@gmail.com> Co-authored-by: Matteo Manica <tte@zurich.ibm.com>
* Add neural_code_search_prompts * Add social_i_qa prompts * Improve template for more compact version + randomly select answer for classification task * Update prompt social_i_qa3 to better ask for a index type answer, and migrate random to choice * Check task template * Maske task template false for question hallucination
* Templates for stsb_multi_mt_en * Added rounding and protected value in the prompt * Fixing issues and checking the box
* add: prompt for atmoic * add common gen dataset template * removed template for atomic * add: template with choice and sentence to concept for common_gen * indicated task template for common_gen
* Add templates for DuoRC * Fix templates * Add changes from review * Add answerable_question back to ParaphraseRC * Fix structure issues with templates * Fix styling issues * Add choice for duorc templates
* wiki split * more templates and removed ' ''
* Add app reviews template * Fix template * Fix typo * Fix app reviews templates * Fix app_reviews template
* Prompts for quartz * Update templates.yaml All marked as task templates
* Add scientific papers templates * Add changes from review * Fix pubmed templates * Fix scientific_papers templates * Update scientific_papers templates
* Templates for spider * Adding recommendations and checking the box * Adding reverse prompts
* Prompts for art more natural punctuation * Change sentence to statement Yest that's more accurate
* added ansq * removed bad template
* newspop template * made prompts clearer and capitalized relevant words
Deleting pull request. Creating a new one with updated |
As the dataset is smaller, I only add two prompts for each of the tasks.
Two major issue to follow,
[ISSUE 1]:
After writing the template, on the right side, there is a sample shown in the app.
I see
<p>
tags inside the samples (from theinspect
of the browser). I am not sure if this is because of HTML or not.[ISSUE 2]:
After writing the template I don't see any error generated. But when I go the
Prompted dataset viewer
I get the following error,