Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validate predictions fails on Text2Text #2252

Closed
dvsrepo opened this issue Jan 27, 2023 · 1 comment · Fixed by #2271 or #2304
Closed

Validate predictions fails on Text2Text #2252

dvsrepo opened this issue Jan 27, 2023 · 1 comment · Fixed by #2271 or #2304
Assignees
@leiyre @keithCuniah
Labels
type: bug Indicates an unexpected problem or unintended behavior
Milestone
v1.3.0

Comments

@dvsrepo
Copy link
Member

dvsrepo commented Jan 27, 2023

Describe the bug
On version 1.2.1 when validating the prediction on text2text (see for example news-summary dataset, we get an error that the annotation couldn't be validated. The reason is that apparently, the JSON payload that we send does not contain the text field inside the annotation.text objects.

To Reproduce
Steps to reproduce the behavior:

  1. Go to news-summary
  2. Open the browser inspector and refresh if neccesary
  3. Click on Hand labelling
  4. Find a record with predictions but not validated
  5. You'll see the error pop and on the inspector, you'll that the call to save the annotations fails because the above missing field text error.

I've only tested with page size 1 but I guess it's the same with other page sizes as it shouldn't affect this

Expected behavior
Save the annotation when validating the predictions

Screenshots
Screenshot 2023-01-27 at 10 13 26

Screenshot 2023-01-27 at 10 13 33

Environment (please complete the following information):

  • OS [e.g. iOS]:
  • Browser [e.g. chrome, safari]:
  • Argilla Version [e.g. 1.0.0]:
  • ElasticSearch Version [e.g. 7.10.2]:
  • Docker Image (optional) [e.g. argilla:v1.0.0]:

Additional context
Add any other context about the problem here.
@dvsrepo dvsrepo added the type: bug Indicates an unexpected problem or unintended behavior label Jan 27, 2023
@dvsrepo dvsrepo added this to the v1.3.0 milestone Jan 27, 2023
@keithCuniah keithCuniah mentioned this issue Feb 1, 2023
Merged
13 tasks
@keithCuniah
Copy link
Contributor

Note : It seems that was failling in previous release also ( I've checked 1.2.0/1.x/0.7)

keithCuniah added a commit that referenced this issue Feb 1, 2023
@keithCuniah
fix: validate predictions fails on text2text (#2271)
f68856e 
# Description
Fix validate button action in text2text task 

Please include a summary of the changes and the related issue. Please
also include relevant motivation and context. List any dependencies that
are required for this change.

Closes #2252 

**Type of change**

(Please delete options that are not relevant. Remember to title the PR
according to the type of change)

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Refactor (change restructuring the codebase without changing
functionality)
- [ ] Improvement (change adding some improvement to an existing
functionality)
- [ ] Documentation update

**How Has This Been Tested**

(Please describe the tests that you ran to verify your changes. And
ideally, reference `tests`)

- [x] Text2Text validate whitout modifying the text area
- [x] Text2Text validate after modifying the text area
- [x] Text2Text refresh the page after modifying the text area => the
text is updated

**Checklist**

- [x] I have merged the original branch into my forked branch
- N/A I added relevant documentation
- [x] follows the style guidelines of this project
- [x] I did a self-review of my code
- N/A I added comments to my code
- N/A I made corresponding changes to the documentation
- [x] My changes generate no new warnings
- N/A I have added tests that prove my fix is effective or that my
feature works

---------

Co-authored-by: keithCuniah <keithcuniah@gmail.comh>
@frascuchon frascuchon mentioned this issue Feb 7, 2023
Merged
frascuchon added a commit that referenced this issue Feb 9, 2023
@frascuchon
chore: prepare release v1.3.0 (#2304)
de0983f 
# Changelog

All notable changes to this project will be documented in this file. See
[standard-version](https://github.com/conventional-changelog/standard-version)
for commit guidelines.

## [1.3.0](v1.2.1...v1.3.0)
(2023-02-09)


### Features

* better log error handling
([#2245](#2245))
([66e5cce](66e5cce)),
closes [#2005](#2005)
* Change view mode order in sidebar
([#2215](#2215))
([dff1ea1](dff1ea1)),
closes [#2214](#2214)
* **Client:** Expose keywords dataset metrics
([#2290](#2290))
([a945c5e](a945c5e)),
closes [#2135](#2135)
* **Client:** relax client constraints for rules management
([#2242](#2242))
([6e749b7](6e749b7)),
closes [#2048](#2048)
* Create a multiple contextual help component
([#2255](#2255))
([a35fae2](a35fae2)),
closes [#1926](#1926)
* Include record event_timestamp
([#2156](#2156))
([3992b8f](3992b8f)),
closes [#1911](#1911)
* updated the `prepare_for_training` methods
([#2225](#2225))
([e53c201](e53c201)),
closes [#2154](#2154)
[#2132](#2132)
[#2122](#2122)
[#2045](#2045)
[#1697](#1697)


### Bug Fixes

* **Client:** formatting caused offset in prediction
([#2241](#2241))
([d65db5a](d65db5a))
* **Client:** Log remaining data when shutdown the dataset consumer
([#2269](#2269))
([d78963e](d78963e)),
closes [#2189](#2189)
* validate predictions fails on text2text
([#2271](#2271))
([f68856e](f68856e)),
closes [#2252](#2252)


### Visual enhancements

* Fine tune menu record card
([#2240](#2240))
([62148e5](62148e5)),
closes [#2224](#2224)
* Rely on box-shadow to provide the secondary underline
([#2283](#2283))
([d786171](d786171)),
closes [#2282](#2282)
[#2282](#2282)


### Documentation

* Add deploy on Spaces buttons
([#2293](#2293))
([60164a0](60164a0))
* fix typo in documentation
([#2296](#2296))
([ab8e85e](ab8e85e))
* Improve deployment and quickstart docs and tutorials
([#2201](#2201))
([075bf94](075bf94)),
closes [#2162](#2162)
* More spaces! ([#2309](#2309))
([f02eb60](f02eb60))
* Remove cut-off sentence in docs codeblock
([#2287](#2287))
([7e87f20](7e87f20))
* Rephrase `to know more` into `to learn more` in Quickstart login page
([#2305](#2305))
([6082a26](6082a26))
* Replace leftover `rubrix.apikey` with `argilla.apikey`
([#2286](#2286))
([4871127](4871127)),
closes [#2254](#2254)
[#2254](#2254)
* Simplify token attributions code block
([#2322](#2322))
([4cb6ae1](4cb6ae1))
* Tutorial buttons
([#2310](#2310))
([d6e02de](d6e02de))
* Update colab guide
([#2320](#2320))
([e48a7cc](e48a7cc))
* Update HF Spaces creation image
([#2314](#2314))
([e4b2a04](e4b2a04))


## As always, thanks to our amazing contributors!


- add repr method for Rule, Dataset. (#2148) by @Ankush-Chander
- opensearch docker compose file doesn't run (#2228) by @kayvane1
- Docs: fix typo in documentation (#2296) by @anakin87
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@leiyre leiyre

@keithCuniah keithCuniah

Labels
type: bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Milestone
v1.3.0
3 participants
@dvsrepo @leiyre @keithCuniah