Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Annotation agent] Agent is not changed when validating records with annotation but default status #1447

Closed
dvsrepo opened this issue May 3, 2022 · 0 comments · Fixed by #1480
Closed

[Annotation agent] Agent is not changed when validating records with annotation but default status #1447

dvsrepo opened this issue May 3, 2022 · 0 comments · Fixed by #1480
Assignees
@frascuchon @leiyre
Labels
type: bug Indicates an unexpected problem or unintended behavior
Projects
Release
Milestone
v0.15.0

Comments

@dvsrepo
Copy link
Member

dvsrepo commented May 3, 2022

Describe the bug
I have not reproduced this myself, so this should be tested. It was raised yesterday during a session with one of our production users.

The workflow:

  • They created text classification records with both annotation and prediction and status = Default.
  • In this way, it's easy for the annotators to just validate those records that are already labeled correctly.
  • Users validate records (they actually review ~3000 per week).
  • The issue is that only records for which they change the annotation get assigned the annotation_agent. For those validated by accepting the annotations, the annotation_agent is still the machine_process agent.

I think the proper way would be to change the annotation_agent even if no annotation was modified as the user is confirming they agree with the annotation so the record is manually annotated.

To Reproduce
Not possible on my side today
@dvsrepo dvsrepo added type: bug Indicates an unexpected problem or unintended behavior app labels May 3, 2022
@dvsrepo dvsrepo added this to Backlog in Release via automation May 3, 2022
@frascuchon frascuchon moved this from Backlog to Planified in Release May 5, 2022
leiyre added a commit that referenced this issue May 10, 2022
@leiyre
fix(#1447): change agent when validating records with annotation but …
7717310 
…default status

fix #1447

This PR overwrites the agent when validating records with annotation but default status
leiyre added a commit that referenced this issue May 10, 2022
@leiyre
fix(#1447): change agent when validating records with annotation but …
5928c2c 
…default

fix #1447

This PR overwrites the agent when validating records with annotation but default status
@leiyre leiyre mentioned this issue May 10, 2022
Merged
@leiyre leiyre linked a pull request May 10, 2022 that will close this issue
fix(#1447): change agent when validating records with annotation but default status #1480
Merged
@leiyre leiyre moved this from Planified to In progress in Release May 10, 2022
@leiyre leiyre moved this from In progress to Pending Review in Release May 11, 2022
@frascuchon frascuchon added this to the v0.15.0 milestone May 11, 2022
frascuchon pushed a commit that referenced this issue May 12, 2022
@leiyre
fix(#1447): change agent when validating records with annotation but …
eadb3df 
…default status (#1480)
@frascuchon frascuchon moved this from Pending Review to Waiting Release in Release May 12, 2022
@frascuchon frascuchon moved this from Waiting Release to Ready to Release QA in Release May 19, 2022
frascuchon pushed a commit that referenced this issue May 19, 2022
@leiyre @frascuchon
fix(#1447): change agent when validating records with annotation but …
126e6f4 
…default status (#1480)

(cherry picked from commit eadb3df)
@leiyre leiyre moved this from Ready to Release QA to Approved Release QA in Release May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@frascuchon frascuchon

@leiyre leiyre

Labels
type: bug Indicates an unexpected problem or unintended behavior
Projects
No open projects
Release
Approved Release QA
Milestone
v0.15.0
3 participants
@dvsrepo @frascuchon @leiyre