Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: expose allow overlapping property #4697

Conversation

frascuchon
Copy link
Member

@frascuchon frascuchon commented Apr 3, 2024

Description

This PR adds the allow_overlapping field for span questions to configure span questions with overlapping support. The default value is set to False to align the previous version of the span question.

It must be merged after argilla-io/argilla-server#89

span_question = rg.SpanQuestion(name="spans", .., allow_overlapping=True)
...

Closes #4694

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

  • New feature (non-breaking change which adds functionality)
  • Refactor (change restructuring the codebase without changing functionality)
  • Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

(Please describe the tests that you ran to verify your changes. And ideally, reference tests)

  • Test A
  • Test B

Checklist

  • I added relevant documentation
  • I followed the style guidelines of this project
  • I did a self-review of my code
  • I made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I filled out the contributor form (see text above)
  • I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. area: api Indicates that an issue or pull request is related to the Fast API server or REST endpoints language: python Pull requests or issues that update Python code severity: minor Indicates that the issue isn't urgent or blocking team: backend Indicates that the issue or pull request is owned by the backend team type: improvement Indicates updates or improvements on existing features labels Apr 3, 2024
Copy link

github-actions bot commented Apr 3, 2024

The URL of the deployed environment for this PR is https://argilla-quickstart-pr-4697-ki24f765kq-no.a.run.app

Copy link
Contributor

@burtenshaw burtenshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionality looks good.

I left some comments on testing. We could expand testing for all use cases, but I'm not sure how you want the suite to flow.

CHANGELOG.md Outdated Show resolved Hide resolved
src/argilla/client/feedback/schemas/questions.py Outdated Show resolved Hide resolved
@@ -72,15 +72,17 @@ def test_create_dataset_with_span_questions(argilla_user: "ServerUser") -> None:

ds = FeedbackDataset(
fields=[TextField(name="text")],
questions=[SpanQuestion(name="spans", field="text", labels=["label1", "label2"])],
questions=[SpanQuestion(name="spans", field="text", labels=["label1", "label2"], allow_overlapping=True)],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Should we separate these tests so that we have with/without overlap?
  • Should we add records to both that have overlaps or not?
  • Should we assert raises of the overlap error code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I've created a new separate test checking possible values for the allow_overlapping flag.

For related records questions, I will include some integration tests. Validations for responses and suggestions are not covered on this SDK, we will cover them in the new SDK

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Apr 3, 2024
Copy link

codecov bot commented Apr 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 45.84%. Comparing base (ffeb2cd) to head (c0c527d).
Report is 32 commits behind head on feat/overlapped-span-questions.

❗ Current head c0c527d differs from pull request most recent head 99de15c. Consider uploading reports for the commit 99de15c to get more accurate results

Additional details and impacted files
@@                        Coverage Diff                         @@
##           feat/overlapped-span-questions    #4697      +/-   ##
==================================================================
- Coverage                           45.85%   45.84%   -0.01%     
==================================================================
  Files                                 193      193              
  Lines                               11810    11812       +2     
==================================================================
  Hits                                 5415     5415              
- Misses                               6395     6397       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Apr 6, 2024
@frascuchon frascuchon merged commit 1f8aa08 into feat/overlapped-span-questions Apr 8, 2024
13 of 14 checks passed
@frascuchon frascuchon deleted the feat/expose-allow-overlapping-property branch April 8, 2024 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: api Indicates that an issue or pull request is related to the Fast API server or REST endpoints language: python Pull requests or issues that update Python code lgtm This PR has been approved by a maintainer severity: minor Indicates that the issue isn't urgent or blocking size:L This PR changes 100-499 lines, ignoring generated files. team: backend Indicates that the issue or pull request is owned by the backend team type: improvement Indicates updates or improvements on existing features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants