Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Accept non searchable fields in metadata #2570

Merged
merged 7 commits into from
Mar 21, 2023

Conversation

frascuchon
Copy link
Member

@frascuchon frascuchon commented Mar 17, 2023

Description

This PR includes changes for those cases when users want to add some fields in metadata but do not enable searches on them with a leading underscore _.

Refs: #2571

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

  • New feature (non-breaking change which adds functionality)
  • Improvement (change adding some improvement to an existing functionality)

Checklist

  • I have merged the original branch into my forked branch
  • I added relevant documentation
  • follows the style guidelines of this project
  • I did a self-review of my code
  • I made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

@frascuchon frascuchon marked this pull request as ready for review March 17, 2023 12:16
@codecov
Copy link

codecov bot commented Mar 17, 2023

Codecov Report

Patch coverage: 88.37% and no project coverage change.

Comparison is base (d687380) 93.53% compared to head (11a3376) 93.53%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #2570   +/-   ##
========================================
  Coverage    93.53%   93.53%           
========================================
  Files          157      158    +1     
  Lines         7839     7861   +22     
========================================
+ Hits          7332     7353   +21     
- Misses         507      508    +1     
Flag Coverage Δ
pytest 93.53% <88.37%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/argilla/client/apis/datasets.py 90.40% <66.66%> (-2.23%) ⬇️
src/argilla/client/datasets.py 85.77% <66.66%> (-0.30%) ⬇️
src/argilla/server/commons/telemetry.py 88.67% <88.88%> (+0.54%) ⬆️
src/argilla/_constants.py 100.00% <100.00%> (ø)
src/argilla/_version.py 100.00% <100.00%> (ø)
src/argilla/client/client.py 93.53% <100.00%> (+0.02%) ⬆️
src/argilla/client/models.py 93.93% <100.00%> (+0.15%) ⬆️
...0/handlers/text_classification_dataset_settings.py 100.00% <100.00%> (ø)
.../handlers/token_classification_dataset_settings.py 100.00% <100.00%> (ø)
src/argilla/server/apis/v0/helpers.py 100.00% <100.00%> (ø)
... and 6 more

... and 3 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

CHANGELOG.md Outdated Show resolved Hide resolved
docs/_source/guides/query_datasets.md Show resolved Hide resolved
src/argilla/server/daos/backend/generic_elastic.py Outdated Show resolved Hide resolved
frascuchon and others added 2 commits March 17, 2023 18:36
Co-authored-by: José Francisco Calvo <jose@argilla.io>
@frascuchon frascuchon requested a review from jfcalvo March 20, 2023 18:15
Copy link
Member

@jfcalvo jfcalvo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks fine.

Only one last thing. I have not found any of the tests testing if the non-searchable fields are actually non-searchable. I mean adding a _this: myspecialanduniqueword field and try to search for it with myspecialanduniqueword and no documents are returned. If you think this could be useful please add it.

@jfcalvo
Copy link
Member

jfcalvo commented Mar 21, 2023

Feel free to merge.

@frascuchon frascuchon merged commit a96a65e into develop Mar 21, 2023
@frascuchon frascuchon deleted the feat/accept-non-searchable-fields-in-metadata branch March 21, 2023 10:56
@frascuchon frascuchon mentioned this pull request Mar 21, 2023
frascuchon added a commit that referenced this pull request Mar 22, 2023
## [1.5.0](v1.4.0...v1.5.0) -
2023-03-21

### Added

- Add the fields to retrieve when loading the data from argilla.
`rg.load` takes too long because of the vector field, even when users
don't need it. Closes
[#2398](#2398)
- Add new page and components for dataset settings. Closes
[#2442](#2003)
- Add ability to show image in records (for TokenClassification and
TextClassification) if an URL is passed in metadata with the key
\_image_url
- Non-searchable fields support in metadata.
[#2570](#2570)

### Changed

- Labels are now centralized in a specific vuex ORM called GlobalLabel
Model, see #2210. This model
is the same for TokenClassification and TextClassification (so both task
have labels with color_id and shortcuts parameters in the vuex ORM)
- The shortcuts improvement for labels
[#2339](#2339) have been moved
to the vuex ORM in dataset settings feature
[#2444](eb37c3b)
- Update "Define a labeling schema" section in docs.
- The record inputs are sorted alphabetically in UI by default.
[#2581](#2581)

### Fixes

- Allow URL to be clickable in Jupyter notebook again. Closes
[#2527](#2527)

### Removed

- Removing some data scan deprecated endpoints used by old clients. This
change will break compatibility with client `<v1.3.0`
- Stop using old scan deprecated endpoints in python client. This logic
will break client compatibility with server version `<1.3.0`
- Remove the previous way to add labels through the dataset page. Now
labels can be added only through dataset settings page.



### As always, thanks to our amazing contributors!
- Documentation update: tutorial for text classification models
comparison (#2426) by @embonhomme
- Docs: fix little typo (#2522) by @anakin87
- Docs: Tutorial on image classification (#2420) by @burtenshaw
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants