Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(#800): limit number of metadata fields #993

Merged
merged 7 commits into from
Jan 18, 2022

Conversation

frascuchon
Copy link
Member

@frascuchon frascuchon commented Jan 18, 2022

This PR adds validations to check limits for stored metadata fields in a dataset.

The default metadata limits is configurable by environment var and for the future, could be a per-index configuration

This PR together with #990 will close #800

@codecov
Copy link

codecov bot commented Jan 18, 2022

Codecov Report

Merging #993 (2676539) into master (544a915) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #993      +/-   ##
==========================================
+ Coverage   95.48%   95.50%   +0.01%     
==========================================
  Files         109      110       +1     
  Lines        4231     4247      +16     
==========================================
+ Hits         4040     4056      +16     
  Misses        191      191              
Flag Coverage Δ
pytest 95.50% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/rubrix/server/commons/settings.py 100.00% <100.00%> (ø)
src/rubrix/server/tasks/commons/api/errors.py 100.00% <100.00%> (ø)
src/rubrix/server/tasks/commons/api/model.py 95.31% <100.00%> (+0.07%) ⬆️
src/rubrix/server/tasks/commons/dao/dao.py 97.39% <100.00%> (+0.11%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 544a915...2676539. Read the comment docs.

@frascuchon frascuchon changed the title fix: limit number of metadata fields fix(#800): limit number of metadata fields Jan 18, 2022
@frascuchon frascuchon merged commit 6c743ab into master Jan 18, 2022
@frascuchon frascuchon deleted the fix/limit-metadata-fields branch January 18, 2022 22:56
frascuchon added a commit that referenced this pull request Jan 19, 2022
* feat(env): configurable metadata fields limit

* feat(log): limit number of (flatten) metadata fields

* test: add tests

* revert: changes

* refactor: compute metadata length including also the stored values

* fix: add missing files

* test: include some tests

(cherry picked from commit 6c743ab)
@frascuchon frascuchon added 0.8.1 and removed 0.8.1 labels Jan 20, 2022
dvsrepo added a commit that referenced this pull request Jan 25, 2022
* 'master' of https://github.com/recognai/rubrix: (28 commits)
  chore(#994): simplify server start (#1014)
  refactor(#945): elasticsearch index configuration per task (#1018)
  docs(#980): Add majorityvote example to weak supervision guide (#1004)
  fix(#1015): manage emojis in Token Classification records (#1016)
  fix(#1010): fix WeakLabels when not providing rules (#1011)
  fix(Breadcrumbs): Copy text function (#1003)
  fix(statics): handle 404 errors for static files (#1006)
  refactor: Rename 'exclude_missing_annotation' argument for WeakLabels.annotation (#1005)
  docs: update video for spacy ner tutorial (#1000)
  fix(#977) Remove redirection when accessing login (#996)
  feat(#951): new uncovered_by_rules records filter (#991)
  feat(#735): add warning when agent but no prediction/annotation is provided (#987)
  fix(#800): limit number of metadata fields (#993)
  docs: improve annotate records section in web app reference (#985)
  fix(#844) Keep header layout when errors occurred in dataset view (#992)
  fix(#974): display the dropdown in the last record of the scroll (#986)
  refactor: compute common aggregations one by one (#990)
  feat(#953): add additional metrics to `LabelModel.score` method (#979)
  refactor(rules): using labeling rules management with vuex store (#878)
  feat(#955): add default for `rules` in WeakLabels (#976)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Search] search.max_buckets configuration could affect to search results
2 participants