Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add named score type #1430

Merged
merged 8 commits into from Dec 21, 2020
Merged

feat: add named score type #1430

merged 8 commits into from Dec 21, 2020

Conversation

JoanFM
Copy link
Member

@JoanFM JoanFM commented Dec 10, 2020

Changes introduced
Add NamedScore type and set a property for document.

Useful to provide expected score values for groundtruth specially for NDCGEvaluation

Fixes #1431

@JoanFM JoanFM requested a review from a team as a code owner December 10, 2020 15:17
@jina-bot jina-bot added size/M area/core This issue/PR affects the core codebase area/helper This issue/PR affects the helper functionality area/testing This issue/PR affects testing component/driver component/type labels Dec 10, 2020
@bwanglzu
Copy link
Member

bwanglzu commented Dec 10, 2020

why not from collections import namedtuple and created a new class?

from collections import namedtuple

NamedScore = namedtuple('NamedScore', ['op_name', 'score'])

and you get all these getters setters for free (as far as I know you didn't use description or ref_id)

@JoanFM
Copy link
Member Author

JoanFM commented Dec 10, 2020

why not from collections import namedtuple and created a new class?

from collections import namedtuple

NamedScore = namedtuple('NamedScore', ['op_name', 'score'])

and you get all these getters setters for free (as far as I know you didn't use description or ref_id

I like the idea; however my idea is to have a baseline to eventually support everything the Proto class accepts

jina/types/score/__init__.py Outdated Show resolved Hide resolved
jina/types/score/__init__.py Outdated Show resolved Hide resolved
jina/types/score/__init__.py Outdated Show resolved Hide resolved
tests/unit/types/score/test_score.py Show resolved Hide resolved
jina/types/score/__init__.py Outdated Show resolved Hide resolved
jina/types/score/__init__.py Outdated Show resolved Hide resolved
jina/types/score/__init__.py Outdated Show resolved Hide resolved
@JoanFM JoanFM force-pushed the document-scores branch 2 times, most recently from ccaf8d5 to 7ebc1c6 Compare December 11, 2020 07:36
@hanxiao hanxiao linked an issue Dec 13, 2020 that may be closed by this pull request
@JoanFM JoanFM marked this pull request as draft December 14, 2020 11:02
@github-actions
Copy link

github-actions bot commented Dec 16, 2020

Latency summary

Current PR yields:

  • 😶 index QPS at 1601, delta to last 3 avg.: +4%
  • 😶 query QPS at 25, delta to last 3 avg.: +0%

Breakdown

Version Index QPS Query QPS
current 1601 25
0.8.11 1540 26
0.8.10 1536 25
0.8.9 1539 23

Backed by latency-tracking. Further commits will update this comment.

@JoanFM JoanFM closed this Dec 21, 2020
@JoanFM JoanFM reopened this Dec 21, 2020
@JoanFM JoanFM marked this pull request as ready for review December 21, 2020 14:27
@codecov
Copy link

codecov bot commented Dec 21, 2020

Codecov Report

Merging #1430 (4f2f1ca) into master (1645a19) will increase coverage by 0.19%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1430      +/-   ##
==========================================
+ Coverage   84.07%   84.26%   +0.19%     
==========================================
  Files         106      106              
  Lines        6102     6158      +56     
==========================================
+ Hits         5130     5189      +59     
+ Misses        972      969       -3     
Impacted Files Coverage Δ
jina/helloworld/__init__.py 100.00% <ø> (ø)
jina/drivers/evaluate.py 98.21% <100.00%> (+0.10%) ⬆️
jina/drivers/rank.py 97.33% <100.00%> (-0.04%) ⬇️
jina/drivers/search.py 94.64% <100.00%> (ø)
jina/excepts.py 100.00% <100.00%> (ø)
jina/helloworld/helper.py 90.52% <100.00%> (-0.57%) ⬇️
jina/logging/profile.py 68.33% <0.00%> (-0.58%) ⬇️
jina/drivers/craft.py 100.00% <0.00%> (ø)
jina/types/ndarray/generic.py 100.00% <0.00%> (ø)
jina/executors/indexers/cache.py 100.00% <0.00%> (ø)
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update abac298...ff67b4c. Read the comment docs.

@JoanFM JoanFM dismissed stale reviews from hanxiao and deepankarm December 21, 2020 16:05

Please review

@github-actions
Copy link

Thanks for your contribution ❤️
💔 Unfortunately, this PR has one ore more bad commit messages, it can not be merged. To fix this problem, please refer to:

Note, other CI tests will not start until the commit messages get fixed.

This message will be deleted automatically when the commit messages get fixed.

@hanxiao hanxiao merged commit f39039d into master Dec 21, 2020
@hanxiao hanxiao deleted the document-scores branch December 21, 2020 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/helper This issue/PR affects the helper functionality area/testing This issue/PR affects testing component/driver component/type size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add NamedScore primitive type
6 participants