Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix eval name from driver #1623

Merged
merged 2 commits into from
Jan 9, 2021
Merged

fix: fix eval name from driver #1623

merged 2 commits into from
Jan 9, 2021

Conversation

JoanFM
Copy link
Member

@JoanFM JoanFM commented Jan 7, 2021

No description provided.

@JoanFM JoanFM requested a review from a team as a code owner January 7, 2021 11:33
@jina-bot jina-bot added size/S area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/driver labels Jan 7, 2021
@github-actions
Copy link

github-actions bot commented Jan 7, 2021

Latency summary

Current PR yields:

  • 😶 index QPS at 1562, delta to last 3 avg.: -4%
  • 😶 query QPS at 26, delta to last 3 avg.: -4%

Breakdown

Version Index QPS Query QPS
current 1562 26
0.9.2 1646 27
0.9.1 1649 27
0.8.22 1635 26

Backed by latency-tracking. Further commits will update this comment.

@codecov
Copy link

codecov bot commented Jan 7, 2021

Codecov Report

Merging #1623 (e4597ee) into master (0685b1d) will increase coverage by 0.94%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1623      +/-   ##
==========================================
+ Coverage   84.14%   85.08%   +0.94%     
==========================================
  Files         127      127              
  Lines        6642     6699      +57     
==========================================
+ Hits         5589     5700     +111     
+ Misses       1053      999      -54     
Impacted Files Coverage Δ
jina/drivers/evaluate.py 98.27% <100.00%> (ø)
jina/logging/profile.py 69.84% <0.00%> (-0.56%) ⬇️
jina/drivers/craft.py 100.00% <0.00%> (ø)
jina/types/ndarray/generic.py 100.00% <0.00%> (ø)
jina/drivers/encode.py 94.91% <0.00%> (+0.08%) ⬆️
jina/enums.py 96.59% <0.00%> (+0.09%) ⬆️
jina/jaml/__init__.py 95.93% <0.00%> (+0.09%) ⬆️
jina/flow/base.py 87.08% <0.00%> (+0.09%) ⬆️
jina/clients/helper.py 94.00% <0.00%> (+0.12%) ⬆️
jina/executors/indexers/vector.py 93.15% <0.00%> (+0.14%) ⬆️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0685b1d...619d6eb. Read the comment docs.

hanxiao
hanxiao previously approved these changes Jan 7, 2021
jina/drivers/evaluate.py Outdated Show resolved Hide resolved
Comment on lines 59 to 61
evaluation.op_name = f'{self.exec.__class__.__name__}@{self.exec.eval_at}'
else:
evaluation.op_name = self.exec.__class__.__name__
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
evaluation.op_name = f'{self.exec.__class__.__name__}@{self.exec.eval_at}'
else:
evaluation.op_name = self.exec.__class__.__name__
evaluation.op_name = f'{self.exec.name}@{self.exec.eval_at}'
else:
evaluation.op_name = self.exec.name

I think we can simply use .name attribute of the executor - one of the metas attribute of any Executor. Also, since eval_at is kind of BaseRankingEvaluator specific, the BaseRankingEvaluator could just have the following somewhere in __init__ of post_init:

if self.eval_at:
   self.name += f`@{self.eval_at}`

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but now changing again in Hub is too much?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hub follows core.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yes makes sense

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this will lead to many changes in Jina Hub?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, that is my bad.

The name is very noisy for what I have seen, though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


        the name of the executor.

        :type: str
        :default: class name plus a random string

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the naming is better with the current changes

@JoanFM JoanFM requested a review from nan-wang January 8, 2021 07:18
Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍

@nan-wang nan-wang merged commit a7f1187 into master Jan 9, 2021
@nan-wang nan-wang deleted the fix-eval-name branch January 9, 2021 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing component/driver size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants