Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: cache hub pull with build env #5061

Merged
merged 6 commits into from Aug 15, 2022
Merged

fix: cache hub pull with build env #5061

merged 6 commits into from Aug 15, 2022

Conversation

JohannesMessner
Copy link
Contributor

there was a bug where pulling an Executor with build env vars would not work, because it tried to cache the keys of these env vars, but DictKeys object can't be pickled. This PR converts these keys to a list, thus fixing the bug.

The diffs are large, but that's only because of black.
The only relevant line is build_env=list(buildEnv.keys()) if buildEnv else [],, line 715 in hubio.py (and the added test).

@github-actions github-actions bot added size/M area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing labels Aug 15, 2022
@codecov
Copy link

codecov bot commented Aug 15, 2022

Codecov Report

Merging #5061 (6c57c7c) into master (9615e5b) will decrease coverage by 0.00%.
The diff coverage is 90.47%.

@@            Coverage Diff             @@
##           master    #5061      +/-   ##
==========================================
- Coverage   88.07%   88.07%   -0.01%     
==========================================
  Files         115      115              
  Lines        9149     9165      +16     
==========================================
+ Hits         8058     8072      +14     
- Misses       1091     1093       +2     
Flag Coverage Δ
jina 88.07% <90.47%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/__init__.py 65.11% <80.00%> (+0.48%) ⬆️
jina/hubble/helper.py 90.99% <80.00%> (ø)
jina/hubble/hubapi.py 95.53% <91.66%> (-0.55%) ⬇️
jina/hubble/hubio.py 87.73% <94.11%> (ø)
jina/orchestrate/helper.py 100.00% <100.00%> (ø)
jina/serve/executors/decorators.py 97.93% <100.00%> (+0.02%) ⬆️
jina/clients/mixin.py 91.66% <0.00%> (-1.20%) ⬇️
...a/orchestrate/deployments/config/docker_compose.py 99.00% <0.00%> (-1.00%) ⬇️
jina/orchestrate/flow/base.py 89.88% <0.00%> (-0.57%) ⬇️
jina/serve/runtimes/gateway/http/__init__.py 100.00% <0.00%> (+2.12%) ⬆️
... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@floralatin
Copy link
Contributor

There is already a pr to fix it #5054

@JohannesMessner
Copy link
Contributor Author

There is already a pr to fix it #5054

I see now. Would it be possible to always add a description to your PRs? Otherwise we won't know what your PR does, and in a past retro we also agreed not to review PRs without description. Thank you!

@floralatin
Copy link
Contributor

floralatin commented Aug 15, 2022

There is already a pr to fix it #5054

I see now. Would it be possible to always add a description to your PRs? Otherwise we won't know what your PR does, and in a past retro we also agreed not to review PRs without description. Thank you!

Yes. of course, and i will write a description about it. thank you. very much.

Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes

jina/hubble/hubio.py Outdated Show resolved Hide resolved
jina/hubble/hubio.py Outdated Show resolved Hide resolved
JohannesMessner and others added 2 commits August 15, 2022 15:48
Co-authored-by: Nan Wang <nan.wang@jina.ai>
Co-authored-by: Nan Wang <nan.wang@jina.ai>
@floralatin
Copy link
Contributor

I think the following code should be used

build_env=list(buildEnv.keys()) if buildEnv and isinstance(buildEnv, dict) else []

nan-wang
nan-wang previously approved these changes Aug 15, 2022
Copy link
Member

@nan-wang nan-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM👍

Copy link
Contributor

@floralatin floralatin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should usebuild_env = build_env_dict if len(list(build_env_dict.keys())) > 0 else None
not build_env = build_env_dict if build_env_dict else None. if use build_env = build_env_dict if build_env_dict else None it is always true

@hanxiao hanxiao merged commit 2256307 into master Aug 15, 2022
@hanxiao hanxiao deleted the fix-cache-hub-pull branch August 15, 2022 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core This issue/PR affects the core codebase area/testing This issue/PR affects testing size/M size/S
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants