-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pref: receive clip embedding as a parameter #1
Conversation
instead of generating it itself. Also some optimizations for generating embeddings.
dalle_flow_glid3/sample.py
Outdated
@@ -95,29 +98,41 @@ def set_requires_grad(model, value): | |||
bert.half().eval() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
half-precision mode should only works on CUDA device
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if device == 'cuda:0':
bert.half().eval()
else:
bert.eval()
is it good?
I am not sure because original code has this https://github.com/Jack000/glid-3-xl/blob/master/sample.py#L294
dalle_flow_glid3/sample.py
Outdated
res = requests.post('https://demo-cas.jina.ai:8443/post', | ||
data=json.dumps({'execEndpoint':'/', 'data': [{'text': runtime_args.negative}]}), | ||
headers={'content-type': 'application/json'} | ||
).json() | ||
|
||
blank_clip_embedding = np.array(res['data'][0]['embedding']).astype( | ||
model_config['use_fp16'] and np.float16 or np.float32 | ||
).reshape(1, -1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not use grpc via cas client? What's more, the endpoints of cas-server would have some broken changes (to support multi-models by a single flow). So I recommend to use the cas-client instead.
instead of generating it itself.
Also some optimisations for generating embeddings.