Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

中文提示词翻译模型使用CPU推理的方法 #33

Open
tyxsspa opened this issue Jan 4, 2024 · 0 comments
Open

中文提示词翻译模型使用CPU推理的方法 #33

tyxsspa opened this issue Jan 4, 2024 · 0 comments

Comments

@tyxsspa
Copy link
Owner

tyxsspa commented Jan 4, 2024

demo内置的翻译模块大约消耗4G显存,如果需要使用使用CPU推理,因官方尚未支持,需要手动修改modelscope中翻译模块的推理代码,通常位于如下类似位置(仅举例):
/home/username/anaconda3/envs/anytext/lib/python3.10/site-packages/modelscope/pipelines/nlp/translation_pipeline.py
然后在TranslationPipeline的__init__()函数中添加如下代码:

...
tf_config.gpu_options.allow_growth = True
tf_config.device_count['GPU'] = 0  # <-- add here!
self._session = tf.Session(config=tf_config)
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant