feat: add gpu support to container runtime #2914

bwanglzu · 2021-07-11T12:42:42Z

this pr close #2900

should allow us deploy a dockerized jina pod with local gpu devices, e.g.

jina executor --uses jinahub+docker://CLIPImageEncoder --gpus all 
jina executor --uses jinahub+docker://CLIPImageEncoder --gpus 2  # use 2 gpu devices
jina executor --uses jinahub+docker://CLIPImageEncoder --gpus device=GPU-1234-567  # use gpu by device id
jina executor --uses jinahub+docker://CLIPImageEncoder --gpus device=GPU-1234-567,device=GPU-1234-789  # use multiple gpus by device id
jina executor --uses jinahub+docker://CLIPImageEncoder --gpus device=GPU-1234-567,driver=nvidia ## and other parameters

for more info check out this link.

…jina into feat-docker-gpu-support

codecov · 2021-07-11T12:49:24Z

Codecov Report

Merging #2914 (13ca9de) into master (0094bbf) will increase coverage by 6.60%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2914      +/-   ##
==========================================
+ Coverage   82.21%   88.82%   +6.60%     
==========================================
  Files         106      138      +32     
  Lines        7064     9567    +2503     
==========================================
+ Hits         5808     8498    +2690     
+ Misses       1256     1069     -187

Flag	Coverage Δ
daemon	`43.24% <ø> (?)`
jina	`88.77% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
jina/__init__.py	`71.64% <ø> (ø)`
jina/checker.py	`96.55% <ø> (ø)`
jina/clients/__init__.py	`100.00% <ø> (ø)`
jina/clients/base/__init__.py	`90.90% <ø> (ø)`
jina/clients/base/grpc.py	`63.82% <ø> (ø)`
jina/clients/base/http.py	`93.18% <ø> (ø)`
jina/clients/base/websocket.py	`86.36% <ø> (ø)`
jina/clients/grpc.py	`100.00% <ø> (ø)`
jina/clients/helper.py	`100.00% <ø> (ø)`
jina/clients/http.py	`100.00% <ø> (ø)`
... and 215 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f59c6c...13ca9de. Read the comment docs.

JoanFM · 2021-07-11T12:51:54Z

jina/peapods/runtimes/container/__init__.py

+                'device': [],
+                'driver': '',
+            }
+            gpu_args = self.args.gpus[0]


if it is a list why to consider only first?

JoanFM · 2021-07-11T12:53:36Z

jina/peapods/runtimes/container/__init__.py

+                    capabilities=[_gpus['capabilities']],
+                )
+            ]
+


remove gpus from args before doing the container run for better backwards compatibility with jinahub on older jina versions

…jina into feat-docker-gpu-support

JoanFM

then add a test to make sure that the device request param is properly sent in the container runtime call

JoanFM · 2021-07-11T16:47:04Z

jina/peapods/runtimes/container/__init__.py

@@ -163,6 +164,36 @@ def _docker_run(self, replay: bool = False):
                    'mode': 'rw',
                }

+        device_requests = []
+        if self.args.gpus:


maybe u can put this in an static method where u can do better unittesting of this logic

…jina into feat-docker-gpu-support

JoanFM · 2021-07-12T08:01:50Z

jina/peapods/runtimes/container/__init__.py

@@ -74,6 +74,38 @@ def _set_network_for_dind_linux(self):
                )
        client.close()

+    @staticmethod
+    def _set_device_requests_for_gpu(gpu_args):


it should be a _get?

JoanFM · 2021-07-12T09:09:55Z

@bwanglzu have u tried in the GPU server to see if it works?

feat: add gpu support to container runtime

dddfe79

jina-bot added size/S area/core This issue/PR affects the core codebase area/network This issue/PR affects network functionality area/testing This issue/PR affects testing component/peapod labels Jul 11, 2021

jina-bot and others added 2 commits July 11, 2021 12:43

style: fix overload and cli autocomplete

757f06d

feat: add gpu support to container runtime

c2bb056

jina-bot added area/cli This issue/PR affects the command line interface component/flow labels Jul 11, 2021

bwanglzu added 2 commits July 11, 2021 14:44

Merge branch 'feat-docker-gpu-support' of https://github.com/jina-ai/…

2752672

…jina into feat-docker-gpu-support

feat: add gpu support to container runtime

0e341bd

JoanFM requested changes Jul 11, 2021

View reviewed changes

bwanglzu and others added 4 commits July 11, 2021 15:38

feat: parse gpu argument as str

5b86930

style: fix overload and cli autocomplete

360c506

feat: parse gpu argument as str

4ce0d80

Merge branch 'feat-docker-gpu-support' of https://github.com/jina-ai/…

e668a66

…jina into feat-docker-gpu-support

bwanglzu self-assigned this Jul 11, 2021

feat: parse gpu argument as str

2ed70a0

JoanFM reviewed Jul 11, 2021

View reviewed changes

JoanFM requested changes Jul 11, 2021

View reviewed changes

feat: test device requests type

25986cf

jina-bot added size/M and removed size/S labels Jul 12, 2021

jina-bot and others added 3 commits July 12, 2021 08:00

style: fix overload and cli autocomplete

3d86c70

feat: test device requests type

a2dcdea

Merge branch 'feat-docker-gpu-support' of https://github.com/jina-ai/…

7c3e189

…jina into feat-docker-gpu-support

JoanFM requested changes Jul 12, 2021

View reviewed changes

feat: test device requests type

13ca9de

bwanglzu marked this pull request as ready for review July 12, 2021 09:01

bwanglzu requested a review from a team as a code owner July 12, 2021 09:01

bwanglzu requested review from mapleeit and Gikiman July 12, 2021 09:01

JoanFM approved these changes Jul 12, 2021

View reviewed changes

JoanFM merged commit f9c50b7 into master Jul 12, 2021

JoanFM deleted the feat-docker-gpu-support branch July 12, 2021 12:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add gpu support to container runtime #2914

feat: add gpu support to container runtime #2914

bwanglzu commented Jul 11, 2021 •

edited

codecov bot commented Jul 11, 2021 •

edited

JoanFM Jul 11, 2021

JoanFM Jul 11, 2021

JoanFM left a comment

JoanFM Jul 11, 2021

JoanFM Jul 12, 2021

JoanFM commented Jul 12, 2021

feat: add gpu support to container runtime #2914

feat: add gpu support to container runtime #2914

Conversation

bwanglzu commented Jul 11, 2021 • edited

codecov bot commented Jul 11, 2021 • edited

Codecov Report

JoanFM Jul 11, 2021

Choose a reason for hiding this comment

JoanFM Jul 11, 2021

Choose a reason for hiding this comment

JoanFM left a comment

Choose a reason for hiding this comment

JoanFM Jul 11, 2021

Choose a reason for hiding this comment

JoanFM Jul 12, 2021

Choose a reason for hiding this comment

JoanFM commented Jul 12, 2021

bwanglzu commented Jul 11, 2021 •

edited

codecov bot commented Jul 11, 2021 •

edited