feat(grpc): IO descriptors #2884

aarnphm · 2022-08-09T11:27:12Z

adding gRPC support for the remaining of the IO descriptor
Pandas will be handled by @Proto007

Test will be with a follow up PR.

This PR also remove setup.cfg and use pyproject.toml. We will also lock version of setuptools==63.4.0 until 64+ is stable.

since bentoml/_version.py is always generated and included inside the distribution, we should use that for __version__ instead of using importlib.metadata. This will also avoid PackageNotFoundError if bentoml is not yet installed, but attributes for version were accessed before.

pep8speaks · 2022-08-09T11:27:18Z

Hello @aarnphm, Thanks for updating this PR.

There are currently no PEP 8 issues detected in this PR. Cheers! 🍻

Comment last updated at 2022-08-14 00:01:59 UTC

codecov · 2022-08-09T11:32:37Z

Codecov Report

Merging #2884 (57bcce6) into grpc (6795656) will increase coverage by 0.45%.
The diff coverage is 21.83%.

@@            Coverage Diff             @@
##             grpc    #2884      +/-   ##
==========================================
+ Coverage   59.77%   60.22%   +0.45%     
==========================================
  Files         123      105      -18     
  Lines        9839     9464     -375     
  Branches        0     1656    +1656     
==========================================
- Hits         5881     5700     -181     
+ Misses       3958     3439     -519     
- Partials        0      325     +325

Impacted Files	Coverage Δ
bentoml/_internal/configuration/__init__.py	`72.85% <0.00%> (-10.00%)`	⬇️
...oml/_internal/server/grpc/interceptors/__init__.py	`0.00% <0.00%> (ø)`
...ntoml/_internal/server/grpc/interceptors/access.py	`0.00% <0.00%> (ø)`
...internal/server/grpc/interceptors/opentelemetry.py	`0.00% <0.00%> (ø)`
...l/_internal/server/grpc/interceptors/prometheus.py	`0.00% <0.00%> (ø)`
bentoml/_internal/server/grpc/server.py	`0.00% <0.00%> (ø)`
bentoml/_internal/server/grpc/servicer.py	`0.00% <0.00%> (ø)`
bentoml/_internal/server/grpc_app.py	`0.00% <ø> (ø)`
bentoml/_internal/server/http/access.py	`85.50% <ø> (-11.60%)`	⬇️
bentoml/_internal/server/http/instruments.py	`100.00% <ø> (ø)`
... and 91 more

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

migrate setup.cfg to pyproject.toml Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

See #2771 #2884 add gRPC context and types add descriptor metaclass add metaclass field for proto value verify_ndarray takes shape and dtype update types imports to public API Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

parano · 2022-08-14T21:18:43Z

bentoml/_internal/io_descriptors/numpy.py

            # https://numpy.org/doc/stable/user/basics.byteswapping.html#introduction-to-byte-ordering-and-ndarrays
+            bytesorder = "C"  # default from numpy (C-order)
+
        self._bytesorder: t.Literal["C", "F", "A"] = bytesorder


remove _bytesorder?

_bytesorder can be used for np.frombuffer.

parano · 2022-08-14T21:21:39Z

bentoml/_internal/io_descriptors/numpy.py

+            if shapepb:
+                shape = tuple(shapepb)
+                if self._shape:
+                    logger.warning(


When force_shape is True, we should always use the shape specified when creating the Numpy io descriptor, not using the shape in request. Let's just raise an exception if the specified shape is different from the shape in the request if enforce shape is True. Make sure to handle the -1 case.

This is checked on L410 right?

parano · 2022-08-14T21:23:17Z

bentoml/_internal/io_descriptors/numpy.py

+                f"'enforce_dtype={self._enforce_dtype}', using '{self._dtype}'..."
+            )
+            dtype = self._dtype
+            field_values = npdtype_to_fieldpb_map()[dtype]


This is not the right behavior. let's just convert the numpy array from proto first, and then check with np.can_cast. reuse the validation function above.

parano · 2022-08-14T21:24:27Z

bentoml/_internal/io_descriptors/numpy.py

+                dtype=dtype,
+                count=num_entries,
+                offset=0,
+            ).reshape(shape)


add self.validate_array here

No need to right? we already cast the dtype and reshape it with shape

parano · 2022-08-14T21:25:17Z

bentoml/_internal/io_descriptors/text.py

+        # validate gRPC content type if content type is specified
+        validate_content_type(context, self)
+
+        field = ast.literal_eval(get_field(request, self))


why do we need this?

You can an put a bytes string.

We should also allow a bytes field in all of the message.

See #2771 #2884 add gRPC context and types add descriptor metaclass add metaclass field for proto value verify_ndarray takes shape and dtype update types imports to public API Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

See bentoml#2771 bentoml#2884 add gRPC context and types add descriptor metaclass add metaclass field for proto value verify_ndarray takes shape and dtype update types imports to public API Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

See bentoml#2771 bentoml#2884 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

See bentoml#2771 bentoml#2884 bentoml#2906 Co-authored-by: Sadab Hafiz <sadabhafizny@gmail.com> Co-authored-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com> Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

See bentoml#2771 bentoml#2884 bentoml#2906 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm requested review from ssheng, parano and a team as code owners August 9, 2022 11:27

aarnphm requested review from bojiang and removed request for a team August 9, 2022 11:27

aarnphm force-pushed the feat/grpc-io-descriptor branch from 50f21e6 to 997d5fd Compare August 9, 2022 11:27

aarnphm changed the title ~~feat: IO descriptors~~ feat(grpc): IO descriptors Aug 9, 2022

aarnphm force-pushed the feat/grpc-io-descriptor branch from a16c8d8 to cd3c0d3 Compare August 12, 2022 07:17

aarnphm added 11 commits August 12, 2022 00:18

chore: add file grpc func

ad64d79

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: fixing ports for windows and freebsd

58a218a

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: updates proto_fields to be inside metaclass

ddc8897

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: fix texts and file io

a5fed3c

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

fix: update numpy tests

689fd08

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

fix: numppy serialization

971b0d2

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

feat: file IO descriptor

e6dede0

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: updates io descriptor

b55c4cb

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: adding types for kind raw message

fa6bdf6

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

fix(test): shape test

064fab9

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: cleanup and updates with new proto

051429e

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm force-pushed the feat/grpc-io-descriptor branch from cd3c0d3 to 051429e Compare August 12, 2022 07:18

aarnphm added 4 commits August 12, 2022 23:09

feat: new IO descriptor interface

fbee948

migrate setup.cfg to pyproject.toml Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

chore: updates dependencies

9858d53

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

fix: numpy IO descriptor

89813fd

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

feat: finished IO descriptor

cc019cb

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm force-pushed the feat/grpc-io-descriptor branch 3 times, most recently from 90f806e to de74ee7 Compare August 13, 2022 14:44

fix: styles

bd124a6

Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

parano reviewed Aug 14, 2022

View reviewed changes

aarnphm added a commit to aarnphm/BentoML that referenced this pull request Sep 11, 2022

feat(io): descriptor implementation

db3984c

See bentoml#2771 bentoml#2884 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm added a commit to aarnphm/BentoML that referenced this pull request Sep 11, 2022

feat(io): descriptor implementation

857e004

See bentoml#2771 bentoml#2884 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm added a commit to aarnphm/BentoML that referenced this pull request Sep 11, 2022

feat(io): ----------descriptor implementation-------------

7381716

See bentoml#2771 bentoml#2884 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm added a commit to aarnphm/BentoML that referenced this pull request Sep 11, 2022

feat(io): descriptor implementation

79a0805

See bentoml#2771 bentoml#2884 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

aarnphm added a commit to aarnphm/BentoML that referenced this pull request Sep 11, 2022

feat(io): descriptor implementation

5a3dd1a

See bentoml#2771 bentoml#2884 bentoml#2906 Signed-off-by: Aaron Pham <29749331+aarnphm@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(grpc): IO descriptors #2884

feat(grpc): IO descriptors #2884

aarnphm commented Aug 9, 2022 •

edited

Loading

pep8speaks commented Aug 9, 2022 •

edited

Loading

codecov bot commented Aug 9, 2022 •

edited

Loading

parano Aug 14, 2022

aarnphm Aug 15, 2022

parano Aug 14, 2022

aarnphm Aug 15, 2022

parano Aug 14, 2022

parano Aug 14, 2022

aarnphm Aug 15, 2022

parano Aug 14, 2022

aarnphm Aug 15, 2022

aarnphm Aug 15, 2022

feat(grpc): IO descriptors #2884

feat(grpc): IO descriptors #2884

Conversation

aarnphm commented Aug 9, 2022 • edited Loading

pep8speaks commented Aug 9, 2022 • edited Loading

Comment last updated at 2022-08-14 00:01:59 UTC

codecov bot commented Aug 9, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aarnphm commented Aug 9, 2022 •

edited

Loading

pep8speaks commented Aug 9, 2022 •

edited

Loading

codecov bot commented Aug 9, 2022 •

edited

Loading