Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add support for text generation tasks #252

Merged
merged 6 commits into from
Feb 17, 2023
Merged

feat: add support for text generation tasks #252

merged 6 commits into from
Feb 17, 2023

Conversation

heiruwu
Copy link
Member

@heiruwu heiruwu commented Feb 15, 2023

Because

  • support text generation tasks in VDP

This commit

  • add support for text generation tasks

github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 15, 2023
View reviewed changes
internal/triton/triton.go
case modelPB.ModelInstance_TASK_TEXT_GENERATION:
textGenerationInputs := inferInput.([]TextGenerationInput)
outputLen := make([]byte, 4)
binary.LittleEndian.PutUint32(outputLen, uint32(textGenerationInputs[0].OutputLen))

Check failure

Code scanning / CodeQL

Incorrect conversion between integer types

Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type uint32 without an upper bound check.
internal/triton/triton.go
outputLen := make([]byte, 4)
binary.LittleEndian.PutUint32(outputLen, uint32(textGenerationInputs[0].OutputLen))
topK := make([]byte, 4)
binary.LittleEndian.PutUint32(topK, uint32(textGenerationInputs[0].TopK))

Check failure

Code scanning / CodeQL

Incorrect conversion between integer types

Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type uint32 without an upper bound check.
Phelan164
Phelan164 reviewed Feb 15, 2023
View reviewed changes
config/config.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Feb 15, 2023 •
edited

Codecov Report

Base: 2.18% // Head: 2.11% // Decreases project coverage by -0.08% ⚠️

Coverage data is based on head (85d09b9) compared to base (ffb9525).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files 
@@           Coverage Diff            @@
##            main    #252      +/-   ##
========================================
- Coverage   2.18%   2.11%   -0.08%     
========================================
  Files          7       7              
  Lines       3973    4110     +137     
========================================
  Hits          87      87              
- Misses      3868    4005     +137     
  Partials      18      18
Flag Coverage Δ
unittests 2.11% <0.00%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
internal/util/util.go 2.29% <0.00%> (-0.05%) ⬇️
pkg/handler/handler.go 0.00% <0.00%> (ø)
pkg/handler/payload.go 0.00% <0.00%> (ø)
pkg/service/service.go 14.36% <0.00%> (-0.65%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Phelan164
Phelan164 reviewed Feb 15, 2023
View reviewed changes
pkg/handler/payload.go Outdated Show resolved Hide resolved
Phelan164
Phelan164 reviewed Feb 15, 2023
View reviewed changes
pkg/handler/payload.go Outdated Show resolved Hide resolved
Phelan164
Phelan164 reviewed Feb 15, 2023
View reviewed changes
pkg/handler/payload.go Outdated Show resolved Hide resolved
Phelan164
Phelan164 reviewed Feb 15, 2023
View reviewed changes
pkg/handler/payload.go Outdated Show resolved Hide resolved
@heiruwu heiruwu marked this pull request as ready for review February 16, 2023 18:58
@Phelan164 Phelan164 self-requested a review February 17, 2023 08:16
Phelan164
Phelan164 approved these changes Feb 17, 2023
View reviewed changes
Copy link
Contributor

@Phelan164 Phelan164 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Phelan164 Phelan164 merged commit 767ec45 into main Feb 17, 2023
@Phelan164 Phelan164 deleted the add-text-generation-task branch February 17, 2023 09:11
@droplet-bot droplet-bot mentioned this pull request Feb 17, 2023
Merged
Phelan164 pushed a commit that referenced this pull request Feb 23, 2023
@droplet-bot
chore(main): release 0.13.0-alpha (#256)
93d46c2 
🤖 I have created a release *beep* *boop*
---


##
[0.13.0-alpha](v0.12.1-alpha...v0.13.0-alpha)
(2023-02-22)


### Features

* add support for text generation tasks
([#252](#252))
([767ec45](767ec45))


### Bug Fixes

* keep format for empty inference output
([#258](#258))
([e2a2e48](e2a2e48))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
This was referenced Feb 23, 2023
Closed
Closed
Merged
Phelan164 pushed a commit that referenced this pull request Feb 23, 2023
@droplet-bot
chore(main): release 0.13.0-alpha (#284)
06f43b5 
🤖 I have created a release *beep* *boop*
---


##
[0.13.0-alpha](v0.12.1-alpha...v0.13.0-alpha)
(2023-02-23)


### Features

* add support for text generation tasks
([#252](#252))
([767ec45](767ec45))


### Bug Fixes

* keep format for empty inference output
([#258](#258))
([e2a2e48](e2a2e48))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
@xiaofei-du xiaofei-du mentioned this pull request Feb 23, 2023
Merged
This was referenced Apr 15, 2023
Closed
Merged
Phelan164 pushed a commit that referenced this pull request Apr 24, 2023
@droplet-bot
chore(main): release 0.11.0-alpha (#340)
8cd420b 
🤖 I have created a release *beep* *boop*
---


##
[0.11.0-alpha](v0.16.0-alpha...v0.11.0-alpha)
(2023-04-24)


### Features

* add codebase for model grpc service
([4defa3e](4defa3e))
* add confidence score for ocr output
([#167](#167))
([e915452](e915452))
* add credential definition
([#109](#109))
([92d3391](92d3391))
* add gRPC Gateway and GetModel API
([#7](#7))
([bff6fc9](bff6fc9))
* add model initialization module
([#332](#332))
([aa753a5](aa753a5))
* add private endpoint and gRPC test cases
([#306](#306))
([bb3c193](bb3c193))
* add release stage for model definition
([#153](#153))
([4e13ba5](4e13ba5))
* add support for text generation tasks
([#252](#252))
([767ec45](767ec45))
* add text to image task
([#239](#239))
([421eb1a](421eb1a))
* **controller:** add model state monitoring with controller
([#323](#323))
([4397826](4397826))
* create model from GitHub
([#61](#61))
([cf763cb](cf763cb))
* handle oom
([#163](#163))
([4db1c45](4db1c45))
* remove model instance
([#320](#320))
([15e1b62](15e1b62))
* support artivc
([#102](#102))
([b8e21a4](b8e21a4))
* support async deploy and undeploy model instance
([#192](#192))
([ed36dc7](ed36dc7))
* support creating a HuggingFace model
([#113](#113))
([1577d87](1577d87))
* support instance segmentation task
([#183](#183))
([d28cfdc](d28cfdc))
* support model caching
([#317](#317))
([d15ffba](d15ffba))
* support model name when creating model
([#25](#25))
([7d799b7](7d799b7))
* support ocr task
([#150](#150))
([7766c6f](7766c6f))
* support semantic segmentation
([#203](#203))
([f22262c](f22262c))
* support url/base64 content prediction
([#34](#34))
([a88ddfd](a88ddfd))


### Bug Fixes

* add link for guideline create Conda environment file
([7ee8e06](7ee8e06))
* add writeonly to description
([f59d98f](f59d98f))
* allow updating emtpy description for a model
([#177](#177))
([100ec84](100ec84))
* bug usage storage
([#103](#103))
([975fdc1](975fdc1))
* clone repository and make folder
([ac79386](ac79386))
* **config:** use private port for mgmt-backend
([#307](#307))
([3264e2b](3264e2b))
* correct version when making inference
([#31](#31))
([c918e77](c918e77))
* create a subfolder in model-repository if needed
([#290](#290))
([7f8d78b](7f8d78b))
* fix build and go version
([#9](#9))
([f8d4346](f8d4346))
* fix client stream server recv wrong file length interval
([#143](#143))
([0e06f7c](0e06f7c))
* fix config path
([a8cf2c0](a8cf2c0))
* fix creating subfolder
([105a11a](105a11a))
* fix duration configuration bug
([ee4a310](ee4a310))
* fix keypoint model payload parser
([#249](#249))
([461d54a](461d54a))
* fix list long-run operation error
([#220](#220))
([472696d](472696d))
* fix subfolder creation
([#292](#292))
([0b6ec3f](0b6ec3f))
* fix unload model issue causing Triton server OOM
([#42](#42))
([fb4d1d1](fb4d1d1))
* fix usage client nil issue when mgmt-backend not ready
([#241](#241))
([4290159](4290159))
* fix variable name
([#293](#293))
([a7995dd](a7995dd))
* HuggingFace batching bug in preprocess model
([b1582e8](b1582e8))
* init config before logger
([9d3fb4a](9d3fb4a))
* keep format for empty inference output
([#258](#258))
([e2a2e48](e2a2e48))
* list models and model instances pagination
([#304](#304))
([1f19ed4](1f19ed4))
* logic when essemble or not
([ab8e7c1](ab8e7c1))
* model configuration response in integration test
([0225c1e](0225c1e))
* model definition in list model and missing zero in output
([#121](#121))
([a90072d](a90072d))
* model instance state update to unspecified state
([#206](#206))
([14c87d5](14c87d5))
* panic error with nil object
([#208](#208))
([a342113](a342113))
* pass the context between package layers
([#345](#345))
([e6e7f2f](e6e7f2f))
* post process for unspecified task output
([ad88068](ad88068))
* post process ocr task
([e387154](e387154))
* postgres host
([a322165](a322165))
* refactor JSON schema
([f24db48](f24db48))
* refactor model definition and model JSON schema
([#73](#73))
([0cce154](0cce154))
* regexp zap logger with new protobuf package
([8b9c463](8b9c463))
* return list of models in list method
([b88ebd7](b88ebd7))
* status code when deploy model error
([#111](#111))
([31d3f11](31d3f11))
* trigger image with 4 channel
([#141](#141))
([7445f5f](7445f5f))
* update db schema, protobuf generated files and create model, version
in upload api
([7573e54](7573e54))
* update description for GitHub model from user input
([#173](#173))
([821dab3](821dab3))
* update docker compose file for building dev image
([#29](#29))
([83cba09](83cba09))
* update model definitions and tasks in usage collection
([#100](#100))
([c593087](c593087))
* update predict for essemble model
([016f11c](016f11c))
* update version order when get model version list
([#38](#38))
([83c054a](83c054a))
* wrong logic when checking user account and service account
([7058db6](7058db6))


### Miscellaneous Chores

* release 0.11.0-alpha
([d592acb](d592acb))
* release 0.3.2-alpha
([9f8cd91](9f8cd91))
* release 0.4.2-alpha
([fc5a14a](fc5a14a))
* release 0.7.2-alpha
([17529d6](17529d6))
* release 0.7.3-alpha
([9033c50](9033c50))
* release v0.5.1-alpha
([895056d](895056d))
* release v0.6.1-alpha
([f18dc30](f18dc30))
* release v0.6.2-alpha
([4365f32](4365f32))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
This was referenced Apr 24, 2023
Closed
Merged
@droplet-bot droplet-bot mentioned this pull request May 6, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Phelan164 Phelan164 Phelan164 approved these changes

@pinglin pinglin Awaiting requested review from pinglin pinglin is a code owner

@xiaofei-du xiaofei-du Awaiting requested review from xiaofei-du xiaofei-du is a code owner

Assignees
No one assigned
Labels
None yet
Projects
🔮 Instill Core
Archived in project
Milestone
No milestone
2 participants
@heiruwu @Phelan164