Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Async Batch Inference API #500

Closed
14 tasks done
vlad-tokarev opened this issue Feb 19, 2021 · 0 comments · Fixed by #528 or #532
Closed
14 tasks done

New Async Batch Inference API #500

vlad-tokarev opened this issue Feb 19, 2021 · 0 comments · Fixed by #528 or #532
Assignees
@vlad-tokarev
Labels
1.5
Projects
odahu-kanban

Comments

@vlad-tokarev
Copy link
Member

vlad-tokarev commented Feb 19, 2021 •
edited
Loading

  • API Server CRUD, Validation, Persistence
  • Spec/Status Syncer API Server <-> Kubernetes
  • Kubernetes Operator of BatchInferenceJob (Convert to TaskRun)
  • Pipeline implemented as TaskRun
    • Tool to convert ODAHU Connection to Rclone config
    • Tool to send local jsons to model log storage
    • Step 1: Generate Rclone Config
    • Step 2: Sync request data and model
    • Step 3: Validate and filter request data for next step (only .json files) in kubeflow format
    • Step 4: Dump request data to model log (Enrich request id if required)
    • Step 5: Run user container with appropriate configuration
    • Step 6: Upload results to model log
  • Python SDK
  • Airflow Operator

Out of scope:

  • UI
  • CLI
  • Tensor types and shape validation (e.g. using config.pbtxt)

image
@vlad-tokarev vlad-tokarev self-assigned this Feb 19, 2021
@vlad-tokarev vlad-tokarev added this to Backlog in odahu-kanban via automation Feb 19, 2021
@vlad-tokarev vlad-tokarev moved this from Backlog to In development in odahu-kanban Feb 19, 2021
vlad-tokarev added a commit that referenced this issue Feb 20, 2021
@vlad-tokarev
[#500] batch structs
2b1f11a
vlad-tokarev added a commit that referenced this issue Feb 20, 2021
@vlad-tokarev
[#500] * New tools package that provide GenerateRClone API to confi…
f1d08c6 
…gure RClone storages using ODAHU connections

* New odahu-tools binary that provide CLI for `GenerateRClone` API and other platform features in the future
vlad-tokarev added a commit that referenced this issue Feb 20, 2021
@vlad-tokarev
[#500] add makefile targets and add odahu-tools Dockerfile
ce0e62a
vlad-tokarev added a commit that referenced this issue Feb 20, 2021
@vlad-tokarev
[#500] types kubeflow predict v2 api are generated
3ffc2e4
vlad-tokarev added a commit that referenced this issue Feb 20, 2021
@vlad-tokarev
[#500] temp delete batch data structures
9fdd913
vlad-tokarev added a commit that referenced this issue Feb 20, 2021
@vlad-tokarev
[#500] ValidateDir function that validate and copy requests
92e4c75
@vlad-tokarev vlad-tokarev mentioned this issue Feb 24, 2021
Merged
vlad-tokarev added a commit that referenced this issue Feb 24, 2021
@vlad-tokarev
[#500] batch structs
3a76f4d
vlad-tokarev added a commit that referenced this issue Feb 24, 2021
@vlad-tokarev
[#500] * New tools package that provide GenerateRClone API to confi…
c356529 
…gure RClone storages using ODAHU connections

* New odahu-tools binary that provide CLI for `GenerateRClone` API and other platform features in the future
vlad-tokarev added a commit that referenced this issue Feb 24, 2021
@vlad-tokarev
[#500] add makefile targets and add odahu-tools Dockerfile
7586a54
vlad-tokarev added a commit that referenced this issue Feb 24, 2021
@vlad-tokarev
[#500] types kubeflow predict v2 api are generated
1f904fb
vlad-tokarev added a commit that referenced this issue Feb 24, 2021
@vlad-tokarev
[#500] temp delete batch data structures
bd2d4a8
vlad-tokarev added a commit that referenced this issue Feb 24, 2021
@vlad-tokarev
[#500] ValidateDir function that validate and copy requests
91d043b
This was referenced Feb 27, 2021
Merged
Merged
Closed
This was referenced Mar 11, 2021
Merged
Merged
Merged
This was referenced Mar 16, 2021
Merged
Merged
This was linked to pull requests Mar 16, 2021
Feat/syncer #528
Merged
Feat/batch test #532
Merged
Feat/batch inference api server #516
Closed
@vlad-tokarev vlad-tokarev moved this from In development to In QA in odahu-kanban Mar 17, 2021
@dsuslov dsuslov mentioned this issue Apr 5, 2021
Closed
@dsuslov dsuslov changed the title BatchInferenceJob New Async Batch Inference API Apr 5, 2021
@BPylypenko BPylypenko moved this from In QA to Done in odahu-kanban Apr 29, 2021
@viravit viravit mentioned this issue Apr 30, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vlad-tokarev vlad-tokarev

Labels
1.5
Projects
odahu-kanban
  
Done
Milestone
No milestone
1 participant
@vlad-tokarev