Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support API parallelism #590

Closed
ospillinger opened this issue Nov 20, 2019 · 1 comment · Fixed by #838
Closed

Support API parallelism #590

ospillinger opened this issue Nov 20, 2019 · 1 comment · Fixed by #838
Assignees
@vishalbollu
Labels
enhancement New feature or request research Determine technical constraints
Projects
Cortex
Milestone
v0.14

Comments

@ospillinger
Copy link
Member

ospillinger commented Nov 20, 2019 •
edited by vishalbollu

Description

Replace Flask with another web framework (Fast API?)

Motivation

Improve API, concurrency, performance and efficiency.

Additional Context

Note from previous ticket #525:

Something like FastAPI would support multithreading, which may improve throughput

API refactor checklist

  • Revisit Python error wrapping
  • Expose multiple-workers for parallelism
@ospillinger ospillinger added enhancement New feature or request research Determine technical constraints labels Nov 20, 2019
@ospillinger ospillinger added this to Prioritize in Cortex via automation Nov 20, 2019
@deliahu deliahu moved this from Prioritize to Design in Cortex Nov 25, 2019
@deliahu deliahu changed the title Replace Flask Replace Flask [.5] Nov 25, 2019
@deliahu deliahu changed the title Replace Flask [.5] Replace Flask [0.5] Nov 25, 2019
@deliahu deliahu added the v0.12 label Nov 25, 2019
@deliahu deliahu changed the title Replace Flask [0.5] Support API parallelism [0.5] Nov 29, 2019
@kinoute
Copy link

kinoute commented Dec 7, 2019

Starlette would be a nice alternative too. It seems faster than Fast-API:

https://fastapi.tiangolo.com/benchmarks/

@deliahu deliahu moved this from Design to In progress in Cortex Dec 9, 2019
@vishalbollu vishalbollu moved this from In progress to Prioritize in Cortex Dec 18, 2019
@deliahu deliahu removed the v0.12 label Dec 19, 2019
@deliahu deliahu changed the title Support API parallelism [0.5] Support API parallelism Dec 20, 2019
@deliahu deliahu moved this from Prioritize to Design in Cortex Jan 22, 2020
@deliahu deliahu added v0.14 and removed v0.14 labels Jan 22, 2020
@vishalbollu vishalbollu mentioned this issue Mar 2, 2020
Merged
4 tasks
Cortex automation moved this from Design to Done Mar 4, 2020
@deliahu deliahu added v0.14 and removed v0.15 labels Mar 20, 2020
@deliahu deliahu added this to the v0.14 milestone Nov 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vishalbollu vishalbollu

Labels
enhancement New feature or request research Determine technical constraints
Projects
No open projects
Cortex
Done
Milestone
v0.14
Development

Successfully merging a pull request may close this issue.

Update in-flight request counter, switch to FastAPI + Uvicorn cortexlabs/cortex
4 participants
@kinoute @deliahu @vishalbollu @ospillinger