Skip to content
/ JetStream Public

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

License

Apache-2.0 license
346 stars 46 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

AI-Hypercomputer/JetStream

Error
Looks like something went wrong!

About

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

Topics

gpu inference pytorch transformer llama gpt gemma model-serving tpu jax mlops large-language-models llm llmops llm-inference llama2

Resources

Readme

License

Apache-2.0 license
Activity
Custom properties

Stars

346 stars

Watchers

21 watching

Forks

46 forks
Report repository

Releases 4

v0.3 Latest
Dec 18, 2024
+ 3 releases

Packages

No packages published

Contributors 38
+ 24 contributors

Languages