🗺️ Keras Development Roadmap

[fchollet]

Here's an overview of the features we intend to work on in the near future.

Core Keras

Performance and Optimization

• Introduce comprehensive support for model quantization, including:
  • Post-training quantization techniques like GPTQ and AWQ.
  • Quantization-Aware Training (QAT) int8 support.

Scale and Distribution

• Distributed Training
  • Comprehensive guides for multi-host TPU and multi-host GPU training.
  • Official performance benchmarks
  • A Backup and Restore callback to handle preemptions gracefully during long training runs.

Integrations and Ecosystem

• Add support for exporting models to the ODML LiteRT format, simplifying deployment on edge and mobile devices.
• Integrate Qwix, a new JAX-based library for quantization.
• [Contributions Welcome] Integrate PyGrain for creating efficient, large-scale data loading and preprocessing pipelines.

Guides and Tutorials

• Deployment Guides: End-to-end tutorials on deploying Keras models to Vertex AI, and on-device via LiteRT.
• Guide on efficient inference using KerasHub models with vLLM.
• AI Agents and RAG: Advanced examples of building AI agents with function calling and creating Retrieval-Augmented Generation (RAG) pipelines.
• Training Techniques: Guides on model distillation, handling training preemptions on TPUs, and best practices for image augmentation (e.g., CutMix and MixUp).
• Others: Orbax checkpointing, FLUX model guide/example, etc.

KerasHub

See the roadmap here.

KerasRS

See the roadmap here.

[pure-rgb]

Unify KerasCV and KerasNLP into a single Keras Models package.

1. does it mean that kerascv and kerasnlp will be merged together into one single package, like tensorflow model garden?

2. kerascv development is terribly slow, and lacks of resources. no release since february (0.8.2), may models doesn't have weight, i.e. vit, mit. Object detection models that is added (yolo-v8, retina-net, faster-rcnn) all are broken and doesn't have evaluation scripts to holds their validity. Generative model only stable diffusion, where many (too many impactfull models are available now).

3. like huggigface, why enigneer or researcher are not hired and reinformce into this package. Check this repo, the ownver can add any new models written in keras. so why not official keras team? Is it because the api desgin is too rich and complicated for contributor? Not to mention, many reviews are pending, contributor left at th end.

[IMvision12]

I concur with @pure-rgb's observation that the API design has become overly elaborate and intricate for contributors. Previously, it was straightforward and comprehensible, but with the introduction of the Backbone API, it has become more convoluted.

[pure-rgb]

 with the introduction of the Backbone API, it has become more convoluted.

The person who introduced such API design already left google. :p

[kernel-loophole]

@fchollet I've been going through the features you're planning to implement, and I'm particularly interested in contributing to KerasNLP. Specifically, I'm eager to get involved in the development of dynamic sequence length inference for PyTorch LLMs.

[fchollet]

@kernel-loophole please open an issue on the KerasNLP repo if you'd like to contribute this feature!

[kernel-loophole]

okay thanks

[MoFHeka]

I am a developer of tensorflow recommenders-addons and I now need to develop an all-to-all embedding layer for multi-GPU distributed training of recommendation models. The old tensorflow distributed strategy clearly did not meet this need.
So the question is, should I develop on TF DTensor or Jax? Because it seems that Keras support for TF DTensor is not friendly. But Jax lacks the ability to online inference services and the functional components used by various recommendation algorithms. Also recommenders-addons has a lot of custom operators.