Skip to content
@project-codeflare

project-codeflare

Simplifying and automating scaling of distributed AI/ML

Project CodeFlare

About

Project CodeFlare provides a simple, user-friendly abstraction for developing, resource-scaling, queuing, and management of distributed AI/ML and Python workloads, whilst maximising the utilization of accelerators and compute resources, on OpenShift Container Platform.

Project CodeFlare consists of the following components:

CodeFlare SDK to define, develop, and control remote distributed compute jobs and infrastructure from either a python-based environment or command-line interface

AppWrapper a flexible and workload-agnostic mechanism to enable Kueue to manage a group of Kubernetes resources as a single logical unit and to provide an additional level of automatic fault detection and recovery

CodeFlare Operator for automating deployment and configuration of the Project CodeFlare stack

Project CodeFlare Ecosystem

In addition to running standalone, Project CodeFlare is deployed as part of and integrated with the Open Data Hub.

Getting Started

Learning

Watch this video for an introduction to Project CodeFlare and what the stack can do. (Nov. 2022)

See this video as well for an updated demonstration of the basic stack functionality in-action. (Jun. 2023)

Quick Start

To get started using the Project CodeFlare stack, try this end-to-end example!

For more basic walk-throughs and in-depth tutorials, see our demo notebooks!

Development

See more details in any of the component repos linked above, and go to their issues page for open tasks/issues!

Architecture

We attempt to document all architectural decisions in our ADR documents. Start here to understand the architectural details of Project CodeFlare.

Getting Involved

Join our Slack community to get involved or ask questions.

License

Unless otherwise noted at a per-component level, this Project CodeFlare is licensed under the Apache-2.0 License.

Pinned Loading

  1. codeflare codeflare Public

    Simplifying the definition and execution, scaling and deployment of pipelines on the cloud.

    Jupyter Notebook 224 36

  2. multi-cluster-app-dispatcher multi-cluster-app-dispatcher Public

    Holistic job manager on Kubernetes

    Go 108 63

  3. codeflare-cli codeflare-cli Public

    TypeScript 11 12

  4. codeflare-sdk codeflare-sdk Public

    An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute…

    Python 22 44

  5. codeflare-operator codeflare-operator Public

    Operator for installation and lifecycle management of CodeFlare distributed workload stack

    Go 7 43

  6. instascale instascale Public

    On-demand Kubernetes/OpenShift cluster scaling and aggregated resource provisioning

    Go 10 20

Repositories

Showing 10 of 30 repositories
  • codeflare-sdk Public

    An intuitive, easy-to-use python interface for batch resource requesting, access, job submission, and observation. Simplifying the developer's life while enabling access to high-performance compute resources, either in the cloud or on-prem.

    project-codeflare/codeflare-sdk’s past year of commit activity
    Python 22 Apache-2.0 44 60 12 Updated Oct 31, 2024
  • appwrapper Public

    AppWrapper controller for Kueue

    project-codeflare/appwrapper’s past year of commit activity
    Go 5 Apache-2.0 7 6 0 Updated Oct 31, 2024
  • mlbatch Public

    Queuing and quota management for AI/ML batch jobs on Kubernetes

    project-codeflare/mlbatch’s past year of commit activity
    JavaScript 1 Apache-2.0 3 2 0 Updated Oct 25, 2024
  • codeflare-common Public

    Common packages for use with CodeFlare Distributed Workload stack.

    project-codeflare/codeflare-common’s past year of commit activity
    Go 0 Apache-2.0 14 2 0 Updated Oct 21, 2024
  • .github Public
    project-codeflare/.github’s past year of commit activity
    0 6 0 0 Updated Oct 18, 2024
  • notebooks Public Forked from opendatahub-io/notebooks

    Notebook images for ODH

    project-codeflare/notebooks’s past year of commit activity
    Python 0 Apache-2.0 60 0 2 Updated Oct 17, 2024
  • community-operators-prod Public Forked from redhat-openshift-ecosystem/community-operators-prod

    community-operators metadata backing OpenShift OperatorHub

    project-codeflare/community-operators-prod’s past year of commit activity
    Dockerfile 0 Apache-2.0 537 0 1 Updated Oct 17, 2024
  • codeflare-operator Public

    Operator for installation and lifecycle management of CodeFlare distributed workload stack

    project-codeflare/codeflare-operator’s past year of commit activity
    Go 7 Apache-2.0 43 28 7 Updated Oct 17, 2024
  • instascale Public

    On-demand Kubernetes/OpenShift cluster scaling and aggregated resource provisioning

    project-codeflare/instascale’s past year of commit activity
    Go 10 Apache-2.0 20 36 (4 issues need help) 6 Updated Oct 15, 2024
  • instaslice Public

    InstaSlice facilitates the use of Dynamic Resource Allocation (DRA) on Kubernetes clusters for GPU sharing

    project-codeflare/instaslice’s past year of commit activity
    Go 22 2 0 0 Updated Aug 2, 2024

Top languages

Loading…

Most used topics

Loading…