Skip to content

Latest commit

 

History

History
103 lines (30 loc) · 5.3 KB

RNP-008.md

File metadata and controls

103 lines (30 loc) · 5.3 KB
Raw

RNP-008: Nosana Compute Client

RNP # Title Category Authors Created Status
008 Nosana Compute Client Core Laurens and Sjoerd 09-01-2024 Approved + On the Roadmap

Overview

This proposal outlines onboarding Nosana as the next Compute Client to leverage Render Network's GPU supply for AI inference workloads.

Nosana (https://nosana.io) is a DePIN that enables developers and businesses to effortlessly, affordably, and securely construct, deploy, and profit from their AI inference applications. It offers comprehensive support with high-performance inference model deployment libraries, user-friendly inference operations and a robust GPU marketplace.

Nosana will serve as a conduit between the AI inference community and Render's dispersed GPU network.

Motivation

The swift advancement of inference models, particularly in the realms of Large Language Models (LLMs) and image generation, has led to extraordinary demand for GPUs around the world. This heightened demand is accelerating upward price pressure and is largely responsible for global GPU shortages, which in turn has a negative effect on developers and researchers who rely on these resources to drive innovation in AI inference applications.

Render Network's global infrastructure of varied nodes stands as an optimal solution to meet these AI inference demands. Render is adept at providing immediate access to the specialized GPUs necessary for large-scale inference. Despite this, current AI enthusiast and startups are not fully equipped to handle the deployment of their resource-intensive inference tasks across a distributed network of retail GPUs, highlighting a need for more robust and scalable solutions.

The Nosana platform (https://app.nosana.io/) bridges this gap through a distributed compute market, solving three key industry problems:

  1. GPU Shortage: To address the GPU shortage due to the soaring demand for inference models in business applications, Nosana offers a GPU marketplace that allows GPU providers to join easily through its "Share and Earn" interface. This connects consumer hardware to the marketplace, effectively alleviating the GPU shortage.

  2. Idle compute: Nosana tackles the issue of idle compute. Many end-users do not utilize their GPU resources 24x7, including miners, gamers, and high-end device owners. By issuing a simple Nosana launch command, these idle resources can be instantly allocated to start inference jobs on the most cost-effective GPU resources, without the need for complex resource provisioning, environment setup, and management.

  3. Price mismatch: Nosana offers a solution to exorbitant pricing in the (public cloud) industry, by providing AI inference clients with GPU resources that are up to 85% cheaper compared to traditional cloud providers. This ensures developers can affordably deploy their inference models.

Technical Integration

We recommend granting an independent working group (The Nosana-Render Working Group) 30K RENDER tokens on a milestone basis to integrate Nosana as a Compute Client.

The Nosana-Render Working Group (NRWG) is entrusted with developing the essential infrastructure to realize the following objectives, with the corresponding percentage of the total grant unlocking upon completing each milestone:

Nosana-Render Connectors (50%)

Implement Nosana-Render Connectors that receive [compute jobs] from the Nosana Network, convert the Nosana JSON flow job to a job that can be run on a Render Node (preferably in a docker/podman container with GPU passthrough) and post that job with the Render Network API. The connector waits for the results to finish on Render Network before posting the result back to the Nosana Network.

Direct connection to Nosana Network on Render Nodes (25%)

Render Nodes can run a container with the Nosana Container Engine to directly join decentralized compute markets on the Nosana Network and pick up and execute jobs.

Node Discovery (15%)

The Nosana Container Engine will identify and report node hardware capabilities such as GPU type, RAM, CPU cores, storage and network speed and use this information to register as a Nosana Node. This way jobs can be more easily matched to the correct nodes.

Pre-loaded AI models (10%)

The Nosana Engine will pre-load AI models on the Render Nodes, allowing for quick AI inference jobs. We could create some services on top of this to showcase the power of running quick and scalable AI inferences with open source AI models, like Stable Diffusion (text-to-image) or Whisper (speech-to-text)