+
+## Benefits of Llama stack
+
+### Current challenges in custom AI applications
+
+Building production AI applications today requires solving multiple challenges:
+
+**Infrastructure Complexity**
+
+- Running large language models efficiently requires specialized infrastructure.
+- Different deployment scenarios (local development, cloud, edge) need different solutions.
+- Moving from development to production often requires significant rework.
+
+**Essential Capabilities**
+
+- Safety guardrails and content filtering are necessary in an enterprise setting.
+- Just model inference is not enough - Knowledge retrieval and RAG capabilities are required.
+- Nearly any application needs composable multi-step workflows.
+- Without monitoring, observability and evaluation, you end up operating in the dark.
+
+**Lack of Flexibility and Choice**
+
+- Directly integrating with multiple providers creates tight coupling.
+- Different providers have different APIs and abstractions.
+- Changing providers requires significant code changes.
+
+### Our Solution: A Universal Stack
+
+Llama Stack addresses these challenges through a service-oriented, API-first approach:
+
+**Develop Anywhere, Deploy Everywhere**
+- Start locally with CPU-only setups
+- Move to GPU acceleration when needed
+- Deploy to cloud or edge without code changes
+- Same APIs and developer experience everywhere
+
+**Production-Ready Building Blocks**
+- Pre-built safety guardrails and content filtering
+- Built-in RAG and agent capabilities
+- Comprehensive evaluation toolkit
+- Full observability and monitoring
+
+**True Provider Independence**
+- Swap providers without application changes
+- Mix and match best-in-class implementations
+- Federation and fallback support
+- No vendor lock-in
+
+**Robust Ecosystem**
+- Llama Stack is already integrated with distribution partners (cloud providers, hardware vendors, and AI-focused companies).
+- Ecosystem offers tailored infrastructure, software, and services for deploying a variety of models.
+
+
+## Our Philosophy
+
+- **Service-Oriented**: REST APIs enforce clean interfaces and enable seamless transitions across different environments.
+- **Composability**: Every component is independent but works together seamlessly
+- **Production Ready**: Built for real-world applications, not just demos
+- **Turnkey Solutions**: Easy to deploy built in solutions for popular deployment scenarios
+
+
+With Llama Stack, you can focus on building your application while we handle the infrastructure complexity, essential capabilities, and provider integrations.
diff --git a/versioned_docs/version-v0.2.23/concepts/distributions.mdx b/versioned_docs/version-v0.2.23/concepts/distributions.mdx
new file mode 100644
index 0000000..5680996
--- /dev/null
+++ b/versioned_docs/version-v0.2.23/concepts/distributions.mdx
@@ -0,0 +1,16 @@
+---
+title: Distributions
+description: Pre-packaged provider configurations for different deployment scenarios
+sidebar_label: Distributions
+sidebar_position: 3
+---
+
+# Distributions
+
+While there is a lot of flexibility to mix-and-match providers, often users will work with a specific set of providers (hardware support, contractual obligations, etc.) We therefore need to provide a _convenient shorthand_ for such collections. We call this shorthand a **Llama Stack Distribution** or a **Distro**. One can think of it as specific pre-packaged versions of the Llama Stack. Here are some examples:
+
+**Remotely Hosted Distro**: These are the simplest to consume from a user perspective. You can simply obtain the API key for these providers, point to a URL and have _all_ Llama Stack APIs working out of the box. Currently, [Fireworks](https://fireworks.ai/) and [Together](https://together.xyz/) provide such easy-to-consume Llama Stack distributions.
+
+**Locally Hosted Distro**: You may want to run Llama Stack on your own hardware. Typically though, you still need to use Inference via an external service. You can use providers like HuggingFace TGI, Fireworks, Together, etc. for this purpose. Or you may have access to GPUs and can run a [vLLM](https://github.com/vllm-project/vllm) or [NVIDIA NIM](https://build.nvidia.com/nim?filters=nimType%3Anim_type_run_anywhere&q=llama) instance. If you "just" have a regular desktop machine, you can use [Ollama](https://ollama.com/) for inference. To provide convenient quick access to these options, we provide a number of such pre-configured locally-hosted Distros.
+
+**On-device Distro**: To run Llama Stack directly on an edge device (mobile phone or a tablet), we provide Distros for [iOS](/docs/distributions/ondevice_distro/ios_sdk) and [Android](/docs/distributions/ondevice_distro/android_sdk)
diff --git a/versioned_docs/version-v0.2.23/concepts/evaluation_concepts.mdx b/versioned_docs/version-v0.2.23/concepts/evaluation_concepts.mdx
new file mode 100644
index 0000000..c7a13fd
--- /dev/null
+++ b/versioned_docs/version-v0.2.23/concepts/evaluation_concepts.mdx
@@ -0,0 +1,78 @@
+---
+title: Evaluation Concepts
+description: Running evaluations on Llama Stack
+sidebar_label: Evaluation Concepts
+sidebar_position: 5
+---
+
+# Evaluation Concepts
+
+The Llama Stack Evaluation flow allows you to run evaluations on your GenAI application datasets or pre-registered benchmarks.
+
+We introduce a set of APIs in Llama Stack for supporting running evaluations of LLM applications:
+- `/datasetio` + `/datasets` API
+- `/scoring` + `/scoring_functions` API
+- `/eval` + `/benchmarks` API
+
+This guide goes over the sets of APIs and developer experience flow of using Llama Stack to run evaluations for different use cases. Checkout our Colab notebook on working examples with evaluations [here](https://colab.research.google.com/drive/10CHyykee9j2OigaIcRv47BKG9mrNm0tJ?usp=sharing).
+
+The Evaluation APIs are associated with a set of Resources. Please visit the Resources section in our [Core Concepts](./index.mdx) guide for better high-level understanding.
+
+- **DatasetIO**: defines interface with datasets and data loaders.
+ - Associated with `Dataset` resource.
+- **Scoring**: evaluate outputs of the system.
+ - Associated with `ScoringFunction` resource. We provide a suite of out-of-the box scoring functions and also the ability for you to add custom evaluators. These scoring functions are the core part of defining an evaluation task to output evaluation metrics.
+- **Eval**: generate outputs (via Inference or Agents) and perform scoring.
+ - Associated with `Benchmark` resource.
+
+## Open-benchmark Eval
+
+### List of open-benchmarks Llama Stack support
+
+Llama stack pre-registers several popular open-benchmarks to easily evaluate model perfomance via CLI.
+
+The list of open-benchmarks we currently support:
+- [MMLU-COT](https://arxiv.org/abs/2009.03300) (Measuring Massive Multitask Language Understanding): Benchmark designed to comprehensively evaluate the breadth and depth of a model's academic and professional understanding
+- [GPQA-COT](https://arxiv.org/abs/2311.12022) (A Graduate-Level Google-Proof Q&A Benchmark): A challenging benchmark of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry.
+- [SimpleQA](https://openai.com/index/introducing-simpleqa/): Benchmark designed to access models to answer short, fact-seeking questions.
+- [MMMU](https://arxiv.org/abs/2311.16502) (A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI)]: Benchmark designed to evaluate multimodal models.
+
+You can follow this [contributing guide](../references/evals_reference/#open-benchmark-contributing-guide) to add more open-benchmarks to Llama Stack
+
+### Run evaluation on open-benchmarks via CLI
+
+We have built-in functionality to run the supported open-benckmarks using llama-stack-client CLI
+
+#### Spin up Llama Stack server
+
+Spin up llama stack server with 'open-benchmark' template
+```bash
+llama stack run llama_stack/distributions/open-benchmark/run.yaml
+```
+
+#### Run eval CLI
+There are 3 necessary inputs to run a benchmark eval
+- `list of benchmark_ids`: The list of benchmark ids to run evaluation on
+- `model-id`: The model id to evaluate on
+- `output_dir`: Path to store the evaluate results
+
+```bash
+llama-stack-client eval run-benchmark | Local Inference | +Remote Inference | +
|---|---|
| + +``` +client = LlamaStackClientLocalClient + .builder() + .modelPath(modelPath) + .tokenizerPath(tokenizerPath) + .temperature(temperature) + .build() +``` + | ++ +``` +// remoteURL is a string like "http://localhost:5050" +client = LlamaStackClientOkHttpClient + .builder() + .baseUrl(remoteURL) + .build() +``` + | +
+
+Our goal is to provide pre-packaged implementations (aka "distributions") which can be run in a variety of deployment environments. LlamaStack can assist you in your entire app development lifecycle - start iterating on local, mobile or desktop and seamlessly transition to on-prem or public cloud deployments. At every point in this transition, the same set of APIs and the same developer experience is available.
+
+## How does Llama Stack work?
+
+Llama Stack consists of a server (with multiple pluggable API providers) and Client SDKs meant to be used in your applications. The server can be run in a variety of environments, including local (inline) development, on-premises, and cloud. The client SDKs are available for Python, Swift, Node, and Kotlin.
+
+## Quick Links
+
+- Ready to build? Check out the [Getting Started Guide](https://llama-stack.github.io/getting_started/quickstart) to get started.
+- Want to contribute? See the [Contributing Guide](https://github.com/llamastack/llama-stack/blob/main/CONTRIBUTING.md).
+- Explore [Example Applications](https://github.com/llamastack/llama-stack-apps) built with Llama Stack.
+
+## Rich Ecosystem Support
+
+Llama Stack provides adapters for popular providers across all API categories:
+
+- **Inference**: Meta Reference, Ollama, Fireworks, Together, NVIDIA, vLLM, AWS Bedrock, OpenAI, Anthropic, and more
+- **Vector Databases**: FAISS, Chroma, Milvus, Postgres, Weaviate, Qdrant, and others
+- **Safety**: Llama Guard, Prompt Guard, Code Scanner, AWS Bedrock
+- **Training & Evaluation**: HuggingFace, TorchTune, NVIDIA NEMO
+
+:::info Provider Details
+For complete provider compatibility and setup instructions, see our [Providers Documentation](https://llamastack.github.io/providers/).
+:::
+
+## Get Started Today
+
+
diff --git a/versioned_docs/version-v0.2.23/providers/agents/index.mdx b/versioned_docs/version-v0.2.23/providers/agents/index.mdx
new file mode 100644
index 0000000..5cd3777
--- /dev/null
+++ b/versioned_docs/version-v0.2.23/providers/agents/index.mdx
@@ -0,0 +1,27 @@
+---
+description: "Agents API for creating and interacting with agentic systems.
+
+ Main functionalities provided by this API:
+ - Create agents with specific instructions and ability to use tools.
+ - Interactions with agents are grouped into sessions (\"threads\"), and each interaction is called a \"turn\".
+ - Agents can be provided with various tools (see the ToolGroups and ToolRuntime APIs for more details).
+ - Agents can be provided with various shields (see the Safety API for more details).
+ - Agents can also use Memory to retrieve information from knowledge bases. See the RAG Tool and Vector IO APIs for more details."
+sidebar_label: Agents
+title: Agents
+---
+
+# Agents
+
+## Overview
+
+Agents API for creating and interacting with agentic systems.
+
+ Main functionalities provided by this API:
+ - Create agents with specific instructions and ability to use tools.
+ - Interactions with agents are grouped into sessions ("threads"), and each interaction is called a "turn".
+ - Agents can be provided with various tools (see the ToolGroups and ToolRuntime APIs for more details).
+ - Agents can be provided with various shields (see the Safety API for more details).
+ - Agents can also use Memory to retrieve information from knowledge bases. See the RAG Tool and Vector IO APIs for more details.
+
+This section contains documentation for all available providers for the **agents** API.
diff --git a/versioned_docs/version-v0.2.23/providers/agents/inline_meta-reference.mdx b/versioned_docs/version-v0.2.23/providers/agents/inline_meta-reference.mdx
new file mode 100644
index 0000000..fd96174
--- /dev/null
+++ b/versioned_docs/version-v0.2.23/providers/agents/inline_meta-reference.mdx
@@ -0,0 +1,29 @@
+---
+description: "Meta's reference implementation of an agent system that can use tools, access vector databases, and perform complex reasoning tasks."
+sidebar_label: Meta-Reference
+title: inline::meta-reference
+---
+
+# inline::meta-reference
+
+## Description
+
+Meta's reference implementation of an agent system that can use tools, access vector databases, and perform complex reasoning tasks.
+
+## Configuration
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `persistence_store` | `utils.kvstore.config.RedisKVStoreConfig \| utils.kvstore.config.SqliteKVStoreConfig \| utils.kvstore.config.PostgresKVStoreConfig \| utils.kvstore.config.MongoDBKVStoreConfig` | No | sqlite | |
+| `responses_store` | `utils.sqlstore.sqlstore.SqliteSqlStoreConfig \| utils.sqlstore.sqlstore.PostgresSqlStoreConfig` | No | sqlite | |
+
+## Sample Configuration
+
+```yaml
+persistence_store:
+ type: sqlite
+ db_path: ${env.SQLITE_STORE_DIR:=~/.llama/dummy}/agents_store.db
+responses_store:
+ type: sqlite
+ db_path: ${env.SQLITE_STORE_DIR:=~/.llama/dummy}/responses_store.db
+```
diff --git a/versioned_docs/version-v0.2.23/providers/batches/index.mdx b/versioned_docs/version-v0.2.23/providers/batches/index.mdx
new file mode 100644
index 0000000..2c64b27
--- /dev/null
+++ b/versioned_docs/version-v0.2.23/providers/batches/index.mdx
@@ -0,0 +1,31 @@
+---
+description: "The Batches API enables efficient processing of multiple requests in a single operation,
+ particularly useful for processing large datasets, batch evaluation workflows, and
+ cost-effective inference at scale.
+
+ The API is designed to allow use of openai client libraries for seamless integration.
+
+ This API provides the following extensions:
+ - idempotent batch creation
+
+ Note: This API is currently under active development and may undergo changes."
+sidebar_label: Batches
+title: Batches
+---
+
+# Batches
+
+## Overview
+
+The Batches API enables efficient processing of multiple requests in a single operation,
+ particularly useful for processing large datasets, batch evaluation workflows, and
+ cost-effective inference at scale.
+
+ The API is designed to allow use of openai client libraries for seamless integration.
+
+ This API provides the following extensions:
+ - idempotent batch creation
+
+ Note: This API is currently under active development and may undergo changes.
+
+This section contains documentation for all available providers for the **batches** API.
diff --git a/versioned_docs/version-v0.2.23/providers/batches/inline_reference.mdx b/versioned_docs/version-v0.2.23/providers/batches/inline_reference.mdx
new file mode 100644
index 0000000..f438005
--- /dev/null
+++ b/versioned_docs/version-v0.2.23/providers/batches/inline_reference.mdx
@@ -0,0 +1,27 @@
+---
+description: "Reference implementation of batches API with KVStore persistence."
+sidebar_label: Reference
+title: inline::reference
+---
+
+# inline::reference
+
+## Description
+
+Reference implementation of batches API with KVStore persistence.
+
+## Configuration
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `kvstore` | `utils.kvstore.config.RedisKVStoreConfig \| utils.kvstore.config.SqliteKVStoreConfig \| utils.kvstore.config.PostgresKVStoreConfig \| utils.kvstore.config.MongoDBKVStoreConfig` | No | sqlite | Configuration for the key-value store backend. |
+| `max_concurrent_batches` | ``{input}`
`<|assistant|>`
`{output}` | | +| `model_specific_config` | `
`{input}`
`<|assistant|>`
`{output}` | | +| `model_specific_config` | `
`{input}`
`<|assistant|>`
`{output}` | | +| `model_specific_config` | `