Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/docs/examples/jan.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Nitro with Jan
description: Nitro integrates with Jan to enable a ChatGPT-like functional app, optimized for local AI.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

You can effortlessly utilize Nitro through [Jan](https://jan.ai/), as it is fully integrated with all its functions. With Jan, using Nitro becomes straightforward without the need for any coding.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/examples/openai-node.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Nitro with openai-node
description: Nitro intergration guide for Node.js.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

You can migrate from OAI API or Azure OpenAI to Nitro using your existing NodeJS code quickly
Expand Down
1 change: 1 addition & 0 deletions docs/docs/examples/openai-python.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Nitro with openai-python
description: Nitro intergration guide for Python.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---


Expand Down
1 change: 1 addition & 0 deletions docs/docs/examples/palchat.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Nitro with Pal Chat
description: Nitro intergration guide for mobile device usage.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

This guide demonstrates how to use Nitro with Pal Chat, enabling local AI chat capabilities on mobile devices.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/chat.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Chat Completion
description: Inference engine for chat completion, the same as OpenAI's
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

The Chat Completion feature in Nitro provides a flexible way to interact with any local Large Language Model (LLM).
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/cont-batch.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Continuous Batching
description: Nitro's continuous batching combines multiple requests, enhancing throughput.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

Continuous batching boosts throughput and minimizes latency in large language model (LLM) inference. This technique groups multiple inference requests, significantly improving GPU utilization.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/embed.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Embedding
description: Inference engine for embedding, the same as OpenAI's
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

Embeddings are lists of numbers (floats). To find how similar two embeddings are, we measure the [distance](https://en.wikipedia.org/wiki/Cosine_similarity) between them.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/feat.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Nitro Features
description: What Nitro supports
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

Nitro enhances the `llama.cpp` research base, optimizing it for production environments with advanced features:
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/load-unload.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Load and Unload models
description: Nitro loads and unloads local AI models (local LLMs).
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

## Load model
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/multi-thread.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Multithreading
description: Nitro utilizes multithreading to optimize hardware usage.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

Multithreading in programming allows concurrent task execution, improving efficiency and responsiveness. It's key for optimizing hardware and application performance.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/prompt.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Prompt Role Support
description: Setting up Nitro prompts to build an AI assistant.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

System, user, and assistant prompt is crucial for effectively utilizing the Large Language Model. These prompts work together to create a coherent and functional conversational flow.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/features/warmup.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Warming Up Model
description: Nitro warms up the model to optimize delays.
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

Model warming up involves pre-running requests through an AI model to fine-tune its components for production. This step minimizes delays during initial inferences, ensuring readiness for immediate use.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: About Nitro
slug: /docs
description: Efficient LLM inference engine for edge computing
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

Nitro is a high-efficiency C++ inference engine for edge computing, powering [Jan](https://jan.ai/). It is lightweight and embeddable, ideal for product integration.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Architecture
slug: /achitecture
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

![Nitro Architecture](img/architecture.drawio.png)
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/build-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: Build From Source
slug: /build-source
description: Install Nitro manually
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

This guide provides step-by-step instructions for building Nitro from source on Linux, macOS, and Windows systems.
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: FAQs
slug: /faq
description: Frequently Asked Questions about Nitro
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

<details>
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: Installation
slug: /install
description: How to install Nitro
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

# Nitro Installation Guide
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/model-cycle.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Model Life Cycle
slug: /model-cycle
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

## Load model
Expand Down
1 change: 1 addition & 0 deletions docs/docs/new/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: Quickstart
slug: /quickstart
description: How to use Nitro
keywords: [Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama]
---

## Step 1: Install Nitro
Expand Down
19 changes: 15 additions & 4 deletions docs/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ const config = {
markdown: {
mermaid: true,
},

// Plugins we added
plugins: [
"docusaurus-plugin-sass",
Expand Down Expand Up @@ -125,13 +124,25 @@ const config = {
playgroundPosition: "bottom",
},
metadata: [

{ name: 'description', content: 'Nitro is a high-efficiency Large Language Model inference engine for edge computing.'},
{ name: 'keywords', content: 'Nitro, OpenAI compatible, fast inference, local AI, llm, small AI, free, open source, production ready' },
{ property: 'og:title', content: 'Embeddable AI | Nitro' },
{ name: 'keywords', content: 'Nitro, Jan, fast inference, inference server, local AI, large language model, OpenAI compatible, open source, llama' },

// Canonical URL
{ name: 'canonical', content: 'https://nitro.jan.ai/' },

// Robots tags
{ name: "robots", content: "index, follow" },

// Open Graph tags
{ property: 'og:title', content: 'Fast inference engine | Nitro' },
{ property: 'og:description', content: 'Nitro is a high-efficiency Large Language Model inference engine for edge computing.' },
{ property: 'og:type', content: 'website'},

// Twitter card tags
{ property: 'twitter:card', content: 'summary_large_image' },
{ property: 'twitter:site', content: '@janhq_' },
{ property: 'twitter:title', content: 'Embeddable AI | Nitro' },
{ property: 'twitter:title', content: 'Fast inference engine | Nitro' },
{ property: 'twitter:description', content: 'Nitro is a high-efficiency Large Language Model inference engine for edge computing.' },
],
headTags: [
Expand Down
2 changes: 1 addition & 1 deletion docs/static/robots.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
User-agent: *
Disallow:
Allow: /