Skip to content

InferDoctor v0.4.1 - Beginner-Friendly Local AI App Setup

Choose a tag to compare

View all tags
@anguoyang anguoyang released this 08 Jul 14:38
v0.4.1
c8b8bca

InferDoctor v0.4.1 is the first PyPI release of InferDoctor.

InferDoctor is a local AI stack doctor and setup assistant. It helps developers diagnose local AI environments, understand what their machine can realistically run, and start building practical local AI apps.

Local AI is powerful, but setup is often confusing. Ports, CUDA, drivers, endpoints, runtimes, model size, Docker, templates, and app scaffolding all get in the way. InferDoctor helps you check the stack, understand what is missing, and take the next practical step.

Install

pip install inferdoctor

Quick Start

inferdoctor

Example Workflow

pip install inferdoctor

inferdoctor

inferdoctor stack plan --goal customer-service --vram 24

inferdoctor template create customer-service --output ./customer-service-demo

inferdoctor template validate ./customer-service-demo

inferdoctor template smoke-test ./customer-service-demo

What Is New

  • First PyPI release
  • Local AI stack health check
  • Overall health score and Top Fixes
  • Hardware capacity guidance
  • Model fit estimates
  • Local AI stack planning
  • Starter template generation
  • Template validation
  • Template smoke tests
  • Dry-run bootstrap planning
  • Beginner-friendly workflow docs
  • Support for common local AI tools and runtimes

Supported Checks and Ecosystem

InferDoctor includes lightweight checks and guidance for:

  • Ollama
  • vLLM
  • SGLang
  • Xinference
  • Dify
  • Open WebUI
  • LM Studio
  • llama.cpp
  • Docker
  • CUDA
  • NVIDIA GPUs
  • OpenAI-compatible local endpoints

InferDoctor does not replace these tools. It helps you understand whether your local stack is ready to use them, what is missing, and what to try next.

Setup Assistant Workflow

InferDoctor is no longer just a checker. It now helps users move through a practical local AI setup flow:

  1. Diagnose the machine and local AI stack.
  2. Understand hardware capacity.
  3. Choose a reasonable setup path.
  4. Generate a starter app template.
  5. Validate the generated project.
  6. Run a safe template smoke test.
  7. Follow a dry-run bootstrap plan before doing anything risky.

Useful commands:

inferdoctor

inferdoctor capacity

inferdoctor recommend --goal customer-service --vram 24

inferdoctor stack plan --goal customer-service --vram 24

inferdoctor stack bootstrap --goal customer-service --dry-run

inferdoctor template list

inferdoctor template show customer-service

inferdoctor template create customer-service --output ./customer-service-demo

inferdoctor template validate ./customer-service-demo

inferdoctor template smoke-test ./customer-service-demo

inferdoctor model fit --size 14b --quant q4 --vram 24

Starter Templates

InferDoctor includes starter templates for local AI app scenarios such as:

  • Customer service chatbot
  • Restaurant ordering assistant
  • Local document Q&A
  • Personal knowledge base
  • Meeting notes assistant
  • OpenAI-compatible API demo
  • Ollama chat starter
  • Dify RAG starter
  • Open WebUI starter
  • vLLM API starter
  • SGLang API starter

Templates are designed to be beginner-friendly and local-first. They generate project files, configuration examples, sample data, prompts, and troubleshooting notes.

Why InferDoctor?

Model recommendation tools help you choose a model.

InferDoctor helps you understand whether your local AI stack is ready, what is broken, and what to fix next.

It is designed for people who want to run local AI but do not want to spend hours debugging ports, drivers, endpoints, configs, runtimes, and model size assumptions.

Safety

InferDoctor is lightweight and safe by default.

It does not download models by default.

It does not run model inference by default.

It does not install AI runtimes.

It does not modify system settings.

Template and bootstrap flows are dry-run or file-generation oriented unless explicitly extended in the future.

Capacity and model-fit outputs are heuristic guidance, not benchmark results or hardware guarantees.

Known Limitations

  • InferDoctor does not install Ollama, vLLM, SGLang, Dify, Open WebUI, or other runtimes for you.
  • It does not download model weights.
  • It does not benchmark real inference speed.
  • It does not guarantee that a specific model will fit or perform well.
  • Some checks depend on local services being reachable on their configured endpoints.
  • Capacity and model-fit estimates are intentionally conservative heuristics.

Roadmap

Next planned areas include:

  • More starter templates
  • Safer bootstrap workflows
  • Docker Compose generation improvements
  • Dify and Open WebUI starter flows
  • Japanese enterprise RAG setup examples
  • Community template registry foundation
  • More practical local AI workflow guides

Links

GitHub:

https://github.com/anguoyang/inferdoctor

PyPI:

https://pypi.org/project/inferdoctor/

Install:

pip install inferdoctor
Assets 2
Loading