refactor: reorganize dependencies following PEP 735

[ajbozarth]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: chore: clean up dependencies #583

Summary

Reorganizes dependencies in `pyproject.toml` following PEP 735 standards for better modularity and lighter installs.

Changes

• Removed unused dependencies: json5, ansicolors
• Moved to optional groups: uvicorn, fastapi (server), llm-sandbox (sandbox), huggingface-hub (hf)
• Created new optional groups: server, sandbox, granite_retriever
• Split dev into modular groups: lint, test, typecheck, build, release
• Added dev group using PEP 735 include-group syntax (auto-installed by uv run)
• Reorganized tools group: Added langchain-core, moved langchain-community/ddgs to test
• Updated documentation: CONTRIBUTING.md

Benefits

• 40% lighter core install (removed 8 unnecessary dependencies)
• Clear organization by purpose
• Modular dev workflow (install only what you need)
• Modern PEP 735 standard
• Better DX (uv run automatically includes dev dependencies)

Validation

• ✅ 365 tests passed
• ✅ Linting passed (ruff)
• ✅ Type checking passed (mypy - 394 files)
• ✅ uv run automatically installs dev dependencies
• ✅ All dependency group combinations work correctly

Closes #583

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[ajbozarth]

@nrfulton @jakelorocco here's what I've come up with on cleaning up the dependencies. So far I've stuck to just cleaning up the pyproject for now. As noted in the description I only removed two unused dependencies, the rest were only moved around into different locations based on detected usage.

I now plan to explore the optional follow up task of finding and cleaning up any imports in the code that may or may not need refactoring based on these updates.

This should be thoroughly tested by multiple people on multiple envs before merge (@planetf1 @psschwei)

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[ajbozarth]

I now plan to explore the optional follow up task of finding and cleaning up any imports in the code that may or may not need refactoring based on these updates.

This was simpler than I expected, pushed c0e45df addressing it

In short, since dependencies only moved out of core and not into it there were no checks to be removed. The above commit addresses imports who changed groups. Lastly the imports moved into optional deps were already "wrapped" in existing checks (eg hf)

So development is complete and this is ready for review and testing

[jakelorocco]

Thank you for cleaning this up. I will try to run the tests today and make sure everything runs on a clean install. Left a few questions.

[ajbozarth]

@jakelorocco I believe I've addressed all your comments if you could take a another look

[jakelorocco]

tests passed locally on macos and on linux