feat: OpenAI Compatible Chat Completions server

[maxim-saplin]

The change adds "server" mode to MassGen which spins up a self-hosted OpenAI compatible Chat Completions endpoint and proxies requests to MassGen agents. Fixes #628.

Closes MAS-152

Example:

• Start the server with a config

massgen serve --config balanced.yaml --port 4000

• Any OpenAI-compatible client works

 from openai import OpenAI
 client = OpenAI(base_url="http://localhost:4000/v1", api_key="unused")

 response = client.chat.completions.create(
     model="massgen",  # Ignored when config is provided
     messages=[{"role": "user", "content": "Your question"}]
 )
 print(response.choices[0].message.content)  # Final answer
 print(response.choices[0].message.reasoning_content)  # Traces

[arita37]

@maxim-saplin
It's a nice one thanks !

Just noticed, in the github actions,
there is no code to "live test" the massgen library.

github provides free tokens on gpt4-mini (enough for basic testing)

Example

https://docs.github.com/en/github-models/use-github-models/prototyping-with-ai-models

https://docs.github.com/en/rest/models/catalog?apiVersion=2022-11-28


import os
from openai import OpenAI

# Initialize client with the GitHub Models inference endpoint
client = OpenAI(
    base_url="https://models.github.ai/inference",
    api_key=os.environ.get("GITHUB_TOKEN"),
)

try:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {
                "role": "system", 
                "content": "You are a helpful assistant."
            },
            {
                "role": "user", 
                "content": "Explain recursion in one sentence."
            },
        ],
        temperature=0.7,
    )

    print(response.choices[0].message.content)

except Exception as e:
    print(f"Error: {e}")

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[ncrispino]

@maxim-saplin It's a nice one thanks !

Just noticed, in the github actions, there is no code to "live test" the massgen library.

github provides free tokens on gpt4-mini (enough for basic testing)

Example

https://docs.github.com/en/github-models/use-github-models/prototyping-with-ai-models

https://docs.github.com/en/rest/models/catalog?apiVersion=2022-11-28


import os
from openai import OpenAI

# Initialize client with the GitHub Models inference endpoint
client = OpenAI(
    base_url="https://models.github.ai/inference",
    api_key=os.environ.get("GITHUB_TOKEN"),
)

try:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {
                "role": "system", 
                "content": "You are a helpful assistant."
            },
            {
                "role": "user", 
                "content": "Explain recursion in one sentence."
            },
        ],
        temperature=0.7,
    )

    print(response.choices[0].message.content)

except Exception as e:
    print(f"Error: {e}")

Created issue #747 for this