[V1] Logits processors extensibility

[afeldman-nm]

Purpose

Enable V1 logits processors support to be extended with custom logits processors.

New Python interface for custom logitsprocs

    # Specify logitproc by entrypoint
    llm = LLM(
        model="facebook/opt-125m",
        logits_processors_entrypoints=["dummy_logitproc"],
    )

    # Specify logitproc by fully-qualified name
    llm = LLM(
        model="facebook/opt-125m",
        logits_processors_fqns=["vllm.v1.sample.logits_processor.impls:DummyLogitsProcessor"],
    )

New CLI interface for custom logitsprocs

# Engine CLI args for specifying logitsprocs by entrypoint
--logits-processors-entrypoints=dummy_logitproc,<other logitproc>,...

# Engine CLI args for specifying logitsprocs by fully-qualified name
--logits-processors-fqns=vllm.v1.sample.logits_processor.impls:DummyLogitsProcessor,<other logitproc>,...

Test Plan

(WIP)

• Configuring a contrived custom logits proc
• E2E tests using REST API and Python interfaces
• Wrap a V0-style logits proc to create a V1-style logits proc

Test Result

WIP

(Optional) Documentation Update

WIP

Fixes #17799
Fixes #12678

[njhill]

Thanks @afeldman-nm it's looking a lot better now. A few remaining comments but getting close I think.

[njhill]

Thanks @afeldman-nm

[njhill]

Does it make sense to support LP for pooling requests? Maybe we should just keep this as SamplingParams for now?

[afeldman-nm]

Okay. I guess since logitsprocs are enabled at server init time, we can't have a given request fail because it "enabled" some particular logitsproc. So the proper behavior is probably just to bypass logitsprocs logic for pooling requests

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @afeldman-nm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[njhill]

Wondering whether we should keep this to just SamplingParams