[misc][data.llm] Generalize the builder pattern in ray.data.llm

[jeffreyjeffreywang]

Description

Briefly describe what this PR accomplishes and why it's needed.

As discussed in https://docs.google.com/document/d/1danbyJjd3Zl_Q-CSsS3PjxtG4K9dZkyn0A9t7i4Fyjg/edit?disco=AAABtNCDbfw, the current builder function build_llm_processor is overly specific to LLM inference workloads and not flexible enough to support additional processors, such as those for multimodal preprocessing. To address this, we’ve generalized it to build_processor to better accommodate a broader range of LLM-related workloads.

Related issues

Link related issues: "Fixes #1234", "Closes #1234", or "Related to #1234".

N/A

Additional information

Optional: Add implementation details, API changes, usage examples, screenshots, etc.

Original:

import ray
from ray.data.llm import vLLMEngineProcessorConfig, build_processor

config = vLLMEngineProcessorConfig(
    model_source="meta-llama/Meta-Llama-3.1-8B-Instruct",
    concurrency=1,
    batch_size=64,
)

processor = build_llm_processor(
    config,
    preprocess=lambda row: dict(
        messages=[{"role": "user", "content": row["prompt"]}],
        sampling_params=dict(temperature=0.3, max_tokens=20),
    ),
    postprocess=lambda row: dict(resp=row["generated_text"]),
    preprocess_map_kwargs={"num_cpus": 0.5},
    postprocess_map_kwargs={"num_cpus": 0.25},
)

ds = ray.data.range(300)
ds = processor(ds)
for row in ds.take_all():
    print(row)

Updated:

import ray
from ray.data.llm import vLLMEngineProcessorConfig, build_processor

config = vLLMEngineProcessorConfig(
    model_source="meta-llama/Meta-Llama-3.1-8B-Instruct",
    concurrency=1,
    batch_size=64,
)

processor = build_processor( # This is the only difference. Arguments remain the same.
    config,
    preprocess=lambda row: dict(
        messages=[{"role": "user", "content": row["prompt"]}],
        sampling_params=dict(temperature=0.3, max_tokens=20),
    ),
    postprocess=lambda row: dict(resp=row["generated_text"]),
    preprocess_map_kwargs={"num_cpus": 0.5},
    postprocess_map_kwargs={"num_cpus": 0.25},
)

ds = ray.data.range(300)
ds = processor(ds)
for row in ds.take_all():
    print(row)

[jeffreyjeffreywang]

cc: @nrghosh @kouroshHakha

[gemini-code-assist]

Code Review

This pull request refactors build_llm_processor to the more generic build_processor. This is a good change that makes the API more flexible for various processing workloads beyond just LLM inference. The renaming has been applied consistently and thoroughly across the entire codebase, including source files, documentation, examples, and tests. The changes are well-executed and I found no issues in my review.

[jeffreyjeffreywang]

Will wait for #58298 to finalize before deciding if this PR is still necessary.

[nrghosh]

conflicts resolved, good for merge

[jeffreyjeffreywang]

Rebase to address failing doc tests.

[nrghosh]

cc @kouroshHakha ready to merge 🚀

[kouroshHakha]

needs some changes

[kouroshHakha]

wait this should be a decorator and we should also remove the PublicAPI annotation from build_llm_processor

[jeffreywang-anyscale]

I see similar usages of directly using the deprecation_warning helper: https://github.com/search?q=repo%3Aray-project%2Fray+%22deprecation_warning%28%22&type=code&p=2. I'm not able to find associated decorators though.

[jeffreywang-anyscale]

Will remove the publicAPI annotation.

[jeffreywang-anyscale]

Done.

[kouroshHakha]

oh I guess I was talking about Deprecated class:

ray/python/ray/_common/deprecation.py

Line 61 in b8f22e9

"""Decorator for documenting a deprecated class, method, or function.

Maybe we can use that instead?

[jeffreywang-anyscale]

ah yeah, this is neater. adjusted in my latest revision.

[alexeykudinkin]

Rubber-stamping

[jeffreywang-anyscale]

Rebasing onto latest master