Skip to content

Add new args in auto-docstring - #47737

Merged
zucchini-nlp merged 16 commits into
huggingface:mainfrom
zucchini-nlp:autodoc-grid-thw
Aug 7, 2026
Merged

Add new args in auto-docstring#47737
zucchini-nlp merged 16 commits into
huggingface:mainfrom
zucchini-nlp:autodoc-grid-thw

Conversation

@zucchini-nlp

@zucchini-nlp zucchini-nlp commented Aug 3, 2026

Copy link
Copy Markdown
Member

CI

What does this PR do?

As per title, add image/video grid thw in auto docs and deletes it from modeling code. Also noticed that labels can be deleted as it is in ModelArgs already. And, allows video processors be auto-doc'ced

IMPORTANT:
From this PR on we will be explicitly unpacking all kwargs in the docstring if we can. For example, trying to inspect LlavaImageProccessor.preprocess.__doc__ will print the following

        Args:
            images (`Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, list[PIL.Image.Image], list[numpy.ndarray], list[torch.Tensor]]`):
                Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If
                passing in images with pixel values between 0 and 1, set `do_rescale=False`.
            do_convert_rgb (`bool`, *kwargs*, *optional*):
                Whether to convert the image to RGB.
            do_resize (`bool`, *kwargs*, *optional*):
                Whether to resize the image.
            size (`Annotated[int | list[int] | tuple[int, ...] | dict[str, int] | None, None]`, *kwargs*):
                Describes the maximum input dimensions to the model.
            default_to_square (`bool`, *kwargs*, *optional*):
                Whether to default to a square image when resizing, if size is an int.
            crop_size (`Annotated[int | list[int] | tuple[int, ...] | dict[str, int] | None, None]`, *kwargs*):
                Size of the output image after applying `center_crop`.
            resample (`Annotated[Union[PILImageResampling, int, NoneType], None]`, *kwargs*):
                Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`. Only
                has an effect if `do_resize` is set to `True`.
            do_rescale (`bool`, *kwargs*, *optional*):
                Whether to rescale the image.
            rescale_factor (`float`, *kwargs*, *optional*):
                Rescale factor to rescale the image by if `do_rescale` is set to `True`.
            do_normalize (`bool`, *kwargs*, *optional*):
                Whether to normalize the image.
            image_mean (`Union[float, list[float], tuple[float, ...]]`, *kwargs*, *optional*):
                Image mean to use for normalization. Only has an effect if `do_normalize` is set to `True`.
            image_std (`Union[float, list[float], tuple[float, ...]]`, *kwargs*, *optional*):
                Image standard deviation to use for normalization. Only has an effect if `do_normalize` is set to
                `True`.
            do_pad (`bool`, *kwargs*, *optional*):
                Whether to pad the image. Padding is done either to the largest size in the batch
                or to a fixed square size per image. The exact padding strategy depends on the model.
            pad_size (`Annotated[int | list[int] | tuple[int, ...] | dict[str, int] | None, None]`, *kwargs*):
                The size in `{"height": int, "width" int}` to pad the images to. Must be larger than any image size
                    provided for preprocessing. If `pad_size` is not provided, images will be padded to the largest
                    height and width in the batch. Applied only when `do_pad=True.`
            do_center_crop (`bool`, *kwargs*, *optional*):
                Whether to center crop the image.
            data_format (`Union[str, ~image_utils.ChannelDimension]`, *kwargs*, *optional*):
                Only `ChannelDimension.FIRST` is supported. Added for compatibility with slow processors.
            input_data_format (`Union[str, ~image_utils.ChannelDimension]`, *kwargs*, *optional*):
                The channel dimension format for the input image. If unset, the channel dimension format is inferred
                from the input image. Can be one of:
                - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format.
                - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format.
                - `"none"` or `ChannelDimension.NONE`: image in (height, width) format.
            device (`Annotated[Union[str, torch.device, NoneType], None]`, *kwargs*):
                The device to process the videos on. If unset, the device is inferred from the input videos.
            return_tensors (`Annotated[str | ~utils.generic.TensorType | None, None]`, *kwargs*):
                Returns stacked tensors if set to `'pt'`, otherwise returns a list of tensors.
            disable_grouping (`bool`, *kwargs*, *optional*):
                Whether to disable grouping of images by size to process them individually and not in batches.
                If None, will be set to True if the images are on CPU, and False otherwise. This choice is based on
                empirical observations, as detailed here: https://github.com/huggingface/transformers/pull/38157
            image_seq_length (`int`, *kwargs*, *optional*):
                The number of image tokens to be used for each image in the input.
                Added for backward compatibility but this should be set as a processor attribute in future models.

        Returns:
            `~image_processing_base.BatchFeature`:
            - **data** (`dict`) -- Dictionary of lists/arrays/tensors returned by the __call__ method ('pixel_values', etc.).
            - **tensor_type** (`Union[None, str, TensorType]`, *optional*) -- You can give a tensor_type here to convert the lists of integers in PyTorch/Numpy Tensors at
              initialization.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Comment on lines -630 to 627
@accepts_precomputed_kwargs(modality="image")
@can_return_tuple
@auto_docstring
def get_image_features(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be copied from parent, and I am seeing it overriden many times. IMO copying makes maintaining easier, maybe we can have an mlinter rule, i dunno

Comment on lines +287 to +293
# Can inherit as they share common set of kwargs
class VideoProcessorArgs(ImageProcessorArgs):
videos = {
"description": """
Video to preprocess. Expects a single or batch of videos with pixel values ranging from 0 to 255. If
passing in videos with pixel values between 0 and 1, set `do_rescale=False`.
""",

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

found an interesting thing. When we print docstring of a random ImageProcessor class with no special kwargs, we get smth like below where kwargs are not unpacked:

images: inputs images
kwargs: any other keyword args you can pass, check ImagesKwargs for more

But if we take processor with custom kwargs (e.g. qwen), then the whole list is unpacked:

images: inputs images
do_resize: if to resize or not
size: a dict of sizes
patch_size: the patch size to use etc

Seems like this was intended looking at the first auto-doc PR, to me it looked weird tho

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh interesting, agree that its a bit weird 馃憖

so two processors can have the same args - but kwargs are documented differently - but which rendering we get depends on how it was implemented. i prefer the unpacked version so users don't have to click through to ImagesKwargs to find out what it takes while Qwen users get everything on the page

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i agree, I can update it here tmrw, it should be only a few lines

@zucchini-nlp
zucchini-nlp requested a review from stevhliu August 5, 2026 14:43

@stevhliu stevhliu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

Comment thread src/transformers/models/qwen2_vl/video_processing_qwen2_vl.py Outdated
Comment thread src/transformers/models/ernie4_5_vl_moe/video_processing_ernie4_5_vl_moe.py Outdated
Comment thread src/transformers/video_processing_utils.py
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: colqwen2, cosmos3_edge, cosmos3_omni, ernie4_5_vl_moe, exaone4_5, gemma4, gemma4_unified, glm46v, glm4v, glm4v_moe, glm_image, glm_ocr, glmga, hunyuan_vl, kimi_k25, minicpmv4_6

@zucchini-nlp
zucchini-nlp requested a review from stevhliu August 6, 2026 13:47
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 31106485028:1
Result: success | Jobs: 16 | Tests: 115,514 | Failures: 0 | Duration: 12h 36m

@zucchini-nlp

Copy link
Copy Markdown
Member Author

@stevhliu stevhliu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice, thanks for improving!

@zucchini-nlp
zucchini-nlp added this pull request to the merge queue Aug 7, 2026
Merged via the queue into huggingface:main with commit e1d50b2 Aug 7, 2026
111 checks passed
@zucchini-nlp
zucchini-nlp deleted the autodoc-grid-thw branch August 7, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants