Custom pipeline docs are not accurate

[T145]

Describe the bug

The description reads as such:

custom_pipeline (`str`, *optional*):

                <Tip warning={true}>

                🧪 This is an experimental feature and may change in the future.

                </Tip>

                Can be either:

                    - A string, the *repo id* (for example `hf-internal-testing/diffusers-dummy-pipeline`) of a custom
                      pipeline hosted on the Hub. The repository must contain a file called pipeline.py that defines
                      the custom pipeline.
                    - A string, the *file name* of a community pipeline hosted on GitHub under
                      [Community](https://github.com/huggingface/diffusers/tree/main/examples/community). Valid file
                      names must match the file name and not the pipeline script (`clip_guided_stable_diffusion`
                      instead of `clip_guided_stable_diffusion.py`). Community pipelines are always loaded from the
                      current main branch of GitHub.
                    - A path to a directory (`./my_pipeline_directory/`) containing a custom pipeline. The directory
                      must contain a file called `pipeline.py` that defines the custom pipeline.

                For more information on how to load and create custom pipelines, please have a look at [Loading and
                Adding Custom
                Pipelines](https://huggingface.co/docs/diffusers/using-diffusers/custom_pipeline_overview)

It mentions that "community pipelines are always loaded from the current main branch of GitHub," however this is not true. Note that in the logs the branch it tries to fetch from is v0.25.0. It's easy to work around by doing a manual HF Hub download and passing the physical path to the pipeline, however it would be nice to have things work as described.

Reproduction

Try to load a custom pipeline that doesn't exist.

Logs

Could not locate the pipeline.py inside test.
Traceback (most recent call last):
  File "-\huggingface_hub\utils\_errors.py", line 286, in hf_raise_for_status
    response.raise_for_status()
  File "-\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://raw.githubusercontent.com/huggingface/diffusers/v0.25.0/examples/community/test.py

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "-\paint.py", line 199, in <module>
    generate_image(prompt, neg_prompt, steps=60, seed=1954469072)
  File "-\paint.py", line 85, in generate_image
    pipe = StableDiffusionXLPipeline.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "-\diffusers\pipelines\pipeline_utils.py", line 1096, in from_pretrained
    cached_folder = cls.download(
                    ^^^^^^^^^^^^^
  File "-\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "-\diffusers\pipelines\pipeline_utils.py", line 1768, in download
    pipeline_class = _get_pipeline_class(
                     ^^^^^^^^^^^^^^^^^^^^
  File "-\diffusers\pipelines\pipeline_utils.py", line 380, in _get_pipeline_class
    return get_class_from_dynamic_module(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "-\diffusers\utils\dynamic_modules_utils.py", line 441, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "-\diffusers\utils\dynamic_modules_utils.py", line 287, in get_cached_module_file
    resolved_module_file = cached_download(
                           ^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\file_download.py", line 683, in cached_download
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\file_download.py", line 385, in _request_wrapper
    response = _request_wrapper(
               ^^^^^^^^^^^^^^^^^
  File "-\huggingface_hub\file_download.py", line 409, in _request_wrapper
    hf_raise_for_status(response)
  File "-\huggingface_hub\utils\_errors.py", line 333, in hf_raise_for_status
    raise HfHubHTTPError(str(e), response=response) from e
huggingface_hub.utils._errors.HfHubHTTPError: 404 Client Error: Not Found for url: https://raw.githubusercontent.com/huggingface/diffusers/v0.25.0/examples/community/test.py

System Info

channels:
  - pytorch
  - nvidia
  - conda-forge
  - defaults
dependencies:
  - accelerate=0.25.0=pyhd8ed1ab_0
  - aiohttp=3.9.1=py311ha68e1ae_0 
  - aiosignal=1.3.1=pyhd8ed1ab_0  
  - attrs=23.1.0=pyh71513ae_1
  - aws-c-auth=0.7.8=h6bf0135_3
  - aws-c-cal=0.6.9=h7f0e5be_2
  - aws-c-common=0.9.10=hcfcfb64_0
  - aws-c-compression=0.2.17=h7f0e5be_7
  - aws-c-event-stream=0.3.2=h51e6447_8
  - aws-c-http=0.7.15=h80119a0_0
  - aws-c-io=0.13.36=ha737126_3
  - aws-c-mqtt=0.10.0=h9e0d744_1
  - aws-c-s3=0.4.6=he94bf34_0
  - aws-c-sdkutils=0.1.13=h7f0e5be_0
  - aws-checksums=0.1.17=h7f0e5be_6
  - aws-crt-cpp=0.25.0=h1554317_4
  - aws-sdk-cpp=1.11.210=h0441c79_5
  - blas=2.120=mkl
  - blas-devel=3.9.0=20_win64_mkl
  - brotli-python=1.1.0=py311h12c1d0e_1
  - bzip2=1.0.8=hcfcfb64_5
  - c-ares=1.24.0=hcfcfb64_0
  - ca-certificates=2023.12.12=haa95532_0
  - certifi=2023.11.17=py311haa95532_0
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - colorama=0.4.6=pyhd8ed1ab_0
  - cuda-cccl=12.3.101=0
  - cuda-cudart=12.1.105=0
  - cuda-cudart-dev=12.1.105=0
  - cuda-cupti=12.1.105=0
  - cuda-libraries=12.1.0=0
  - cuda-libraries-dev=12.1.0=0
  - cuda-nvrtc=12.1.105=0
  - cuda-nvrtc-dev=12.1.105=0
  - cuda-nvtx=12.1.105=0
  - cuda-opencl=12.3.101=0
  - cuda-opencl-dev=12.3.101=0
  - cuda-profiler-api=12.3.101=0
  - cuda-runtime=12.1.0=0
  - datasets=2.16.0=pyhd8ed1ab_0
  - diffusers=0.25.0=pyhd8ed1ab_0
  - dill=0.3.7=pyhd8ed1ab_0
  - filelock=3.13.1=pyhd8ed1ab_0
  - freetype=2.12.1=hdaf720e_2
  - frozenlist=1.4.1=py311ha68e1ae_0
  - fsspec=2023.10.0=pyhca7485f_0
  - huggingface_hub=0.20.0=pyhd8ed1ab_0
  - idna=3.6=pyhd8ed1ab_0
  - importlib-metadata=7.0.1=pyha770c72_0
  - intel-openmp=2023.2.0=h57928b3_50497
  - jinja2=3.1.2=pyhd8ed1ab_1
  - krb5=1.21.2=heb0366b_0
  - lcms2=2.16=h67d730c_0
  - lerc=4.0.0=h63175ca_0
  - libabseil=20230802.1=cxx17_h63175ca_0
  - libarrow=14.0.2=h1048771_0_cpu
  - libarrow-acero=14.0.2=h63175ca_0_cpu
  - libarrow-dataset=14.0.2=h63175ca_0_cpu
  - libarrow-flight=14.0.2=h53b1db0_0_cpu
  - libarrow-flight-sql=14.0.2=h78eab7c_0_cpu
  - libarrow-gandiva=14.0.2=hb2eaab1_0_cpu
  - libarrow-substrait=14.0.2=hd4c9904_0_cpu
  - libblas=3.9.0=20_win64_mkl
  - libbrotlicommon=1.1.0=hcfcfb64_1
  - libbrotlidec=1.1.0=hcfcfb64_1
  - libbrotlienc=1.1.0=hcfcfb64_1
  - libcblas=3.9.0=20_win64_mkl
  - libcrc32c=1.1.2=h0e60522_0
  - libcublas=12.1.0.26=0
  - libcublas-dev=12.1.0.26=0
  - libcufft=11.0.2.4=0
  - libcufft-dev=11.0.2.4=0
  - libcurand=10.3.4.101=0
  - libcurand-dev=10.3.4.101=0
  - libcurl=8.5.0=hd5e4a3a_0
  - libcusolver=11.4.4.55=0
  - libcusolver-dev=11.4.4.55=0
  - libcusparse=12.0.2.55=0
  - libcusparse-dev=12.0.2.55=0
  - libdeflate=1.19=hcfcfb64_0
  - libevent=2.1.12=h3671451_1
  - libexpat=2.5.0=h63175ca_1
  - libffi=3.4.2=h8ffe710_5
  - libgoogle-cloud=2.12.0=h39f2fc6_4
  - libgrpc=1.59.3=h5bbd4a7_0
  - libhwloc=2.9.3=default_haede6df_1009
  - libiconv=1.17=hcfcfb64_2
  - libjpeg-turbo=3.0.0=hcfcfb64_1
  - liblapack=3.9.0=20_win64_mkl
  - liblapacke=3.9.0=20_win64_mkl
  - libnpp=12.0.2.50=0
  - libnpp-dev=12.0.2.50=0
  - libnvjitlink=12.1.105=0
  - libnvjitlink-dev=12.1.105=0
  - libnvjpeg=12.1.1.14=0
  - libnvjpeg-dev=12.1.1.14=0
  - libparquet=14.0.2=h7ec3a38_0_cpu
  - libpng=1.6.39=h19919ed_0
  - libprotobuf=4.24.4=hb8276f3_0
  - libre2-11=2023.06.02=h8c5ae5e_0
  - libsqlite=3.44.2=hcfcfb64_0
  - libssh2=1.11.0=h7dfc565_0
  - libthrift=0.19.0=ha2b3283_1
  - libtiff=4.6.0=h6e2ebb7_2
  - libutf8proc=2.8.0=h82a8f57_0
  - libuv=1.44.2=hcfcfb64_1
  - libwebp-base=1.3.2=hcfcfb64_0
  - libxcb=1.15=hcd874cb_0
  - libxml2=2.11.6=hc3477c8_0
  - libzlib=1.2.13=hcfcfb64_5
  - lz4-c=1.9.4=hcfcfb64_0
  - m2w64-gcc-libgfortran=5.3.0=6
  - m2w64-gcc-libs=5.3.0=7
  - m2w64-gcc-libs-core=5.3.0=7
  - m2w64-gmp=6.1.0=2
  - m2w64-libwinpthread-git=5.0.0.4634.697f757=2
  - markupsafe=2.1.3=py311ha68e1ae_1
  - mkl=2023.2.0=h6a75c08_50497
  - mkl-devel=2023.2.0=h57928b3_50497
  - mkl-include=2023.2.0=h6a75c08_50497
  - mpmath=1.3.0=pyhd8ed1ab_0
  - msys2-conda-epoch=20160418=1
  - multidict=6.0.4=py311ha68e1ae_1
  - multiprocess=0.70.15=py311ha68e1ae_1
  - networkx=3.2.1=pyhd8ed1ab_0
  - numpy=1.26.2=py311h0b4df5a_0
  - openjpeg=2.5.0=h3d672ee_3
  - openssl=3.2.0=hcfcfb64_1
  - orc=1.9.2=hf0b6bd4_0
  - packaging=23.2=pyhd8ed1ab_0
  - pandas=2.1.4=py311hf63dbb6_0
  - peft=0.7.1=pyhd8ed1ab_0
  - pillow=10.1.0=py311h4dd8a23_0
  - pip=23.3.2=pyhd8ed1ab_0
  - psutil=5.9.7=py311ha68e1ae_0
  - pthread-stubs=0.4=hcd874cb_1001
  - pthreads-win32=2.9.1=hfa6e2cd_3
  - pyarrow=14.0.2=py311h6a6099b_0_cpu
  - pyarrow-hotfix=0.6=pyhd8ed1ab_0
  - pysocks=1.7.1=pyh0701188_6
  - python=3.11.7=h2628c8c_1_cpython
  - python-dateutil=2.8.2=pyhd8ed1ab_0
  - python-slugify=8.0.1=pyhd8ed1ab_2
  - python-tzdata=2023.3=pyhd8ed1ab_0
  - python-xxhash=3.4.1=py311ha68e1ae_0
  - python_abi=3.11=4_cp311
  - pytorch=2.1.2=py3.11_cuda12.1_cudnn8_0
  - pytorch-cuda=12.1=hde6ce7c_5
  - pytorch-mutex=1.0=cuda
  - pytz=2023.3.post1=pyhd8ed1ab_0
  - pyyaml=6.0.1=py311ha68e1ae_1
  - re2=2023.06.02=hcbb65ff_0
  - regex=2023.12.25=py311ha68e1ae_0
  - requests=2.31.0=pyhd8ed1ab_0
  - safetensors=0.3.3=py311hc37eb10_1
  - scipy=1.11.4=py311h0b4df5a_0
  - setuptools=68.2.2=pyhd8ed1ab_0
  - six=1.16.0=pyh6c4a22f_0
  - snappy=1.1.10=hfb803bf_0
  - sympy=1.12=pyh04b8f61_3
  - tbb=2021.11.0=h91493d7_0
  - text-unidecode=1.3=pyhd8ed1ab_1
  - tk=8.6.13=h5226925_1
  - tokenizers=0.15.0=py311h91c4a10_1
  - tqdm=4.66.1=pyhd8ed1ab_0
  - transformers=4.36.2=pyhd8ed1ab_0
  - typing-extensions=4.9.0=hd8ed1ab_0
  - typing_extensions=4.9.0=pyha770c72_0
  - tzdata=2023d=h0c530f3_0
  - ucrt=10.0.22621.0=h57928b3_0
  - urllib3=2.1.0=pyhd8ed1ab_0
  - vc=14.3=hcf57466_18
  - vc14_runtime=14.38.33130=h82b7239_18
  - vs2015_runtime=14.38.33130=hcb4865c_18
  - wheel=0.42.0=pyhd8ed1ab_0
  - win_inet_pton=1.1.0=pyhd8ed1ab_6
  - xorg-libxau=1.0.11=hcd874cb_0
  - xorg-libxdmcp=1.1.3=hcd874cb_0
  - xxhash=0.8.2=hcfcfb64_0
  - xz=5.2.6=h8d14728_0
  - yaml=0.2.5=h8ffe710_2
  - yarl=1.9.3=py311ha68e1ae_0
  - zipp=3.17.0=pyhd8ed1ab_0
  - zstd=1.5.5=h12be248_0
  - pip:
      - compel==2.0.2
      - pyparsing==3.1.1
      - torchao==0.0.1
      - torchaudio==2.1.2
      - torchvision==0.16.2

Who can help?

@sayakpaul @patrickvonplaten

[sayakpaul]

Cc: @patrickvonplaten

[patrickvonplaten]

Hey @T145,

You are right we should change the docs here to state that community pipelines are always loaded from the latest diffusers version. Would you like to open a PR to change it?

Note: You can also just pass custom_revision="main" and it should work

[T145]

I tried that and it didn't work. My present workaround is:

pipeline = cached_download(
	url='https://raw.githubusercontent.com/huggingface/diffusers/main/examples/community/lpw_stable_diffusion_xl.py',
	cache_dir='data',
	force_filename='lpw_stable_diffusion_xl.py'
)

Also when specifying a physical path to a pipeline whether the '.py' extension is present shouldn't matter, but custom_pipeline will not function w/out it. I can change the docs but the real problems need code changes to fix.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[yiyixuxu]

fixing here #6929