enable the device_map parameter in huggingface pipeline

[praveenv]

Enabling device_map in HuggingFacePipeline

For multi-gpu settings with large models, the accelerate library provides the device_map parameter to automatically distribute the model across GPUs / disk.

The Transformers pipeline enables users to specify device (or) device_map, and handles cases (with warnings) when both are specified.

However, Langchain's HuggingFacePipeline only supports specifying device when calling transformers which limits large models and multi-gpu use-cases.
Additionally, the default value of device is initialized to -1 , which is incompatible with the transformers pipeline when device_map is specified.

This PR addresses the addition of device_map as a parameter , and solves the incompatibility of device = -1 when device_map is also specified.
An additional test has been added for this feature.

Additionally, some existing tests no longer work since

1. max_new_tokens has to be specified under pipeline_kwargs and not model_kwargs
2. The GPT2 tokenizer raises a ValueError: Pipeline with tokenizer without pad_token cannot do batching, since the tokenizer.pad_token is None (related issue on the transformers repo).

This PR handles fixing these tests as well.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Nov 1, 2023 7:40pm

[baskaryan]

thanks @praveenv!

[importTthis]

Sorry to bother you，why do I use device_map but only use CPU for inference? I have two GPUs and have installed accelerate.
code:

from langchain.llms.huggingface_pipeline import HuggingFacePipeline


template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)


gpus_llm = HuggingFacePipeline.from_model_id(
    model_id="/mnt/chinese-alpaca-7b/",
    task="text-generation",
    device_map="auto",  # replace with device_map="auto" to use the accelerate library.
    pipeline_kwargs={"max_new_tokens": 4000}
)

gpu_chain = prompt | gpus_llm

question = "What is electroencephalography?"

print(gpu_chain.invoke({"question": question}))

accelerate env:

- `Accelerate` version: 0.25.0
- Platform: Linux-5.4.0-153-generic-x86_64-with-glibc2.31
- Python version: 3.10.13
- Numpy version: 1.26.2
- PyTorch version (GPU?): 1.12.1 (True)
- PyTorch XPU available: False
- PyTorch NPU available: False
- System RAM: 94.30 GB
- GPU type: Quadro RTX 5000
- `Accelerate` default config:
   - compute_environment: LOCAL_MACHINE
   - distributed_type: MULTI_GPU
   - mixed_precision: fp16
   - use_cpu: False
   - debug: False
   - num_processes: 2
   - machine_rank: 0
   - num_machines: 1
   - rdzv_backend: static
   - same_network: False
   - main_training_function: main
   - downcast_bf16: False
   - tpu_use_cluster: False
   - tpu_use_sudo: False