Try to run the basic sample

[allanchamon]

I am trying to run basic sample and the error has been reported:
(.venv) PS C:\Users\allan\Documents\promptbench> & c:/Users/allan/Documents/promptbench/.venv/Scripts/python.exe c:/Users/allan/Documents/promptbench/examples/basic.py
Traceback (most recent call last):
File "c:\Users\allan\Documents\promptbench\examples\basic.py", line 5, in
import promptbench as pb
File "C:\Users\allan\Documents\promptbench\promptbench_init_.py", line 6, in
from .dataload import DatasetLoader, SUPPORTED_DATASETS, SUPPORTED_DATASETS_VLM
File "C:\Users\allan\Documents\promptbench\promptbench\dataload_init_.py", line 4, in
from .dataset import *
File "C:\Users\allan\Documents\promptbench\promptbench\dataload\dataset.py", line 13, in
from datasets import load_dataset
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\datasets_init_.py", line 18, in
from .arrow_dataset import Dataset
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\datasets\arrow_dataset.py", line 76, in
from .arrow_reader import ArrowReader
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\datasets\arrow_reader.py", line 32, in
from .download.download_config import DownloadConfig
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\datasets\download_init_.py", line 9, in
from .download_manager import DownloadManager, DownloadMode
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\datasets\download\download_manager.py", line 35, in
from ..utils.file_utils import (
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\datasets\utils\file_utils.py", line 37, in
from aiohttp.client_exceptions import ClientError
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\aiohttp_init_.py", line 6, in
from .client import (
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\aiohttp\client.py", line 85, in
from .connector import (
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\aiohttp\connector.py", line 34, in
import aiohappyeyeballs
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\aiohappyeyeballs_init_.py", line 3, in
from .impl import start_connection
File "C:\Users\allan\Documents\promptbench.venv\lib\site-packages\aiohappyeyeballs\impl.py", line 22, in
socket_factory: Optional[SocketFactoryType] = None,
File "C:\Python39\lib\typing.py", line 243, in inner
return func(*args, **kwds)
File "C:\Python39\lib\typing.py", line 316, in getitem
return self._getitem(self, parameters)
File "C:\Python39\lib\typing.py", line 433, in Optional
return Union[arg, type(None)]
File "C:\Python39\lib\typing.py", line 243, in inner
return func(*args, **kwds)
File "C:\Python39\lib\typing.py", line 316, in getitem
return self._getitem(self, parameters)
File "C:\Python39\lib\typing.py", line 421, in Union
parameters = _remove_dups_flatten(parameters)
File "C:\Python39\lib\typing.py", line 215, in _remove_dups_flatten
all_params = set(params)
TypeError: unhashable type: 'list'

Any help about that? Thanks in advance.

[kaijiezhu11]

Hi, can you share the code you are running with me?

I tested the following code, it worked well.

import promptbench as pb

dataset = pb.DatasetLoader.load_dataset("sst2")
print(dataset)

BTW, can you please check if the datasets package has been successfully installed?

[allanchamon]

The code is a python source copied from the notebook "basic.ipynb" and is following attached. Thanks. Em seg., 17 de mar. de 2025 às 17:03, Kaijie Zhu ***@***.***> escreveu:

…

Hi, can you share the code you are running with me? I tested the following code, it worked well. import promptbench as pb dataset = pb.DatasetLoader.load_dataset("sst2") print(dataset) BTW, can you please check if the datasets package has been successfully installed? — Reply to this email directly, view it on GitHub <#94 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE> . You are receiving this because you authored the thread.Message ID: ***@***.***> [image: Immortalise]*Immortalise* left a comment (microsoft/promptbench#94) <#94 (comment)> Hi, can you share the code you are running with me? I tested the following code, it worked well. import promptbench as pb dataset = pb.DatasetLoader.load_dataset("sst2") print(dataset) BTW, can you please check if the datasets package has been successfully installed? — Reply to this email directly, view it on GitHub <#94 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[allanchamon]

May I ask for more help? I am trying to load a pre-trained model locally, it is a derivation of llama 3.1, can I load this model locally and make some attacks against it? BTW! Em qui., 20 de mar. de 2025 às 12:41, Allan Chamon Figueiredo < ***@***.***> escreveu:

…

The code is a python source copied from the notebook "basic.ipynb" and is following attached. Thanks. Em seg., 17 de mar. de 2025 às 17:03, Kaijie Zhu ***@***.***> escreveu: > Hi, can you share the code you are running with me? > > I tested the following code, it worked well. > > import promptbench as pb > > dataset = pb.DatasetLoader.load_dataset("sst2") > print(dataset) > > BTW, can you please check if the datasets package has been successfully > installed? > > — > Reply to this email directly, view it on GitHub > <#94 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > [image: Immortalise]*Immortalise* left a comment > (microsoft/promptbench#94) > <#94 (comment)> > > Hi, can you share the code you are running with me? > > I tested the following code, it worked well. > > import promptbench as pb > > dataset = pb.DatasetLoader.load_dataset("sst2") > print(dataset) > > BTW, can you please check if the datasets package has been successfully > installed? > > — > Reply to this email directly, view it on GitHub > <#94 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A7KQ4JWU33OHEPJJOW6RDVL2U4TCBAVCNFSM6AAAAABYSVCNZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQG4YDOMRRHE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[allanchamon]

I am using datasets package version 2.15.0, I´ve tried to use the 2.19.2 as well, the same erros happens when I try to execute the code snippet you suggested:

`import promptbench as pb

dataset = pb.DatasetLoader.load_dataset("sst2")
print(dataset)`

TypeError: unhashable type: 'list'

[allanchamon]

Solved! I have changed the version of packages below in order to create the conda enviroment correctly:
sentencepiece==0.1.99->0.2.0
tokenizers==0.15.0->0.15.1

I have the packages below so that I could use CUDA support:
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

[github-actions]

Stale issue message