TypeError: str expected, not bool when setting HF_DATASETS_TRUST_REMOTE_CODE environment variable

[kostum123]

Description of the issue:

When executing the lm_eval command with specific model arguments, a TypeError is encountered. This issue arises due to the assignment of a boolean value to the HF_DATASETS_TRUST_REMOTE_CODE environment variable. Environment variables are expected to be strings, but the current implementation assigns a boolean, resulting in the error: TypeError: str expected, not bool.

Steps to Reproduce:

1. Run the lm_eval command with model arguments that necessitate setting the HF_DATASETS_TRUST_REMOTE_CODE environment variable.
2. Observe the TypeError in the command output.

Expected Behavior:

The HF_DATASETS_TRUST_REMOTE_CODE environment variable should be set correctly as a string, allowing the lm_eval command to run without errors.

Actual Behavior:

A TypeError is thrown due to the incorrect assignment of a boolean value to the environment variable.

Proposed Changes:

Modify the assignment of the HF_DATASETS_TRUST_REMOTE_CODE environment variable to convert the boolean value to a string. This can be done by explicitly using the str() function for conversion.

Affected File: lm_eval/__main__.py

Suggested Modification:
Replace the current code for setting HF_DATASETS_TRUST_REMOTE_CODE with:

# Respect user's value passed in via CLI, otherwise default to True and add to comma-separated model args
if args.trust_remote_code is not None:
    os.environ["HF_DATASETS_TRUST_REMOTE_CODE"] = str(args.trust_remote_code)
    args.model_args = (
        args.model_args
        + f",trust_remote_code={os.environ['HF_DATASETS_TRUST_REMOTE_CODE']}"
    )

This change ensures that the environment variable is always set as a string, thus preventing the TypeError.

Additional Context:

This issue was encountered in a Colab environment. However, it's likely to affect other environments where the lm_eval command is used.

Error Logs:

2024-03-04 01:03:23.315122: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-03-04 01:03:23.315172: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-03-04 01:03:23.316977: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-03-04 01:03:24.449914: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-03-04:01:03:28,375 INFO     [__main__.py:223] Verbosity set to INFO
2024-03-04:01:03:28,375 INFO     [__init__.py:369] lm_eval.tasks.initialize_tasks() is deprecated and no longer necessary. It will be removed in v0.4.2 release. TaskManager will instead be used.
Traceback (most recent call last):
  File "/usr/local/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/content/lm-evaluation-harness/lm_eval/__main__.py", line 301, in cli_evaluate
    os.environ["HF_DATASETS_TRUST_REMOTE_CODE"] = (
  File "/usr/lib/python3.10/os.py", line 685, in __setitem__
    value = self.encodevalue(value)
  File "/usr/lib/python3.10/os.py", line 757, in encode
    raise TypeError("str expected, not %s" % type(value).__name__)
TypeError: str expected, not bool