Error out of default_collate for lists of unequal size

[choidongyeon]

Fix issue #23141

In the below example default_collate collates each element of the list. Since the second element isn't present in all samples, it is discarded:

from torch.utils.data import Dataset
from torch.utils.data import DataLoader
import numpy as np

class CustomDataset(Dataset):
    def __len__(self):
        return 2

    def __getitem__(self, idx):
        tmp = {
            "foo": np.array([1, 2, 3]),
            "bar": ["X"] * (idx+1),
        }

        return tmp

training = CustomDataset()

for batch in DataLoader(training, batch_size=2):
    print(batch)

Yields

{
  'foo': tensor(
    [
      [1, 2, 3],
      [1, 2, 3]
    ]
  ),
  'bar': [
      ('X', 'X'),
    ]
}

Based on discussion in the issue, it seems the best course of action is to error out in this case. This seems consistent with what is done for tensor elements, as seen in TensorShape.cpp line 1066 which is called when torch.stack is called. In this PR, I introduce a similar message to error out for lists.

@ssnl

[dr-ci]

💊 CI failures summary and remediations

As of commit d30f02b (more details on the Dr. CI page):

---

• 2/2 failures introduced in this PR

---

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_xla_linux_bionic_py3_6_clang9_build (1/2)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (add/add): Merge conflict in .circleci/ecr_gc_docker/gc.py 
Auto-merging .circleci/ecr_gc_docker/gc.py 
CONFLICT (add/add): Merge conflict in .circleci/config.yml 
Auto-merging .circleci/config.yml 
CONFLICT (add/add): Merge conflict in .circleci/codegen_validation/compare_normalized_yaml.sh 
Auto-merging .circleci/codegen_validation/compare_normalized_yaml.sh 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/lib/miniyaml.py 
Auto-merging .circleci/cimodel/lib/miniyaml.py 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/windows_build_definitions.py 
Auto-merging .circleci/cimodel/data/windows_build_definitions.py 
Automatic merge failed; fix conflicts and then commit the result. 

pytorch_linux_xenial_py3_6_gcc5_4_build (2/2)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (add/add): Merge conflict in .circleci/ecr_gc_docker/gc.py 
Auto-merging .circleci/ecr_gc_docker/gc.py 
CONFLICT (add/add): Merge conflict in .circleci/config.yml 
Auto-merging .circleci/config.yml 
CONFLICT (add/add): Merge conflict in .circleci/codegen_validation/compare_normalized_yaml.sh 
Auto-merging .circleci/codegen_validation/compare_normalized_yaml.sh 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/lib/miniyaml.py 
Auto-merging .circleci/cimodel/lib/miniyaml.py 
CONFLICT (add/add): Merge conflict in .circleci/cimodel/data/windows_build_definitions.py 
Auto-merging .circleci/cimodel/data/windows_build_definitions.py 
Automatic merge failed; fix conflicts and then commit the result. 

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 6 times.

[ssnl]

could you add a test in test_dataloader.py too? something that asserts default_collate raises is fine!

[ssnl]

Thank you :)

[ssnl]

@pytorchbot merge this please

[choidongyeon]

Thanks for taking a look @ssnl!

[facebook-github-bot]

@ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@ezyang merged this pull request in 8c07a98.