Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Commit

Permalink
Moves vision models to allennlp-models (#4918)
Browse files Browse the repository at this point in the history
* Moves vision models to allennlp-models

* Also move test fixtures

* Don't return so many instances if we're cutting them out later anyways

* We actually need this image

* Formatting

* Fixing more paths
  • Loading branch information
dirkgr committed Jan 21, 2021
1 parent 412896b commit 9a4a424
Show file tree
Hide file tree
Showing 51 changed files with 9 additions and 3,940 deletions.
4 changes: 0 additions & 4 deletions allennlp/data/dataset_readers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,3 @@
from allennlp.data.dataset_readers.sequence_tagging import SequenceTaggingDatasetReader
from allennlp.data.dataset_readers.sharded_dataset_reader import ShardedDatasetReader
from allennlp.data.dataset_readers.text_classification_json import TextClassificationJsonReader
from allennlp.data.dataset_readers.vision_reader import VisionReader
from allennlp.data.dataset_readers.vqav2 import VQAv2Reader
from allennlp.data.dataset_readers.visual_entailment import VisualEntailmentReader
from allennlp.data.dataset_readers.gqa import GQAReader
6 changes: 6 additions & 0 deletions allennlp/data/dataset_readers/dataset_reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,12 @@ def shard_iterable(self, iterable: Iterable[_T]) -> Iterator[_T]:
sharded_slice, self._worker_info.id, None, self._worker_info.num_workers
)

# We don't know for sure how many instances we have to produce.
# _multi_worker_islice() figures that out. But we know for sure
# it won't be more than max_instances.
if self.max_instances is not None:
sharded_slice = itertools.islice(sharded_slice, self.max_instances)

return sharded_slice

def _multi_worker_islice(
Expand Down
213 changes: 0 additions & 213 deletions allennlp/data/dataset_readers/gqa.py

This file was deleted.

0 comments on commit 9a4a424

Please sign in to comment.