Unit tests require internet access

[adamjstewart]

In #185, @ashnair1 reported that our CI was failing because the trainer tests require internet access:

=================================== FAILURES ===================================
_____________________ TestLandCoverAITrainer.test_unet_ce ______________________

self = <tests.trainers.test_landcoverai.TestLandCoverAITrainer object at 0x1427c0160>
default_config = {'encoder_name': 'resnet18', 'encoder_output_stride': 16, 'encoder_weights': 'imagenet', 'learning_rate': 0.001, ...}

    def test_unet_ce(self, default_config: Dict[str, Any]) -> None:
        default_config["segmentation_model"] = "unet"
        default_config["loss"] = "ce"
>       task = LandcoverAISegmentationTask(**default_config)

tests/trainers/test_landcoverai.py:26: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
torchgeo/trainers/landcoverai.py:92: in __init__
    self.config_task()
torchgeo/trainers/landcoverai.py:41: in config_task
    self.model = smp.Unet(
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/segmentation_models_pytorch/unet/model.py:65: in __init__
    self.encoder = get_encoder(
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/segmentation_models_pytorch/encoders/__init__.py:62: in get_encoder
    encoder.load_state_dict(model_zoo.load_url(settings["url"]))
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/torch/hub.py:571: in load_state_dict_from_url
    download_url_to_file(url, cached_file, hash_prefix, progress=progress)
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/torch/hub.py:459: in download_url_to_file
    buffer = u.read(8192)
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/http/client.py:459: in read
    n = self.readinto(b)
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/http/client.py:503: in readinto
    n = self.fp.readinto(b)
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/socket.py:669: in readinto
    return self._sock.recv_into(b)
../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/ssl.py:1241: in recv_into
    return self.read(nbytes, buffer)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket [closed] fd=60, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>
len = 7704, buffer = <memory at 0x1423dcdc0>

    def read(self, len=1024, buffer=None):
        """Read up to LEN bytes and return them.
        Return zero-length string on EOF."""
    
        self._checkClosed()
        if self._sslobj is None:
            raise ValueError("Read on closed or unwrapped SSL socket.")
        try:
            if buffer is not None:
>               return self._sslobj.read(len, buffer)
E               ConnectionResetError: [Errno 54] Connection reset by peer

../../../hostedtoolcache/Python/3.8.12/x64/lib/python3.8/ssl.py:1099: ConnectionResetError
----------------------------- Captured stderr call -----------------------------
Downloading: "https://download.pytorch.org/models/resnet18-5c106cde.pth" to /Users/runner/.cache/torch/hub/checkpoints/resnet18-5c106cde.pth

  0%|          | 0.00/44.7M [00:00<?, ?B/s]
  3%|▎         | 1.49M/44.7M [00:00<00:00, 50.6MB/s]

---------- coverage: platform darwin, python 3.8.12-final-0 ----------
Coverage XML written to file coverage.xml

=========================== short test summary info ============================
FAILED tests/trainers/test_landcoverai.py::TestLandCoverAITrainer::test_unet_ce
=========== 1 failed, 514 passed, 66 deselected in 83.95s (0:01:23) ============
Error: Process completed with exit code 1.

We should see if it's possible to monkeypatch things to skip downloads/pre-trained models for testing.

[calebrob6]

+1 for this, having tests randomly fail is quite confusing -- I don't think we actually need to be testing weight downloads

[adamjstewart]

Our tests are starting to fail more often than not. Failures are usually from failed downloads. For trainers, the solution is to simply add a config option for whether or not to download pretrained models and set it to False (we shouldn't be downloading things by default anyway). For our resnet50 pretrained model we'll need to figure out something else. We could generate it and store it locally so that we can still test loading without having to rely on an internet connection.

Anyone want to work on this?

[calebrob6]

For trainers, the solution is to simply add a config option for whether or not to download pretrained models and set it to False (we shouldn't be downloading things by default anyway)

We can just have the trainers default to not use pretrained weights. Is this what you meant @adamjstewart?

[adamjstewart]

Correct