[fix] Fixed issues in reconnecting to device after firmware upgrade #235

[nemesifier]

The test is misssing.

Fixes #235

This may fix also #233 (needs manual testing to confirm).

[pandafy]

While performing manual testing locally, I got this log for the device

Connection successful, starting upgrade...
Image checksum file not found, proceeding with the upload of the new image...
Sysupgrade test passed successfully, proceeding with the upgrade operation...
Upgrade operation in progress...
Commencing upgrade. Closing all shell sessions.
Image metadata not found
Reading partition table from bootdisk...
Reading partition table from image...
Partition layout has changed. Full image will be written.

SSH connection closed, will wait 150 seconds before attempting to reconnect...
Device not reachable yet, (connection failed).
retrying in 30 seconds...
Device not reachable yet, (connection failed).
retrying in 30 seconds...
Trying to reconnect to device at 192.168.56.2 (attempt n.3)...
Connected! Writing checksum file to /etc/openwisp/firmware_checksum
Upgrade completed successfully.

See Trying to reconnect to device at 192.168.56.2 (attempt n.3)... shows third attempt. This is because in previous two attempts the device was unreachable and self._refresh_addresses() raised NoWorkingDeviceConnectionError which led to the logs getting skipped.

How can we get updated addresses without initiating a connection?

[nemesifier]

what about:

```python
def _log(addresses, attempt):
    self.log(
        _(
            f'Trying to reconnect to device at {addresses} '
            '(attempt n.{attempt})...'.format(
                addresses=addresses,
                attempt=attempt
            )
        ),
        save=False,
    )
try:
    self._refresh_addresses()
except NoWorkingDeviceConnectionError as error:
    _log(error.connection.addresses, attempt)
try:
    _log(', '.join(self.addresses), attempt)
    self.connect()
except (NoValidConnectionsError, socket.timeout, SSHException) as error:
    # etc...

[pandafy]

I also tested #233 locally and this code mitigates that problem.

[nemesifier]

shouldn't we update this too?

[pandafy]

I added the following wrapper methods so we don't need to reference the connection object again and again

openwisp-firmware-upgrader/openwisp_firmware_upgrader/upgraders/openwrt.py

Lines 123 to 130 in 82ade68

	def connect(self):
	return self.connection.connect()
	def disconnect(self):
	return self.connection.disconnect()
	def exec_command(self, *args, **kwargs):
	return self.connection.connector_instance.exec_command(*args, **kwargs)

[nemesifier]

what about:

```python
def _log(addresses, attempt):
    self.log(
        _(
            f'Trying to reconnect to device at {addresses} '
            '(attempt n.{attempt})...'.format(
                addresses=addresses,
                attempt=attempt
            )
        ),
        save=False,
    )
try:
    self._refresh_addresses()
except NoWorkingDeviceConnectionError as error:
    _log(error.connection.addresses, attempt)
try:
    _log(', '.join(self.addresses), attempt)
    self.connect()
except (NoValidConnectionsError, socket.timeout, SSHException) as error:
    # etc...

[pandafy]

We do not need to perform the connect operation here. self.connection.get_working_connection will open the SSH connection.

[pandafy]

We no longer need to catch these exceptions here. These are handled by DeviceConnection model

https://github.com/openwisp/openwisp-controller/blob/6adafe3d84763c9e942ccceb6f4c006157e50208/openwisp_controller/connection/base/models.py#L345-L357

[pandafy]

@nemesisdesign since we are not directly backporting these changes to 1.0 branch, we can make this change of not making upgraders inherit SSH connectors.