avoid crashing when a run cannot be sync (issue 1297)

[cdancette]

Description

This is a workaround for issue #1297.

It will just skip a run if it failed to open the wand log file.

Before I applied this, the following error happened, when a run was either not done, or when a run crashed before being done

Traceback (most recent call last):
  File "/gpfswork/rech/dur/uzb95vd/envs/murel/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/gpfswork/rech/dur/uzb95vd/envs/murel/lib/python3.7/site-packages/wandb/sync/sync.py", line 115, in run
    ds.open_for_scan(sync_item)
  File "/gpfswork/rech/dur/uzb95vd/envs/murel/lib/python3.7/site-packages/wandb/sdk/internal/datastore.py", line 100, in open_for_scan
    self._read_header()
  File "/gpfswork/rech/dur/uzb95vd/envs/murel/lib/python3.7/site-packages/wandb/sdk/internal/datastore.py", line 170, in _read_header
    ident, magic, version = struct.unpack("<4sHB", header)
struct.error: unpack requires a buffer of 7 bytes

[cvphelps]

Hi Corentin, thanks for submitting a PR around this issue! My colleague Jeff will take a look to see if we can fit this into one of the next couple of client library releases.

[cdancette]

Hi, thanks for looking into this. As I mentioned in the issue, it would be better to fix directly the underlying problem (being able to sync continuously logs which are written on disk). I use this workaround for now on my local copy. Le ven. 12 févr. 2021 à 18:47, Carey Phelps <notifications@github.com> a écrit :

…

Hi Corentin, thanks for submitting a PR around this issue! My colleague Jeff will take a look to see if we can fit this into one of the next couple of client library releases. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACQPYN253DGL3BCHLPJWHMTS6VSUBANCNFSM4XBMUFTQ> .

[github-actions]

This PR is stale because it has been open 60 days with no activity.