Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors in hivemind.p2p and hivemind.compression #565

Merged
merged 2 commits into from
Apr 26, 2023
Merged

Conversation

borzunov
Copy link
Member

@borzunov borzunov commented Apr 26, 2023

This PR:

  1. Fixes warnings in hivemind.p2p destructors:

    Exception ignored in: <function P2P.__del__ at 0x7fbec8be5f70>
    Traceback (most recent call last):
      File "/home/jheuristic/anaconda3/envs/py38_petals_borzunov/lib/python3.8/site-packages/hivemind/p2p/p2p_daemon.py", line 636, in __del__
        self._terminate()
      File "/home/jheuristic/anaconda3/envs/py38_petals_borzunov/lib/python3.8/site-packages/hivemind/p2p/p2p_daemon.py", line 657, in _terminate
        self._child.terminate()
      File "/home/jheuristic/anaconda3/envs/py38_petals_borzunov/lib/python3.8/asyncio/subprocess.py", line 141, in terminate
        self._transport.terminate()
      File "uvloop/handles/process.pyx", line 636, in uvloop.loop.UVProcessTransport.terminate
      File "uvloop/handles/process.pyx", line 378, in uvloop.loop.UVProcessTransport._check_proc
    ProcessLookupError:
    Exception ignored in: <function Client.__del__ at 0x7fbec8bd7a60>
    Traceback (most recent call last):
      File "/home/jheuristic/anaconda3/envs/py38_petals_borzunov/lib/python3.8/site-packages/hivemind/p2p/p2p_daemon_bindings/p2pclient.py", line 53, in __del__
        self.close()
      File "/home/jheuristic/anaconda3/envs/py38_petals_borzunov/lib/python3.8/site-packages/hivemind/p2p/p2p_daemon_bindings/p2pclient.py", line 50, in close
        self.control.close()
    AttributeError: 'NoneType' object has no attribute 'close'
  2. Makes bfloat16 serialization in hivemind.compression forward- and backward-compatible. The code before this PR (a) didn't work in torch < 1.13.0 (hivemind requires torch >= 1.9.0) and (b) led to this warnings on torch >= 2.0:

    /opt/conda/lib/python3.10/site-packages/hivemind/compression/base.py:115: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
      tensor = torch.as_tensor(storage, dtype=torch.bfloat16)

    The new code works without warnings in all versions of PyTorch.

@codecov
Copy link

codecov bot commented Apr 26, 2023

Codecov Report

Merging #565 (07e98ae) into master (6c3a46c) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #565      +/-   ##
==========================================
+ Coverage   75.85%   75.92%   +0.07%     
==========================================
  Files          81       81              
  Lines        8002     8001       -1     
==========================================
+ Hits         6070     6075       +5     
+ Misses       1932     1926       -6     
Impacted Files Coverage Δ
hivemind/compression/base.py 94.28% <100.00%> (-0.24%) ⬇️
hivemind/p2p/p2p_daemon.py 94.47% <100.00%> (+0.01%) ⬆️
hivemind/p2p/p2p_daemon_bindings/p2pclient.py 100.00% <100.00%> (ø)

... and 2 files with indirect coverage changes

@borzunov borzunov marked this pull request as ready for review April 26, 2023 10:25
@borzunov borzunov changed the title Fix warnings in hivemind.p2p and hivemind.compression Fix warnings and compatibility issues in hivemind.p2p and hivemind.compression Apr 26, 2023
@borzunov borzunov changed the title Fix warnings and compatibility issues in hivemind.p2p and hivemind.compression Fix errors in hivemind.p2p and hivemind.compression Apr 26, 2023
Copy link
Member

@justheuristic justheuristic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

hivemind/compression/base.py Show resolved Hide resolved
hivemind/compression/base.py Show resolved Hide resolved
@borzunov borzunov merged commit 0d2614d into master Apr 26, 2023
13 checks passed
@borzunov borzunov deleted the fix-warnings branch April 26, 2023 11:40
mryab pushed a commit that referenced this pull request May 1, 2023
This PR:

1. Fixes warnings in hivemind.p2p destructors.

2. Makes bfloat16 serialization in hivemind.compression forward- and backward-compatible. The code before this PR (a) didn't work in torch < 1.13.0 (hivemind requires torch >= 1.9.0) and (b) led to warnings on torch >= 2.0. The new code works without warnings in all versions of PyTorch.

(cherry picked from commit 0d2614d)
borzunov added a commit to bigscience-workshop/petals that referenced this pull request May 7, 2023
This PR:

1. Updates hivemind to 1.1.8 (includes learning-at-home/hivemind#565)
2. Enables efficient bfloat16 serialization by default (`USE_LEGACY_BFLOAT16 = False`)
3. Removes logging code that was included to hivemind in learning-at-home/hivemind#542
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants