Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support macOS natively #477

Merged
merged 48 commits into from
Aug 29, 2023
Merged

Support macOS natively #477

merged 48 commits into from
Aug 29, 2023

Conversation

borzunov
Copy link
Collaborator

@borzunov borzunov commented Aug 23, 2023

This PR makes both clients and servers work on macOS. Specifically, it:

  • Follows Use proper p2pd binary on macOS learning-at-home/hivemind#586 to run a macOS-compatible p2pd binary (both x86-64 and ARM64 are supported)
  • Fixes forking issues and tests on macOS, Python 3.10+
  • Introduces basic support for serving model blocks on Apple M1/M2 GPUs (torch.mps)
  • Increases max number of open files by default (it's not enough on Linux and is really small on macOS)

@borzunov borzunov marked this pull request as draft August 24, 2023 22:17
@borzunov borzunov force-pushed the macos-support branch 2 times, most recently from 98ad98f to 70b8418 Compare August 24, 2023 23:10
@@ -31,27 +50,13 @@ def get_pools(self):
PrioritizedTaskPool(dummy_pool_func, name="B", max_batch_size=1),
)

# Simulate requests coming from ConnectionHandlers
proc = mp.context.ForkProcess(target=_submit_tasks, args=(pools, results_valid))
proc.start()
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving that before Runtime.start() since (a) that's how real server works and (b) so we don't fork after creating active threads (this provably improves test stability)

- { model: 'bigscience/bloom-560m', os: 'ubuntu', python-version: '3.8' }
- { model: 'bigscience/bloom-560m', os: 'ubuntu', python-version: '3.11' }
- { model: 'Maykeye/TinyLLama-v0', os: 'ubuntu', python-version: '3.8' }
- { model: 'Maykeye/TinyLLama-v0', os: 'ubuntu', python-version: '3.11' }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • would be nice to add adapters for llama tests since they are the new main
  • if no time, please create an issue

src/petals/__init__.py Outdated Show resolved Hide resolved
@borzunov borzunov force-pushed the macos-support branch 2 times, most recently from ce2a348 to f47e922 Compare August 28, 2023 17:55
Copy link
Collaborator

@justheuristic justheuristic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@borzunov borzunov marked this pull request as ready for review August 28, 2023 20:03
@borzunov borzunov force-pushed the macos-support branch 2 times, most recently from 568a1c4 to 8889cab Compare August 29, 2023 03:30
@borzunov borzunov merged commit 26ebbfe into main Aug 29, 2023
9 checks passed
@borzunov borzunov deleted the macos-support branch August 29, 2023 03:49
@borzunov borzunov changed the title Support macOS Support macOS natively Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants