Skip to content

Conversation

@FanhaiLu1
Copy link
Collaborator

@FanhaiLu1 FanhaiLu1 commented May 31, 2024

This PR integrated with JetSteam and ray for interleave serving. Several points import to this PR:

  1. I'm defining a tpu_chip parameter to pass the devices info, the program will fetch device info from worker in future.
  2. Jax_padding must be set ad false, all the TPU are used by worker, it will throw error for any jax call outside of workers.

@FanhaiLu1 FanhaiLu1 merged commit b234a57 into AI-Hypercomputer:main May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants