You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I am trying to see the overhead of creating a ucc communication context in a 4 node cluster with 64 slots/node.
This is a graph of calling ucc_context_create and ucc_context_destroy for 50 iterations and the average time spent on each operation. I was surprised to see that context creation takes constant time across the 4 nodes. Is this the expected behavior?
I measured the timings for the oob allgather operation (time between when the request was created and when the request completes) and it doesn't come close to this 3s mark. Can anyone shed some light on this?
The text was updated successfully, but these errors were encountered:
This is a graph of calling ucc_context_create and ucc_context_destroy for 50 iterations and the average time spent on each operation. I was surprised to see that context creation takes constant time across the 4 nodes. Is this the expected behavior?
ucc_context create time depends on multiple factors, e.g. what UCC TLs you are using and whether it's global context or not. It's expected if you run with TL UCP only, since we don't connect endpoints in advance and instead connection between peers is established only if it's really needed.
I measured the timings for the oob allgather operation (time between when the request was created and when the request completes) and it doesn't come close to this 3s mark. Can anyone shed some light on this?
Again it depends on TLs being used. Context create for TL UCP includes initializing UCP context and UCP worker.
Hi, I am trying to see the overhead of creating a ucc communication context in a 4 node cluster with 64 slots/node.
![AVERAGE of create ctx and AVERAGE of destroy ctx](https://private-user-images.githubusercontent.com/5582709/283900463-26e17af9-f7fa-4bbd-8341-31c3ae8701b4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUyNzE4MjEsIm5iZiI6MTcyNTI3MTUyMSwicGF0aCI6Ii81NTgyNzA5LzI4MzkwMDQ2My0yNmUxN2FmOS1mN2ZhLTRiYmQtODM0MS0zMWMzYWU4NzAxYjQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MDJUMTAwNTIxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MzZmOWY1MzU4NDkwYTNlY2QyMDJkMDM1MmY2MzI4NzYyNWZlYzc0NGYzM2QxNTU4MzIzZjg0NTk5YzAxZDkxNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.zgQaAj0YbrvhXlvL5ivXZdPkZ0On4Bh0fynwOefUHTY)
This is a graph of calling
ucc_context_create
anducc_context_destroy
for 50 iterations and the average time spent on each operation. I was surprised to see that context creation takes constant time across the 4 nodes. Is this the expected behavior?I measured the timings for the oob allgather operation (time between when the request was created and when the request completes) and it doesn't come close to this 3s mark. Can anyone shed some light on this?
The text was updated successfully, but these errors were encountered: