-
Notifications
You must be signed in to change notification settings - Fork 820
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[trace][feature] Add trace usage telemetry #3368
Conversation
promptflow SDK CLI Azure E2E Test Result zhengfei/feature/trace-telemetry 4 files 4 suites 4m 15s ⏱️ Results for commit b7f04d4. ♻️ This comment has been updated with latest results. |
SDK CLI Global Config Test Result zhengfei/feature/trace-telemetry6 tests 6 ✅ 1m 15s ⏱️ Results for commit b7f04d4. ♻️ This comment has been updated with latest results. |
Executor Unit Test Result zhengfei/feature/trace-telemetry796 tests 796 ✅ 3m 45s ⏱️ Results for commit b7f04d4. ♻️ This comment has been updated with latest results. |
Executor E2E Test Result zhengfei/feature/trace-telemetry246 tests 240 ✅ 5m 19s ⏱️ Results for commit b7f04d4. ♻️ This comment has been updated with latest results. |
SDK CLI Test Result zhengfei/feature/trace-telemetry 4 files 4 suites 1h 6m 12s ⏱️ Results for commit b7f04d4. ♻️ This comment has been updated with latest results. |
Description
This PR targets to add trace usage telemetry in custom dimensions, which looks like below:
There will be a thread scheduled to flush trace telemetry in memory - different process worker stores telemetry in different process, so it's difficult to flush for each, so a scheduled thread shall work best in this case.
Besides, fixes a replay test introduced in #3348 , which changes test class init signature, and leads to our replay tests breaking.
All Promptflow Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines