Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Ensure API errors are tracked #1880

Merged
merged 4 commits into from Nov 21, 2023
Merged

Conversation

jsumners-nr
Copy link
Contributor

This PR is to resolve #1845.

@jsumners-nr jsumners-nr marked this pull request as ready for review November 17, 2023 20:04
Copy link

codecov bot commented Nov 17, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (86d83f2) 96.87% compared to head (516a94a) 96.88%.

❗ Current head 516a94a differs from pull request most recent head 9c155c7. Consider uploading reports for the commit 9c155c7 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1880      +/-   ##
==========================================
+ Coverage   96.87%   96.88%   +0.01%     
==========================================
  Files         209      210       +1     
  Lines       39635    39698      +63     
==========================================
+ Hits        38397    38463      +66     
+ Misses       1238     1235       -3     
Flag Coverage Δ
integration-tests-16.x 78.74% <ø> (ø)
integration-tests-18.x 79.01% <ø> (-0.02%) ⬇️
integration-tests-20.x 79.03% <ø> (ø)
unit-tests-16.x 91.09% <38.02%> (-0.09%) ⬇️
unit-tests-18.x 91.08% <38.02%> (-0.09%) ⬇️
unit-tests-20.x 91.08% <38.02%> (-0.09%) ⬇️
versioned-tests-16.x 73.65% <100.00%> (+0.04%) ⬆️
versioned-tests-18.x 73.65% <100.00%> (+0.04%) ⬆️
versioned-tests-20.x 73.66% <100.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have more comments after another round but I think I called out everything that needs to be addressed

lib/llm-events/openai/llm-error.js Outdated Show resolved Hide resolved
* Represents an error object, to be tracked via `agent.errors`, that is the
* result of some error returned from the OpenAI API.
*/
class OpenAiLlmError extends Error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should just be a class, not inheriting from Error. When you call agent.errors.add the signature is transaction, err, customAttrs. This class should be providing all the custom attrs. Since the NR exceptions do not handle error.code and error.param those will be properties on this class.

lib/instrumentation/openai.js Outdated Show resolved Hide resolved
lib/instrumentation/openai.js Outdated Show resolved Hide resolved
lib/llm-events/openai/error-message.js Outdated Show resolved Hide resolved
lib/instrumentation/openai.js Show resolved Hide resolved
@jsumners-nr jsumners-nr force-pushed the issue-1845 branch 2 times, most recently from 200ad00 to 7dab013 Compare November 20, 2023 18:44
Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work 👏🏻

@@ -35,6 +36,7 @@ function shouldSkipInstrumentation(config, shim) {
return semver.lt(pkgVersion, MIN_VERSION)
}

// eslint-disable-next-line sonarjs/cognitive-complexity
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i noticed this too. I think it's because we're defining all the functions within the module.exports. We try not to add to the cognitive complexity but we can address later

*/
function recordChatCompletionMessages({ segment, request, response }) {
function recordChatCompletionMessages({ segment, request, response, hasError = false }) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: hasError and withError

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is your suggestion?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just calling out the arg coming in is diff than it going into the event creation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my mind there's a difference between the two, but I'm totally open to changing it one way or the other. Which would you prefer?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I guess so. Just keep it.

@jsumners-nr jsumners-nr merged commit 289c2a2 into newrelic:main Nov 21, 2023
22 checks passed
Node.js Engineering Board automation moved this from Needs PR Review to Done: Issues recently completed Nov 21, 2023
@jsumners-nr jsumners-nr deleted the issue-1845 branch November 21, 2023 15:20
@github-actions github-actions bot mentioned this pull request Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Node.js Engineering Board
  
Done: Issues recently completed
Development

Successfully merging this pull request may close these issues.

Create error traces on failures of both embedding and chat creation
2 participants