Skip to content

Fix misleading error message while streaming #1591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 3, 2025
Merged

Conversation

Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Jul 3, 2025

In innerRequest/innerStreamingRequest, if an error occur we raise a InferenceClientProviderApiError with a HTTP Request object in it. To avoid bloating the logs, a requestArgsToJson helper is applied to clean-up the body. However we do not pass the body to its but the requestArgs instead i.e. before provider-specific logic has been applied. This is can be really misleading since the body shown in the error is not necessarily the one sent in the request, making it hard to investigate.

I also took the opportunity to make sure accessToken is never printed in the logs (it is currently the case as part of requestArgs).

(also added a console.log for auto-provider)

@Wauplin
Copy link
Contributor Author

Wauplin commented Jul 3, 2025

Example of misleading stracktrace:

    at async postCreateResponse (/Users/vb/Documents/hf-projects/responses.js/src/routes/responses.ts:185:21) {
  httpRequest: {
    url: 'https://router.huggingface.co/novita/v3/openai/chat/completions',
    method: 'POST',
    headers: {
      Authorization: 'Bearer [redacted]',
      'Content-Type': 'application/json',
      'User-Agent': '@huggingface/inference/4.3.0 Node.js/23'
    },
    body: {
      accessToken: '', # <=========== token leaked here + shouldn't be in body
      model: 'meta-llama/Llama-3.3-70B-Instruct',
      messages: [Array],
      stream: true,
      temperature: 1,
      tools: [Array],
      top_p: 1,
      data: null  # <================= shouldn't be in body
    }
  },
  httpResponse: {
    requestId: 'Root=1-686651cf-4f0ed45361744c1734cb3c1f',
    status: 404,
    body: { message: 'API error', type: 'API_ERROR' }
  }
}

Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you

Copy link
Contributor

@SBrandeis SBrandeis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch !!

@Wauplin Wauplin merged commit 1435c14 into main Jul 3, 2025
5 checks passed
@Wauplin Wauplin deleted the misleading-error-message branch July 3, 2025 13:47
@Wauplin
Copy link
Contributor Author

Wauplin commented Jul 3, 2025

thanks for the quick review :) I'll make a patch release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants