Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Audio::transcribe] Add support for timestamp_granularities #335

Closed

Conversation

jplhomer
Copy link

@jplhomer jplhomer commented Feb 9, 2024

What:

  • Bug Fix
  • New Feature

Description:

This PR adds support for the new timestamp_granularities[] property in Audio Transcriptions.

  • Adds the ability to describe the granularity level of the transcription
  • Processes the response to include words
  • Adjusts the HTTPTransporter Payload class to support sending array-based data

Currently, attempting to pass in a granularity level array results in an error:

$response = OpenAI::audio()->transcribe([
  'model' => 'whisper-1',
  'file' => $file
  'response_format' => 'verbose_json',
  'timestamp_granularities' => ['word'],
]);

Exception:

First argument to "Http\Message\MultipartStream\MultipartStreamBuilder::createStream()" must be a string, resource or StreamInterface.

My solution is to parse one level of array data. I'm not sure if this is the best solution, or if we ought to support recursive array parsing.

// Support only one level of nested arrays
if (is_array($value)) {
foreach ($value as $nestedKey => $nestedValue) {
$streamBuilder->addResource($key.'['.$nestedKey.']', $nestedValue);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This didn't work for me using the provided syntax: 'timestamp_granularities' => ['word'],

For this to work we need to do something like

Suggested change
$streamBuilder->addResource($key.'['.$nestedKey.']', $nestedValue);
$finalkey = is_int($nestedKey) ? $key.'[]' : $key.'['.$nestedKey.']';
$streamBuilder->addResource($finalkey, $nestedValue);

@solomonjames
Copy link

This looks great overall, and worked for me once I made that small tweak to your code. Thank you for pushing this up, I hope we can get it merged into the main project, as I have it forked just to use this feature.

@gehrisandro
Copy link
Collaborator

Resolve here #374

Thanks @jplhomer for your work!

@jplhomer jplhomer deleted the support-transcription-granularity branch April 15, 2024 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants