Add streaming with Faraday #234

alexrudall · 2023-04-02T12:47:00Z

WIP on v4 of the gem - main breaking change is switching HTTP library to improve performance and allow streaming and parallel requests
Switch to Faraday
Add streaming to Chat as MVP

codeclimate

The PR diff size of 22336 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 22335 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 22338 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 22338 lines exceeds the maximum allowed for the inline comments feature.

bf4 · 2023-04-02T14:18:22Z

ruby-openai.gemspec

@@ -25,7 +25,8 @@ Gem::Specification.new do |spec|
  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
  spec.require_paths = ["lib"]

-  spec.add_dependency "httparty", ">= 0.18.1"
+  spec.add_dependency "faraday", "2.7.4"
+  spec.add_dependency "faraday-typhoeus", "1.0.0"


I personally have no interest in using or install typhoeus on my system. The advantage of Faraday is you can leave those decisions to someone else.

If you want to test Faraday with typhoeus in the gem, the pattern for that would be to put typhoeus in the Gemfile but not the gemspec, that way it's not required.

Also, our Faraday requirement in our app

gem "faraday", ">= 2"

We happen to be on 2.7.4, but I'd prefer not to be locked in to that version

- spec.add_dependency "faraday", "2.7.4" + spec.add_dependency "faraday", ">= 2"

Great points, thanks for review! Experimenting with Faraday only version.

lib/openai/client.rb

lib/openai.rb

codenamev · 2023-04-20T12:33:03Z

This looks great! How can I help push this through the finish line?

codeclimate

The PR diff size of 23814 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 23814 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 23812 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 23812 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 23810 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 24977 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 26402 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 26402 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 26397 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 26397 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 26397 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 26998 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 27377 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 27377 lines exceeds the maximum allowed for the inline comments feature.

codeclimate

The PR diff size of 27380 lines exceeds the maximum allowed for the inline comments feature.

alexrudall · 2023-04-24T10:40:39Z

Streaming getting there :)

Apr-24-2023.18-38-28.mp4

alexrudall · 2023-04-25T15:18:01Z

Thanks for your input all. Would appreciate any final reviews 👍

This PR switches the HTTP library from HTTParty to Faraday and adds streaming functionality on the Chat endpoint.
Breaking change: endpoints now return JSON rather than HTTParty objects. You need to update your code to handle this change, changing code like JSON.parse(response.body)["key"] and response.parsed_response["key"] to just response["key"].
README examples should all be updated and working and including Chat streaming.
Edit; To use this branch in your Gemfile do this: gem "ruby-openai", github: "alexrudall/ruby-openai", branch: "faraday", and then bundle update ruby-openai to get the latest

lib/openai/client.rb

Improved stream parsing

kmcphillips · 2023-05-04T03:55:16Z

This is rad. Thanks for this change! ❤️

gastonmorixe · 2023-05-04T17:10:07Z

Thank you everyone!

andresgutgon · 2023-10-11T16:15:24Z

lib/openai/client.rb

+      JSON.parse(response.body)
+    rescue JSON::ParserError
+      # Convert a multiline file of JSON objects to a JSON array.
+      JSON.parse(response.body.gsub("}\n{", "},{").prepend("[").concat("]"))


Hi @alexrudall, I come here now in October 😅. I was wondering when happens this case. I have seen that this happen to us when OpenAI returns empty string. Is this documented somewhere why it happens?

codeclimate bot reviewed Apr 2, 2023

View reviewed changes

alexrudall mentioned this pull request Apr 2, 2023

Streaming #189

Closed

codeclimate bot reviewed Apr 2, 2023

View reviewed changes

This was referenced Apr 2, 2023

Switch to using Typhoeus #194

Closed

feat: simple response proxy #215

Closed

alexrudall changed the title ~~Add streaming with Faraday & Typhoeus~~ WIP: Add streaming with Faraday & Typhoeus Apr 2, 2023

bf4 reviewed Apr 2, 2023

View reviewed changes

lib/openai/client.rb Outdated Show resolved Hide resolved

bf4 reviewed Apr 2, 2023

View reviewed changes

lib/openai.rb Outdated Show resolved Hide resolved

alexrudall changed the title ~~WIP: Add streaming with Faraday & Typhoeus~~ WIP: Add streaming with Faraday Apr 22, 2023

codeclimate bot reviewed Apr 22, 2023

View reviewed changes

codeclimate bot reviewed Apr 24, 2023

View reviewed changes

alexrudall changed the title ~~WIP: Add streaming with Faraday~~ Add streaming with Faraday Apr 24, 2023

codeclimate bot reviewed Apr 24, 2023

View reviewed changes

Improve changelog

ef4348b

alexrudall mentioned this pull request Apr 25, 2023

Slow performance #225

Closed

fwitzke reviewed Apr 25, 2023

View reviewed changes

lib/openai/client.rb Outdated Show resolved Hide resolved

rmontgomery429 added 4 commits April 25, 2023 14:35

Parse chunks of data: {JSON} and only call the user proc when valid.

0cb6860

Linting fixes based on feedback from Rubocop

33f6e18

Do not send bytesize to the user's proc

ce82858

Fix linting suggestions from Code Climate (Rubocop)

683f661

fwitzke mentioned this pull request Apr 25, 2023

Improved stream parsing #248

Merged

3 tasks

rmontgomery429 added 5 commits April 25, 2023 15:28

Send error messages from OpenAI to the user

89e0cbf

Cleanup

455e61a

Update method documentation

7d77b47

Remove trailing space

7b07133

Communicate what this heredoc is more correctly

cbbdd1b

fwitzke reviewed Apr 25, 2023

View reviewed changes

lib/openai/client.rb Outdated Show resolved Hide resolved

alexrudall and others added 11 commits April 26, 2023 16:28

Merge pull request #248 from rmontgomery429/faraday-json-parsing

c02665a

Improved stream parsing

Make method public

9ee2111

Tweak comments

0ab28f7

Rubocop

57dbfd5

Fix timeouts

1ec477a

Try more aggressive timeout for CI

82d6435

Refactor

1cd6a2c

Rubocop

a516b22

Rename spec

58b4003

Update README for streaming

b7df81c

Point changelog readers to README

ef1e2bc

alexrudall merged commit 5683eb9 into main Apr 26, 2023

andresgutgon reviewed Oct 11, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add streaming with Faraday #234

Add streaming with Faraday #234

alexrudall commented Apr 2, 2023 •

edited

Loading

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

bf4 Apr 2, 2023

alexrudall Apr 22, 2023

codenamev commented Apr 20, 2023

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

codeclimate bot left a comment

alexrudall commented Apr 24, 2023

alexrudall commented Apr 25, 2023 •

edited

Loading

kmcphillips commented May 4, 2023

gastonmorixe commented May 4, 2023

andresgutgon Oct 11, 2023

Add streaming with Faraday #234

Add streaming with Faraday #234

Conversation

alexrudall commented Apr 2, 2023 • edited Loading

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

bf4 Apr 2, 2023

Choose a reason for hiding this comment

alexrudall Apr 22, 2023

Choose a reason for hiding this comment

codenamev commented Apr 20, 2023

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

codeclimate bot left a comment

Choose a reason for hiding this comment

alexrudall commented Apr 24, 2023

alexrudall commented Apr 25, 2023 • edited Loading

kmcphillips commented May 4, 2023

gastonmorixe commented May 4, 2023

andresgutgon Oct 11, 2023

Choose a reason for hiding this comment

alexrudall commented Apr 2, 2023 •

edited

Loading

alexrudall commented Apr 25, 2023 •

edited

Loading