Add DODO-GPT #343

mjsutcliffe99 · 2024-07-26T12:06:19Z

Add various new features relating to Chat-GPT and Whisper APIs; namely "Dodo Hint", "Dodo Query", "Import image as ZX-diagram", and the text-to-speech and speech-to-text.

mjsutcliffe99 · 2024-07-26T12:08:10Z

To run the DODO-GPT features, you need a paid OpenAI account (although you can add the minimum of $5 and each query costs a tiny fraction of a penny). Simply paste your API key into the users/key.txt file.

jvdwetering · 2024-07-26T12:27:39Z

The test fails because it doesn't recognise 'openai' as an installed module. You should add this to requirements.txt and the conf.

jvdwetering

I think this is cool, but just checking: do these additional imports add a lot to the loading time or to the memory requirements? Or are these openai, pydub, sounddevice and ffmpeg-python modules all relatively low weight?

jvdwetering · 2024-07-26T12:28:55Z

zxlive/dodo.py

+def get_image_prompt():
+    """Returns the prompt that encourages DODO-GPT to describe the given image like a ZX-diagram."""
+
+    return """


Maybe put this large string at the top of the file

jvdwetering · 2024-07-26T12:29:16Z

zxlive/dodo.py

+
+def get_local_api_key():
+    """Get the API key from key.txt file"""
+    f = open(GET_MODULE_PATH()+"/user/key.txt", "r")


It might be better to have the API key be settable in the settings of ZXLive.

That was my plan! This is just a quick temporary solution so that I can push the changes. I guess we could also save API keys locally if we hash them first, so that way you won't have to re-enter it in every new instance of the program?

jvdwetering · 2024-07-26T12:30:51Z

zxlive/edit_panel.py

+
+        self.dodo_hint = QToolButton(self)
+        self.dodo_hint.setIcon(QIcon(get_data("icons/dodo.png")))
+        self.dodo_hint.setToolTip("Dodo Hint")


Although this is funny, maybe make it more descriptive? "Ask DodoGPT for suggestions on how to rewrite your diagram"

jvdwetering · 2024-07-26T12:31:11Z

zxlive/edit_panel.py

+
+        self.dodo_query = QToolButton(self)
+        self.dodo_query.setIcon(QIcon(get_data("icons/mic.svg")))
+        self.dodo_query.setToolTip("Dodo Query")


Perhaps make it clearer that this will use input from your mic.

Yep, I can add a little note there

jvdwetering · 2024-07-26T12:32:09Z

zxlive/mainwindow.py

@@ -182,6 +186,8 @@ def __init__(self) -> None:
        self.effects = {e: load_sfx(e) for e in SFXEnum}

        QShortcut(QKeySequence("Ctrl+B"), self).activated.connect(self._toggle_sfx)
+
+        get_local_api_key() # load the local user's API key


I think this will currently give a fatal error if the key file does not exist. So then the file should be added to the manifest. Better yet would be to load the key from the ZXLive Qt Settings.

Yeah, there are a few places I need to add error handling yet

mjsutcliffe99 · 2024-07-26T13:07:02Z

I think this is cool, but just checking: do these additional imports add a lot to the loading time or to the memory requirements? Or are these openai, pydub, sounddevice and ffmpeg-python modules all relatively low weight?

I think they're all relatively light, just for adding audio handling and the gpt API. Certainly I don't notice any lag or slowing down of the boot time or runtime, nor the memory

mjsutcliffe99 · 2024-07-26T13:08:19Z

The test fails because it doesn't recognise 'openai' as an installed module. You should add this to requirements.txt and the conf.

It's added to the requirements.txt - I'll try adding it to the conf as well!

mjsutcliffe99 · 2024-07-29T00:43:44Z

I've updated the minor changes you suggested @jvdwetering.

The relevant libraries are included in the requirements.txt, but I'm not sure what exactly needs to be added to the conf.py file to get it to pass the checks - if anyone has any ideas here please help out and try to get this to work.

And lastly, regarding the API key, I have some ideas of how we can handle this a bit better, but perhaps that could be a separate PR as I want to ensure first that all the core functionality works on machines (and API keys) other than my own.

jvdwetering · 2024-07-29T11:24:03Z

Ah sorry, I mentioned conf, but I meant pyproject.toml, that's the one that stores dependencies.

jvdwetering · 2024-07-30T15:55:38Z

The test suite fails because

import sounddevice as sd
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/sounddevice.py:71: in <module>
    raise OSError('PortAudio library not found')
E   OSError: PortAudio library not found

Perhaps add a check that there is indeed a sound device? Just a Try-Except around the import should be sufficient. If you also include some checks in the function that calls it that would also protect against it.

RazinShaikh · 2024-07-31T16:35:59Z

Is there any way to try this if I don't have paid chatgpt?

mjsutcliffe99 · 2024-07-31T17:11:33Z

Is there any way to try this if I don't have paid chatgpt?

Unfortunately not, but as long as you put in the minimum amount ($5) your account will count as a "paid" version and your API key should work - and then every query is very very cheap (a tiny fraction of a penny - so that $5 should last for ages).

A temporary solution until the sound system for DODO-GPT is switched for the same that the SFX uses

Again, a temporary solution to catch a failed sounddevice import

please work :'(

mjsutcliffe99 · 2024-08-02T03:19:18Z

Jubilation! It finally passed the tests. I'm keen to see if this all works on machines (and OpenAI APIs) other than my own so would be great to hear back from anyone testing out the DODO-GPT features! I still want to polish some things in the code but that can wait for another PR.

RazinShaikh · 2024-08-12T12:56:19Z

@mjsutcliffe99 I can't get it to work. I get this error:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Razin Shaikh\\OneDrive\\Documents\\code\\zxlive\\zxlive/temp/Dodo_Dave_latest.mp3'

mjsutcliffe99 · 2024-08-12T13:13:35Z

@mjsutcliffe99 I can't get it to work. I get this error: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Razin Shaikh\\OneDrive\\Documents\\code\\zxlive\\zxlive/temp/Dodo_Dave_latest.mp3'

Did you provide an API key in user/key.txt? (If you don't have one I can probably DM you mine to test with, just let me know)

(Nevertheless, I should probably add more error handling for cases like this)

RazinShaikh · 2024-08-12T13:47:37Z

I provided api key and have created a paid openai account, so I suppose it's not related to that. I also can't get the import from image to work. It can't find import_image_dialog

mjsutcliffe99 · 2024-08-12T14:53:33Z

I provided api key and have created a paid openai account, so I suppose it's not related to that. I also can't get the import from image to work. It can't find import_image_dialog

Hopefully just a trivial error - I'll investigate soon

RazinShaikh · 2024-08-21T12:09:59Z

any update on this?

mjsutcliffe99 · 2024-08-22T21:50:58Z

any update on this?

I haven't had the chance to look yet, I've been super busy! But I'll get round to it as soon as I can!

Add DODO-GPT

5d9f41b

Add various new features relating to Chat-GPT and Whisper APIs; namely "Dodo Hint", "Dodo Query", "Import image as ZX-diagram", and the text-to-speech and speech-to-text.

jvdwetering reviewed Jul 26, 2024

View reviewed changes

jvdwetering marked this pull request as draft July 26, 2024 12:34

mjsutcliffe99 added 2 commits July 29, 2024 01:31

Move long string to top of file

49d1364

Make the DODO-GPT button tooltips more helpful

597161d

mjsutcliffe99 added 2 commits July 29, 2024 22:12

Add DODO-GPT dependencies to pyproject.toml

9062ee0

Merge branch 'master' of https://github.com/mjsutcliffe99/zxlive

5560e81

mjsutcliffe99 added 4 commits August 2, 2024 03:56

Add a try-except around the sounddevice import

b56a00c

A temporary solution until the sound system for DODO-GPT is switched for the same that the SFX uses

Add a try-except around dodo query

1ad2b75

Again, a temporary solution to catch a failed sounddevice import

Make try-except for sd import catch-all

c8c7601

please work :'(

Add missing import

3566a25

mjsutcliffe99 marked this pull request as ready for review August 3, 2024 01:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DODO-GPT #343

Add DODO-GPT #343

mjsutcliffe99 commented Jul 26, 2024

mjsutcliffe99 commented Jul 26, 2024

jvdwetering commented Jul 26, 2024

jvdwetering left a comment

jvdwetering Jul 26, 2024

mjsutcliffe99 Jul 26, 2024

jvdwetering Jul 26, 2024

mjsutcliffe99 Jul 26, 2024

jvdwetering Jul 26, 2024

mjsutcliffe99 Jul 26, 2024

jvdwetering Jul 26, 2024

mjsutcliffe99 Jul 26, 2024

jvdwetering Jul 26, 2024

mjsutcliffe99 Jul 26, 2024

mjsutcliffe99 commented Jul 26, 2024

mjsutcliffe99 commented Jul 26, 2024

mjsutcliffe99 commented Jul 29, 2024

jvdwetering commented Jul 29, 2024

jvdwetering commented Jul 30, 2024

RazinShaikh commented Jul 31, 2024

mjsutcliffe99 commented Jul 31, 2024

mjsutcliffe99 commented Aug 2, 2024

RazinShaikh commented Aug 12, 2024

mjsutcliffe99 commented Aug 12, 2024

RazinShaikh commented Aug 12, 2024

mjsutcliffe99 commented Aug 12, 2024

RazinShaikh commented Aug 21, 2024

mjsutcliffe99 commented Aug 22, 2024

Add DODO-GPT #343

Are you sure you want to change the base?

Add DODO-GPT #343

Conversation

mjsutcliffe99 commented Jul 26, 2024

mjsutcliffe99 commented Jul 26, 2024

jvdwetering commented Jul 26, 2024

jvdwetering left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjsutcliffe99 commented Jul 26, 2024

mjsutcliffe99 commented Jul 26, 2024

mjsutcliffe99 commented Jul 29, 2024

jvdwetering commented Jul 29, 2024

jvdwetering commented Jul 30, 2024

RazinShaikh commented Jul 31, 2024

mjsutcliffe99 commented Jul 31, 2024

mjsutcliffe99 commented Aug 2, 2024

RazinShaikh commented Aug 12, 2024

mjsutcliffe99 commented Aug 12, 2024

RazinShaikh commented Aug 12, 2024

mjsutcliffe99 commented Aug 12, 2024

RazinShaikh commented Aug 21, 2024

mjsutcliffe99 commented Aug 22, 2024