Generative AI Book Italian

BookLed Experiment 3 (AI Book!)

Overview

This script is designed for an experimental setup called "BookLed," where pages of a physical book are read, and AI-generated audio content in Italian is supplied automatically as the pages turn. Here’s a detailed explanation of how the script works:

Initial Setup and Configuration

1. Imports and Dependencies:

   • Various libraries are imported, including os, time, serial, threading, requests, pydub, wave, pyaudio, dotenv, json, and openai.
   • Deprecation warnings are suppressed.

2. Create a Temporary Directory:

   • A temporary directory named temp_mp3 is created to store downloaded MP3 files.

BookLed Serial Connection

3. Configure Serial Connection:
   • A serial connection to the BookLed device is established via a specified COM port (COM4).

Load OpenAI API Key

4. Load API Key:
   • The script loads the OpenAI API key from an environment variable using dotenv.

Prompts and Remote Access Gateway (RAG) Contents

5. Set URLs for Prompts and Images:
   • URLs for prompts and images stored on GitHub are specified. These are used later to retrieve content.

Functions Definition

6. Function: read_last_page_number:

   • Reads data from the serial port to extract the last page number detected by the BookLed device.

7. Function: read_file_to_string:

   • Reads the content of a local file and returns it as a string.

8. Function: download_file_from_github:

   • Downloads a file from a specified GitHub URL and returns its content as a string.

9. Function: download_mp3:

   • Downloads an MP3 file from a URL and saves it to the temporary directory.

10. Function: play_audio:

    • Converts an MP3 file to WAV format and plays it using pyaudio.

11. Function: play_mp3_in_thread:

    • Runs play_audio in a separate thread to allow concurrent playback.

12. Function: call_openai_api:

    • Sends a request to the OpenAI API to generate storytelling content based on the provided prompt and image URL.

13. Function: text_to_speech:

    • Converts generated text content to speech using the OpenAI API and saves it as an MP3 file.

Main Loop for BookLed Interaction

14. Prompt Source Selection:

    • Determines whether to use a local prompt file or download it from GitHub.

15. BookLed and AI Play Loop:

    • Initializes global variables (current_page, stop_audio, audio_thread).
    • Enters a loop where it continuously reads the current page number from the BookLed device.
    • When a new page is detected:
      • Stops any currently playing audio.
      • Calls the OpenAI API to convert the page image into storytelling content.
      • Converts the generated content into speech.
      • Starts playing the new audio in a separate thread.
    • Handles interruptions and ensures proper cleanup on exit.

Summary

The script integrates hardware (BookLed) with AI services (OpenAI API) to create an interactive experience where turning the pages of a physical book triggers the generation and playback of audio content. This is achieved through serial communication with the BookLed device, API calls to OpenAI for content generation and text-to-speech conversion, and concurrent audio playback using threading.