Generative AI Book English with soundtrack

BookLed Experiment 5 (AI Book with Voiceover and Soundtrack) - ENGLISH

Overview

This Python script is designed for an experiment where turning the pages of the BookLed physical book triggers the playback of an MP3 audio, generated in real-time by AI. The AI analyzes the graphical content of the book pages and produces a corresponding story. Each page turn prompts the AI to generate a unique narrative, which is then converted to speech using OpenAI's TTS API. Additionally, a contextual soundtrack is played alongside the narration.

This wiki page provides step-by-step instructions for setting up and running the "AI Book English with soundtrack" jupyter notebook code from this GitHub repository.

Prerequisites

1. BookLed Device: Connect the BookLed to the PC via USB and ensure the correct COM port is configured.
2. OpenAI API: An active OpenAI API account is required.
3. Internet Connection: The OpenAI API and GitHub content are accessed via the internet.
4. FFmpeg: Ensure FFmpeg is installed and configured correctly for audio processing.

1) Copying the Jupyter Notebook Code from GitHub to Your Environment

Manual Download

1. Open the GitHub repository link: GitHub Link
2. Click on the Raw button to view the raw file.
3. Right-click on the page and select Save as... to download the file to your local machine.

Using Git

1. Open your terminal (or Git Bash if on Windows).
2. Navigate to the directory where you want to clone the repository:

   cd /path/to/your/directory

3. Clone the repository:

   git clone https://github.com/robotoons/BookLed.git

4. Navigate to the specific directory containing the Jupyter notebook:

   cd BookLed/codes/jupyter notebooks

2) Installing All Dependencies in the Environment with pip

1. Open your terminal.
2. Navigate to the directory containing the Jupyter notebook if you haven't already:

   cd /path/to/BookLed/codes/jupyter notebooks

3. Create a virtual environment (optional but recommended):

   python -m venv myenv

4. Activate the virtual environment:
   • On Windows:

     myenv\Scripts\activate

   • On macOS/Linux:

     source myenv/bin/activate

5. Install the required dependencies: Manually install the necessary packages:

   pip install pydub serial requests python-dotenv openai pyaudio

3) Activating the Environment and Running the Jupyter Notebook

1. Ensure your virtual environment is activated.
2. Launch Jupyter Notebook:

   jupyter notebook

3. In the Jupyter Notebook interface, navigate to the directory containing your downloaded notebook file (BookLed Experiment 5 (AI book) - ENGLISH V5.ipynb) and open it.

Setup Instructions

1. Connect the BookLed to your PC via USB.
2. Configure the correct COM port number.
3. Ensure you have an active internet connection.
4. Ensure you have an active OpenAI API account.
5. Power on the BookLed device.

For complete Puthon and ffmpeg setup for windows 10, go to the wiki page Python Environment Setup.