This project uses AWS Bedrock's Claude 3 Sonnet model to convert hand-drawn UI sketches into working web applications. It supports two modes of operation:
- Real-time webcam capture for immediate sketching and conversion
- Image upload through a user-friendly Streamlit interface
- Node.js 18.x or higher
- Download the Node.js installer from official website
- Run the installer (.msi file)
- Follow the installation wizard
- Verify installation:
node --version
npm --version
Using package manager:
# For macOS (using Homebrew)
brew install node
# For Ubuntu/Debian
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs
# For other Linux distributions, check your package manager
- Python 3.8 or higher
- Webcam (for webcam-based flow)
- AWS Account with Bedrock access
- Access to Claude 3 Sonnet model in AWS Bedrock
- Git
- Download Python installer from python.org
- Run the installer
- Important: Check "Add Python to PATH" during installation
- Verify installation:
python --version
pip --version
# macOS (using Homebrew)
brew install python
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install python3 python3-pip
# Verify installation
python3 --version
pip3 --version
The system follows this flow:
-
Image Input: The application supports two methods:
- OpenCV webcam capture for real-time sketching
- Streamlit-based image upload interface
-
Image Processing: The captured/uploaded image is processed and converted to a format suitable for analysis.
-
Claude 3 Integration: The processed image is sent to AWS Bedrock's Claude 3 Sonnet model along with carefully crafted prompts that guide the model to:
- Analyze and understand the UI elements in the sketch
- Generate appropriate Next.js components and code
- Ensure the generated code follows best practices
-
Code Generation: Claude 3 generates the necessary Next.js components, styles, and functionality based on the sketch.
-
Application Update: The system automatically:
- Updates the Next.js application files
- Applies any necessary styling
- Implements required functionality
- Hot-reloads the development server
-
Live Preview: The changes are immediately visible in the running Next.js application, providing instant feedback on the generated code.
-
Future Updates: The developers can use the Amazon Q for Developer for future updates to the generated code.
- Clone the repository:
git clone <repository-url>
cd draw-an-app
- Create and activate a virtual environment:
Windows:
python -m venv venv
.\venv\Scripts\activate
macOS/Linux:
python3 -m venv venv
source venv/bin/activate
- Install required Python packages:
Windows:
pip install -r src\requirements.txt
macOS/Linux:
pip install -r src/requirements.txt
- Install Node.js dependencies:
Windows/macOS/Linux:
cd blank-nextjs-app
npm install
cd ..
- Configure AWS credentials:
Windows (PowerShell):
$env:AWS_ACCESS_KEY_ID="your_access_key"
$env:AWS_SECRET_ACCESS_KEY="your_secret_key"
$env:AWS_DEFAULT_REGION="us-west-2" # or your preferred region
macOS/Linux:
export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key
export AWS_DEFAULT_REGION=us-west-2 # or your preferred region
Alternatively, you can configure AWS credentials using the AWS CLI:
aws configure
- Verify Bedrock Access:
- Ensure you have access to the Claude 3 Sonnet model in AWS Bedrock
- Verify your AWS account has the necessary permissions to invoke the Bedrock runtime
Before running the application, verify:
- Your webcam is properly connected and accessible (if using webcam flow)
- You have access to AWS Bedrock's Claude 3 Sonnet model
- AWS credentials are properly configured
- Start the Next.js development server:
Windows/macOS/Linux:
cd blank-nextjs-app
npm install
npm run dev
Keep this server running to view your generated applications at http://localhost:3000
- In a new terminal, run the webcam-based application:
Windows:
# Make sure you're in the project root directory and venv is activated
python src\main_v2.py
macOS/Linux:
# Make sure you're in the project root directory and venv is activated
python src/main_v2.py
- When the webcam window opens:
- Press SPACE to capture and process an image
- Press Q to quit the application
- Start the Next.js development server (if not already running):
cd blank-nextjs-app
npm install
npm run dev
- In a new terminal, launch the Streamlit interface:
Windows:
# Make sure you're in the project root directory and venv is activated
streamlit run src\main_streamlit.py
macOS/Linux:
# Make sure you're in the project root directory and venv is activated
streamlit run src/main_streamlit.py
- Using the Streamlit interface:
- Upload your UI sketch image (supported formats: JPG, JPEG)
- Click "Generate App" to process the image
- The interface will show both original and processed images
- Progress indicators will keep you informed of the generation process
- View the generated application at http://localhost:3000
draw-an-app/
├── src/ # Source code directory
│ ├── main_v2.py # Webcam-based application
│ ├── main_streamlit.py # Streamlit-based application
│ └── utils/ # Utility modules
│ ├── aws_utils.py # AWS-related utilities
│ ├── camera_utils.py # Camera handling utilities
│ ├── image_utils.py # Image processing utilities
│ └── project_utils.py # Project-specific utilities
│ ├── prompt_system.txt # System prompt for Claude
│ ├── prompt_assistant.txt # Assistant prompt for Claude
│ ├── requirements.txt # Python dependencies
├── nextjs-app-template/ # Template for generated web applications
└── blank-nextjs-app/ # Working directory for generated applications
-
Node.js Issues:
- Verify Node.js installation:
node --version
- Clear npm cache:
npm cache clean --force
- Delete node_modules and reinstall:
rm -rf node_modules && npm install
- Verify Node.js installation:
-
Webcam Issues (for webcam-based flow):
- Ensure your webcam is properly connected
- Check if other applications are using the webcam
- Try different camera indices if multiple cameras are present
-
AWS Bedrock Access:
- Verify AWS credentials are correctly set
- Ensure you have access to the Claude 3 Sonnet model
- Check AWS region configuration
-
Python Environment:
- Ensure virtual environment is activated
- Verify all dependencies are installed correctly
- Check Python version compatibility
-
Streamlit Interface Issues:
- Ensure Streamlit is installed correctly (
pip install streamlit
) - Check if the uploaded image is in a supported format
- Verify the image file size is not too large
- Ensure Streamlit is installed correctly (
This code sample is licensed under the MIT-0 License. See the LICENSE file.
- Changelog of the project.
- License of the project.
- Code of Conduct of the project.
- CONTRIBUTING for more information.
Before you begin using the solution, there are certain precautions you must take into account:
-
Cost Management with Bedrock: Be mindful of the costs associated with AWS resources.
-
This is a sample: the code provided in this repository shouldn't be used for production workloads without further reviews and adaptation.