Consolidate to single virtual environment and add cube inspection script

.gitignore

@@ -40,3 +40,10 @@ wheels/
 # OS
 .DS_Store
 Thumbs.db
+
+# Script outputs
+output_*.html
+output_*.json
+
+# UV lock file
+uv.lock

DOCS.md

@@ -2,21 +2,19 @@
 
 ## Development Environment
 
-This project uses [UV](https://github.com/astral-sh/uv), a fast Python package manager, to manage virtual environments and dependencies for different components:
+This project uses [UV](https://github.com/astral-sh/uv), a fast Python package manager, to manage the virtual environment and all Python dependencies.
 
-- **Backend virtual environment** (`backend/.venv/`) - Contains FastAPI and related dependencies
-- **Scripts virtual environment** (`scripts/.venv/`) - Contains scraping and data processing dependencies
+- **Virtual environment** (`.venv/`) - Single shared environment containing all dependencies
 
 **Package management:**
-- Dependencies are managed via `pyproject.toml` files in each component
+- Dependencies are managed via `pyproject.toml` in the project root
 - UV provides fast dependency resolution and installation
-- Virtual environments are created using `uv venv` and dependencies installed with `uv sync`
+- Virtual environment is created using `uv venv` and dependencies installed with `uv sync`
 
-**Setup scripts:**
-- `setup_backend.sh` - Automated setup for backend virtual environment using UV
-- `setup_scripts.sh` - Automated setup for scripts virtual environment using UV
+**Setup script:**
+- `setup.sh` - Automated setup for the virtual environment using UV
 
-Virtual environments are excluded from version control via `.gitignore`.
+Virtual environment is excluded from version control via `.gitignore`.
 
 ## Folder Structure
 
@@ -27,8 +25,6 @@ Contains data collection and processing scripts.
 
 - **`/scripts/scrapers`** - Web scraping scripts for CubeCobra data
 - **`/scripts/data_processing`** - Data transformation and cleaning scripts
-- **`/scripts/.venv`** - Virtual environment for scripts (not committed to git)
-- **`/scripts/pyproject.toml`** - Python dependencies and project metadata
 
 #### Dependencies
 - **beautifulsoup4** - HTML parsing for web scraping
@@ -46,9 +42,7 @@ FastAPI application serving the recommender engine.
   - **`/backend/app/core`** - Core configuration (database, settings)
   - **`/backend/app/services`** - Business logic and services
 - **`/backend/tests`** - Backend test suite
-- **`/backend/.venv`** - Virtual environment for backend (not committed to git)
 - **`/backend/run.py`** - Development server script
-- **`/backend/pyproject.toml`** - Python dependencies and project metadata
 
 #### Technology Stack
 - **FastAPI** - Modern async web framework

README.md

@@ -24,48 +24,35 @@ The Backend is written in Python using FastAPI and is providing the content for
 - Python 3.8 or higher
 - Node.js 16 or higher
 - npm or yarn
-- UV package manager (installed automatically by setup scripts)
+- UV package manager (installed automatically by setup script)
 
 ## Virtual Environment Setup
 
-This project uses [UV](https://github.com/astral-sh/uv), a fast Python package manager, to manage virtual environments and dependencies for both the backend and scripts.
+This project uses [UV](https://github.com/astral-sh/uv), a fast Python package manager, to manage the virtual environment and all Python dependencies.
 
 ### Quick Setup (Recommended)
 
-Run the setup scripts from the project root:
+Run the setup script from the project root:
 
 ```bash
-# Setup backend virtual environment
-./setup_backend.sh
-
-# Setup scripts virtual environment
-./setup_scripts.sh
+./setup.sh
 ```
 
-The setup scripts will automatically install UV if not already present.
+The setup script will automatically install UV if not already present, create a virtual environment, and install all dependencies.
 
 ### Manual Setup with UV
 
 If you prefer to set up manually:
 
-#### Install UV
 ```bash
+# Install UV
 curl -LsSf https://astral.sh/uv/install.sh | sh
-```
 
-#### Backend Virtual Environment
-```bash
-cd backend
+# Create virtual environment and install dependencies
 uv venv
 uv sync
-source .venv/bin/activate  # On Windows: .venv\Scripts\activate
-```
 
-#### Scripts Virtual Environment
-```bash
-cd scripts
-uv venv
-uv sync
+# Activate the virtual environment
 source .venv/bin/activate  # On Windows: .venv\Scripts\activate
 ```
 
@@ -90,8 +77,11 @@ The backend is built with FastAPI and SQLModel.
 **Important:** Make sure to activate the virtual environment first!
 
 ```bash
-cd backend
+# Activate the virtual environment
 source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+
+# Run the backend
+cd backend
 python run.py
 ```
 

backend/pyproject.toml → pyproject.toml

@@ -1,10 +1,10 @@
 [project]
-name = "cubepython-backend"
+name = "cubepython"
 version = "0.1.0"
-description = "Backend API for CubePython MTG cube recommender"
-readme = "README.md"
+description = "MTG Cube recommender engine with backend API and data scraping"
 requires-python = ">=3.8"
 dependencies = [
+    # Backend dependencies
     "fastapi==0.115.5",
     "uvicorn[standard]==0.32.1",
     "sqlmodel==0.0.22",
@@ -14,8 +14,9 @@ dependencies = [
     "httpx==0.28.1",
     "pytest==8.3.4",
     "pytest-asyncio==0.24.0",
+    # Scripts dependencies
+    "beautifulsoup4>=4.12.0",
+    "requests>=2.31.0",
+    "selenium>=4.15.0",
+    "python-dotenv>=1.0.0",
 ]
-
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"

scripts/inspect_cube.py

@@ -0,0 +1,62 @@
+"""
+Simple script to inspect and dump the contents of a CubeCobra cube page.
+"""
+
+import json
+import requests
+from bs4 import BeautifulSoup
+
+# Configuration
+CUBE_ID = '1fdv1'
+URL = f'https://cubecobra.com/cube/overview/{CUBE_ID}'
+OUTPUT_HTML = f'output_{CUBE_ID}.html'
+OUTPUT_JSON = f'output_{CUBE_ID}.json'
+
+def main():
+    print(f"Fetching: {URL}")
+
+    headers = {
+        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
+    }
+
+    # Fetch the page
+    response = requests.get(URL, headers=headers)
+    response.raise_for_status()
+
+    print(f"Status: {response.status_code}")
+    print(f"Content-Length: {len(response.content)} bytes")
+
+    # Parse with BeautifulSoup
+    soup = BeautifulSoup(response.text, 'html.parser')
+
+    # Extract some basic info
+    info = {
+        'url': URL,
+        'title': soup.title.string if soup.title else None,
+        'meta_tags': {},
+    }
+
+    for meta in soup.find_all('meta'):
+        if meta.get('name'):
+            info['meta_tags'][meta.get('name')] = meta.get('content')
+        elif meta.get('property'):
+            info['meta_tags'][meta.get('property')] = meta.get('content')
+
+    # Save raw HTML
+    with open(OUTPUT_HTML, 'w', encoding='utf-8') as f:
+        f.write(response.text)
+    print(f"\nSaved HTML to: {OUTPUT_HTML}")
+
+    # Save parsed info as JSON
+    with open(OUTPUT_JSON, 'w', encoding='utf-8') as f:
+        json.dump(info, f, indent=2)
+    print(f"Saved info to: {OUTPUT_JSON}")
+
+    # Print summary
+    print(f"\nPage Title: {info['title']}")
+    print(f"Meta Tags: {len(info['meta_tags'])} found")
+
+    print("\nDone!")
+
+if __name__ == '__main__':
+    main()

setup_scripts.sh → setup.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
-# Setup script for scripts virtual environment using UV
+# Setup script for CubePython virtual environment using UV
 
-echo "Setting up scripts virtual environment with UV..."
+echo "Setting up CubePython virtual environment with UV..."
 
 # Check if UV is installed
 if ! command -v uv &> /dev/null; then
@@ -11,8 +11,6 @@ if ! command -v uv &> /dev/null; then
     exit 1
 fi
 
-cd scripts
-
 # Check if venv exists
 if [ -d ".venv" ]; then
     echo "Virtual environment already exists. Syncing dependencies..."
@@ -24,5 +22,5 @@ else
 fi
 
 echo ""
-echo "Scripts virtual environment setup complete!"
-echo "To activate: cd scripts && source .venv/bin/activate"
+echo "Virtual environment setup complete!"
+echo "To activate: source .venv/bin/activate"