-
Notifications
You must be signed in to change notification settings - Fork 0
feat: implement Google Drive uploader and image processing pipeline f… #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| SESSIONIZE_API_SLUG=prcjw6ue | ||
| GDRIVE_CREDENTIALS_PATH=path/to/service-account.json | ||
| GDRIVE_FOLDER_ID=your-google-drive-folder-id | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: Generate Speaker Cards | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 6 * * *' | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| generate-and-upload: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v5 | ||
| with: | ||
| enable-cache: true | ||
| cache-dependency-glob: "uv.lock" | ||
|
|
||
| - name: Install Python | ||
| run: uv python install | ||
|
|
||
| - name: Install dependencies | ||
| run: uv sync --locked | ||
|
|
||
| - name: Write Google credentials file | ||
| # We store the JSON string in a GitHub Secret and write it to a temp file | ||
| # This file is NEVER committed to the repository. | ||
| run: echo '${{ secrets.GDRIVE_CREDENTIALS_JSON }}' > /tmp/gdrive-creds.json | ||
|
|
||
| - name: Generate cards and upload | ||
| env: | ||
| SESSIONIZE_API_SLUG: ${{ secrets.SESSIONIZE_API_SLUG }} | ||
| GDRIVE_CREDENTIALS_PATH: /tmp/gdrive-creds.json | ||
| GDRIVE_FOLDER_ID: ${{ secrets.GDRIVE_FOLDER_ID }} | ||
| run: uv run python src/generate_cards.py --upload | ||
|
|
||
| - name: Cleanup credentials | ||
| if: always() | ||
| run: rm -f /tmp/gdrive-creds.json |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,6 @@ | ||
| node_modules/ | ||
| __pycache__/ | ||
| .venv/ | ||
| *.pyc | ||
| .env | ||
| output/ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # AGENTS.md | ||
|
|
||
| ## 🤖 Persona & Description | ||
|
|
||
| **Name:** `card-generator-agent` | ||
| **Description:** Automates the generation of DevBcn speaker cards by parsing REST API data, removing image backgrounds, and compositing text. | ||
| **Persona:** You are a senior Python developer specializing in image processing and automation. You write modular, clean, and type-hinted code using `requests`, `rembg`, and `Pillow`. | ||
|
|
||
| ## ⚡ Commands | ||
|
|
||
| * **Setup environment:** `uv sync` | ||
| * **Run generator:** `uv run python src/generate_cards.py` | ||
| * **Run specific test:** `uv run pytest tests/ -v` | ||
| * **Lint code:** `uv run flake8 src/` | ||
|
|
||
| ## 🚫 Boundaries | ||
|
|
||
| * **Read-Only Assets:** Never modify, resize, or overwrite the base template located at `assets/base_template.jpg`. | ||
| * **Output Restrictions:** Always save generated output images strictly to the `output/` directory. | ||
| * **Secrets:** Never hardcode API URLs or endpoints in the Python scripts. Always use `os.getenv()` and an `.env` file. | ||
| * **Logic Constraints:** Do not attempt to write custom background removal algorithms; always rely on the `rembg` library. | ||
|
|
||
| ## 📂 Project Structure | ||
|
|
||
| * `src/`: Core Python logic (`api_client.py`, `image_processor.py`, `main.py`). | ||
| * `assets/`: Static required files (`base_template.jpg`, `.ttf` fonts). | ||
| * `output/`: Directory for the final generated `.png` or `.jpg` speaker cards. | ||
| * `tests/`: Unit tests for API mocking and image manipulation logic. | ||
|
|
||
| ## ✍️ Code Style & Examples | ||
|
|
||
| Write modular functions with explicit Python type hinting. | ||
|
|
||
| **Good Output Example (Typography with Drop Shadow):** | ||
|
|
||
| ```python | ||
| from PIL import ImageDraw, ImageFont | ||
|
|
||
| def draw_text_with_shadow( | ||
| draw: ImageDraw.ImageDraw, | ||
| text: str, | ||
| position: tuple[int, int], | ||
| font: ImageFont.FreeTypeFont, | ||
| text_color: str = "white", | ||
| shadow_color: str = "black" | ||
| ) -> None: | ||
| x, y = position | ||
| draw.text((x + 2, y + 2), text, font=font, fill=shadow_color) | ||
| draw.text((x, y), text, font=font, fill=text_color) | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # Speaker Card Generator Pipeline | ||
|
|
||
| This document describes the data flow and logic of the speaker card generation pipeline. | ||
|
|
||
| ## Overview | ||
|
|
||
| The pipeline automates the creation of conference speaker cards by fetching data from Sessionize, removing backgrounds from speaker photos, and compositing them onto a template with word-wrapped title text. | ||
|
|
||
| ## Modules | ||
|
|
||
| ### `src/api_client.py` | ||
|
|
||
| - Fetches all data in a single request from the `/view/All` Sessionize endpoint. | ||
| - Maps speaker data to `Speaker` dataclasses. | ||
| - Filters and maps session data to `SessionCard` dataclasses. | ||
| - Skips service and plenum sessions. | ||
|
|
||
| ### `src/image_processor.py` | ||
|
|
||
| - **Background Removal**: Uses `rembg` with a shared session to efficiently remove backgrounds from profile pictures. | ||
| - **Normalization**: Crops transparent margins and resizes the subject to a standardized height (`500px`). | ||
| - **Compositing**: | ||
| - Handles single and dual speaker layouts. | ||
| - Positions subjects relative to an anchor line. | ||
| - Uses alpha channels as masks for clean blending. | ||
| - **Text Rendering**: | ||
| - join speaker names with " & ". | ||
| - Word-wraps the talk title to fit within safe margins. | ||
| - Applies a drop-shadow effect for legibility. | ||
|
|
||
| ### `src/gdrive_uploader.py` | ||
|
|
||
| - **Authentication**: Uses Google Service Account JSON credentials. | ||
| - **Upsert Logic**: Searches for existing files by name in the target folder. | ||
| - If found: Updates the existing file. | ||
| - If not found: Uploads as a new file. | ||
| - **MIME Type**: Enforces `image/png` for all uploads. | ||
|
|
||
| ### `src/generate_cards.py` | ||
|
|
||
| - Entry point that orchestrates the workflow. | ||
| - Loads configuration from `.env`. | ||
| - Iterates over session cards and saves the final PNGs to the `output/` directory. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Python 3.11+ | ||
| - `uv` for dependency management. | ||
| - ML model for `rembg` (downloaded automatically on first run, ~170MB). | ||
|
|
||
| ## Usage | ||
|
|
||
| 1. Place `base_template.png` and your font file in `assets/`. | ||
| 2. Configure coordinates in `src/image_processor.py`. | ||
| 3. Set `SESSIONIZE_API_SLUG` in `.env`. | ||
| 4. Run: `uv run python src/generate_cards.py`. | ||
| 5. For upload: `uv run python src/generate_cards.py --upload` (requires `.env` config). | ||
|
|
||
| ## CI/CD Automation | ||
|
|
||
| A GitHub Actions workflow (`.github/workflows/generate-cards.yml`) is configured to run daily at 06:00 UTC. | ||
|
|
||
| ### Required GitHub Secrets | ||
|
|
||
| - `SESSIONIZE_API_SLUG`: The Sessionize API slug. | ||
| - `GDRIVE_CREDENTIALS_JSON`: The full JSON content of the Google Service Account key. | ||
| - `GDRIVE_FOLDER_ID`: The ID of the target Google Drive folder. | ||
|
|
||
| ### Automation Logic | ||
|
|
||
| 1. **Caching**: Uses `setup-uv` to cache Python dependencies based on `uv.lock`. | ||
| 2. **Ephemeral Credentials**: The `GDRIVE_CREDENTIALS_JSON` secret is written to `/tmp/gdrive-creds.json` during the run and deleted afterward. | ||
| 3. **Daily Generation**: Automatically fetches current data from Sessionize and refreshes the cards in Google Drive. |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,38 @@ | ||||||||||||||||||||||||||||||
| [project] | ||||||||||||||||||||||||||||||
| name = "cover-maker" | ||||||||||||||||||||||||||||||
| version = "0.1.0" | ||||||||||||||||||||||||||||||
| description = "DevBcn Speaker Card Generator" | ||||||||||||||||||||||||||||||
| authors = [ | ||||||||||||||||||||||||||||||
| { name = "DevBcn Team" } | ||||||||||||||||||||||||||||||
| ] | ||||||||||||||||||||||||||||||
| requires-python = ">=3.11" | ||||||||||||||||||||||||||||||
| dependencies = [ | ||||||||||||||||||||||||||||||
| "requests>=2.31", | ||||||||||||||||||||||||||||||
| "rembg[cpu]>=2.0", | ||||||||||||||||||||||||||||||
| "Pillow>=10.0", | ||||||||||||||||||||||||||||||
| "python-dotenv>=1.0", | ||||||||||||||||||||||||||||||
| "google-api-python-client>=2.100", | ||||||||||||||||||||||||||||||
| "google-auth>=2.23", | ||||||||||||||||||||||||||||||
| ] | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [dependency-groups] | ||||||||||||||||||||||||||||||
| dev = [ | ||||||||||||||||||||||||||||||
| "pytest>=8.0", | ||||||||||||||||||||||||||||||
| "pytest-cov>=5.0", | ||||||||||||||||||||||||||||||
| "ruff>=0.4", | ||||||||||||||||||||||||||||||
| "responses>=0.25", | ||||||||||||||||||||||||||||||
| ] | ||||||||||||||||||||||||||||||
|
Comment on lines
+18
to
+24
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [tool.ruff] | ||||||||||||||||||||||||||||||
| line-length = 100 | ||||||||||||||||||||||||||||||
| target-version = "py311" | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [tool.ruff.lint] | ||||||||||||||||||||||||||||||
| select = ["E", "F", "I", "N", "UP", "ANN"] | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [tool.ruff.lint.per-file-ignores] | ||||||||||||||||||||||||||||||
| "tests/*" = ["ANN"] | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| [tool.pytest.ini_options] | ||||||||||||||||||||||||||||||
| addopts = "--cov=src --cov-report=term-missing --cov-fail-under=90" | ||||||||||||||||||||||||||||||
| testpaths = ["tests"] | ||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,42 @@ | ||||||||||||||||||||||||||||||
| import requests | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| from models import SessionCard, Speaker | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| SESSIONIZE_BASE_URL = "https://sessionize.com/api/v2" | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| def fetch_session_cards(api_slug: str) -> list[SessionCard]: | ||||||||||||||||||||||||||||||
| url = f"{SESSIONIZE_BASE_URL}/{api_slug}/view/All" | ||||||||||||||||||||||||||||||
| response = requests.get(url) | ||||||||||||||||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||||||||||||||||||||||||||
| response.raise_for_status() | ||||||||||||||||||||||||||||||
| data = response.json() | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| # Map speakers by ID | ||||||||||||||||||||||||||||||
| speaker_lookup: dict[str, Speaker] = {} | ||||||||||||||||||||||||||||||
| for speaker_data in data.get("speakers", []): | ||||||||||||||||||||||||||||||
| speaker_id = speaker_data.get("id") | ||||||||||||||||||||||||||||||
| speaker_lookup[speaker_id] = Speaker( | ||||||||||||||||||||||||||||||
| id=speaker_id, | ||||||||||||||||||||||||||||||
| full_name=speaker_data.get("fullName", ""), | ||||||||||||||||||||||||||||||
| profile_picture_url=speaker_data.get("profilePicture", ""), | ||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||
|
Comment on lines
+16
to
+22
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If a speaker from the API response lacks an
Suggested change
|
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| cards: list[SessionCard] = [] | ||||||||||||||||||||||||||||||
| for session_data in data.get("sessions", []): | ||||||||||||||||||||||||||||||
| # Skip service and plenum sessions | ||||||||||||||||||||||||||||||
| if session_data.get("isServiceSession") or session_data.get("isPlenumSession"): | ||||||||||||||||||||||||||||||
| continue | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| session_speakers = [] | ||||||||||||||||||||||||||||||
| for speaker_id in session_data.get("speakers", []): | ||||||||||||||||||||||||||||||
| if speaker_id in speaker_lookup: | ||||||||||||||||||||||||||||||
| session_speakers.append(speaker_lookup[speaker_id]) | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| if session_speakers: | ||||||||||||||||||||||||||||||
| cards.append( | ||||||||||||||||||||||||||||||
| SessionCard( | ||||||||||||||||||||||||||||||
| talk_title=session_data.get("title", ""), speakers=tuple(session_speakers) | ||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| return cards | ||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| import logging | ||
| import os | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| from pathlib import Path | ||
|
|
||
| from google.oauth2 import service_account | ||
| from googleapiclient.discovery import build | ||
| from googleapiclient.http import MediaFileUpload | ||
|
|
||
| # Configure logging | ||
| logging.basicConfig(level=logging.INFO) | ||
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def upload_output_folder( | ||
| credentials_path: str, folder_id: str, output_dir: Path = Path("output") | ||
| ) -> int: | ||
| """ | ||
| Uploads all PNG files from output_dir to a Google Drive folder. | ||
| Updates existing files if name matches. | ||
| """ | ||
| if not output_dir.exists(): | ||
| logger.warning(f"Output directory {output_dir} does not exist. Skipping upload.") | ||
| return 0 | ||
|
|
||
| # 1. Authenticate | ||
| try: | ||
| creds = service_account.Credentials.from_service_account_file( | ||
| credentials_path, scopes=["https://www.googleapis.com/auth/drive.file"] | ||
| ) | ||
| service = build("drive", "v3", credentials=creds) | ||
| except Exception as e: | ||
| logger.error(f"Failed to authenticate with Google Drive: {e}") | ||
| raise | ||
|
Comment on lines
+31
to
+33
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Catching a broad |
||
|
|
||
| # 2. Get existing files in folder to avoid duplicates (upsert logic) | ||
| query = f"'{folder_id}' in parents and trashed = false" | ||
| try: | ||
| results = ( | ||
| service.files() | ||
| .list(q=query, spaces="drive", fields="files(id, name)") | ||
| .execute() | ||
| ) | ||
| except Exception as e: | ||
| logger.error(f"Failed to list files in Google Drive folder {folder_id}: {e}") | ||
| raise | ||
|
|
||
| existing_files = {f["name"]: f["id"] for f in results.get("files", [])} | ||
|
|
||
| upload_count = 0 | ||
| # 3. Iterate over PNGs in output/ | ||
| for file_path in output_dir.glob("*.png"): | ||
| file_name = file_path.name | ||
| media = MediaFileUpload(str(file_path), mimetype="image/png") | ||
|
|
||
| try: | ||
| if file_name in existing_files: | ||
| # Update existing file | ||
| file_id = existing_files[file_name] | ||
| logger.info(f"Updating existing file: {file_name} (ID: {file_id})") | ||
| service.files().update(fileId=file_id, media_body=media).execute() | ||
| else: | ||
| # Create new file | ||
| logger.info(f"Uploading new file: {file_name}") | ||
| file_metadata = {"name": file_name, "parents": [folder_id]} | ||
| service.files().create( | ||
| body=file_metadata, media_body=media, fields="id" | ||
| ).execute() | ||
|
|
||
| upload_count += 1 | ||
| except Exception as e: | ||
| logger.error(f"Error uploading {file_name}: {e}") | ||
|
|
||
| return upload_count | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
SESSIONIZE_API_SLUGappears to contain a real value. To prevent accidental leaks of production or staging identifiers, it's best practice to use a generic placeholder in example files, for exampleyour-sessionize-slug.