Side Scraper

English | 한국어

Navigation: Catchers | Installation | Output artifacts | Workflow | Examples | Why split | License

Side Scraper is a bundle of three Chrome extensions ("catchers") for turning a public social profile, an auto-scroll session, and one side-panel click into a clean CSV export of posts.

The catchers share the same UX, side-panel layout, and export schema. They differ only in the platform-specific scraping logic required to reliably extract posts from each site's DOM.

Catchers

linkedin-catcher

• Target: https://www.linkedin.com/*
• Description: Scroll a LinkedIn profile's recent activity and export posts with text, media, URL, and repost status to CSV.
• Permissions: activeTab, scripting, storage, downloads, sidePanel
• Entry points: side panel (popup.html), service worker (background.js), content script (content.js)

threads-catcher

• Target: https://www.threads.net/*, https://www.threads.com/*
• Description: Scroll a Threads profile and export posts with text, media, URL, repost status, and thread tag to CSV.
• Permissions: activeTab, scripting, storage, downloads, sidePanel
• Entry points: side panel (popup.html), service worker (background.js), content script (content.js)

x-catcher

• Target: https://x.com/*, https://twitter.com/*
• Description: Scroll an X (Twitter) profile and export posts with text, media, URL, repost status, and thread tag to CSV.
• Permissions: activeTab, scripting, storage, downloads, sidePanel
• Entry points: side panel (popup.html), service worker (background.js), content script (content.js), MAIN-world injected script (injected.js) loaded at document_start

Back to top

Installation

These are unpacked Manifest V3 Chrome extensions. Install each catcher independently.

Load unpacked in Chrome

1. Open chrome://extensions
2. Enable Developer mode with the top-right toggle
3. Click Load unpacked
4. Select one of:
   • side-scraper/linkedin-catcher
   • side-scraper/threads-catcher
   • side-scraper/x-catcher
5. Pin the extension to the toolbar

Repeat for each catcher you want installed.

Clone and install all three

git clone https://github.com/<owner>/side-scraper.git
cd side-scraper

Then load linkedin-catcher/, threads-catcher/, and x-catcher/ as unpacked extensions using the steps above.

Chrome discovers the extension from the folder containing manifest.json, so each catcher folder must keep its manifest.json at the folder root.

Back to top

Output artifacts

Each catcher produces:

1. a live in-panel preview of captured posts
2. a CSV file downloaded via chrome.downloads

CSV columns:

• text
• media
• url
• repost
• thread tag where applicable for Threads and X

Back to top

Recommended workflow

Use the catchers in this exact sequence per profile you want to capture.

1) Open the profile

Input:

• target profile URL on LinkedIn, Threads, or X

Output:

• a fully loaded profile page in the active tab

2) Open the side panel

Input:

• click the catcher's toolbar icon

Output:

• the side panel UI (popup.html) attached to the current tab, with live post counter and preview pane

3) Start the scroll

Input:

• press Start in the side panel

Output:

• the content script auto-scrolls the feed, deduplicates posts, and streams them into the preview pane in real time

4) Stop and export

Input:

• press Stop when the feed is exhausted or you have enough rows

Output:

• a CSV downloaded via chrome.downloads, containing one row per captured post

This step is intentionally separate from scraping so capture can:

• inspect each post as it streams in
• preserve dedup state across long scroll sessions
• save project-bound CSVs explicitly instead of leaving rows in extension storage

Back to top

Example usage

linkedin-catcher

Use when you want to archive a LinkedIn user's recent activity feed.

1. Navigate to https://www.linkedin.com/in/<handle>/recent-activity/all/
2. Click the LinkedIn Catcher icon -> side panel opens
3. Press Start -> auto-scroll begins
4. Press Stop when done -> CSV downloads

threads-catcher

Use when you want to archive a Threads profile's posts and reposts.

1. Navigate to https://www.threads.net/@<handle>
2. Click the Threads Catcher icon -> side panel opens
3. Press Start -> auto-scroll begins for threads.net and threads.com
4. Press Stop when done -> CSV downloads with thread tag populated

x-catcher

Use when you want to archive an X (Twitter) profile, including replies and reposts.

1. Navigate to https://x.com/<handle>
2. Click the X Catcher icon -> side panel opens
3. Press Start -> injected.js cooperates with content.js to capture posts
4. Press Stop when done -> CSV downloads with thread tag populated

Back to top

Why this is split into 3 extensions

A single all-in-one scraper fails in predictable ways:

• each platform virtualizes its feed differently
• selectors rot at different cadences per site
• permission scopes and host_permissions should be minimal per platform
• X requires a MAIN-world injected script to reach internal APIs; LinkedIn and Threads do not
• mixing all three into one extension would request access to every domain at once

So this repo separates responsibilities:

• linkedin-catcher scrapes a LinkedIn profile's recent activity feed
• threads-catcher scrapes a Threads profile on threads.net or threads.com
• x-catcher scrapes an X (Twitter) profile, including a MAIN-world injected helper

Back to top

License

Side Scraper is released under the MIT License.

Back to top