Organize social profiles, download media, and run recurring syncs — on your Windows machine.

Supports Instagram · TikTok · X / Twitter · optional Chrome Companion

 

_{App v0.31.0 · portable · checksums · all releases · changelog}

_{Companion 0.19.0 · extension ZIP · companion releases}

_{NinjaCrawler desktop workspace.}

---

NinjaCrawler is a local-first Windows app for operators who manage many social profiles: sync media to disk, queue downloads per provider, schedule work, and import browser sessions without sending secrets to third-party servers.

Note

Pre-1.0 software. Configuration formats, provider behavior, and migrations may change before a stable release.

Highlights

Profiles & media

• Manage provider accounts and tracked profiles in a native Windows workspace
• Download and catalog media from Instagram, TikTok, and X/Twitter
• Browse by profile and date, open the original post, or reveal the file on disk

Queues & scheduler

• Persistent download queues per provider — pause, resume, cancel, retry, reorder
• Scheduler sets, plans, groups, filters, and date constraints
• Sync options configurable per account and per profile

Companion & accounts

• NinjaCrawler Companion (Chrome): add profiles, queue syncs, import the signed-in browser account, capture stories
• Session cookies stay on the machine and travel only over loopback to the desktop app

Import & connectors

• Import an existing on-disk media library without duplicating files
• Managed connector runtimes (gallery-dl, yt-dlp, Instaloader) downloaded and verified by the app

Getting started

Desktop app

1. Download NinjaCrawler for Windows (setup installer).
2. Prefer no install? Use the standalone portable executable.
3. Optional: verify the file against SHA256SUMS.txt.

Warning

Builds are currently unsigned. Windows SmartScreen may show an unknown-publisher warning — choose More info → Run anyway only if you trust this repository.

On first launch the app downloads and verifies connector runtimes (internet required once). Application data lives under %LOCALAPPDATA%\NinjaCrawler. Portable mode only skips installing the app binary; data paths stay the same.

Chrome Companion (optional)

1. Download the Companion ZIP (also attached to desktop app releases).
2. Extract it. The archive always contains a stable NinjaCrawler-Companion folder.
3. Start NinjaCrawler, open chrome://extensions, enable Developer mode, then Load unpacked and select that folder.

Recommended install path for updates: load unpacked from
%LOCALAPPDATA%\NinjaCrawler\Companion after NinjaCrawler stages a release there, then use Reload extension in the Companion popup (or Chrome’s Reload).

Manual updates: extract a new ZIP over the loaded folder and click Reload on chrome://extensions.

See Chrome Companion for behavior details.

Supported providers

Provider	Current scope
Instagram	Multiple accounts, posts, reels, stories, highlights, tagged media, saved posts, profile metadata, targeted story downloads
TikTok	Multiple accounts, videos, photo posts, stories, reposts, avatars, date ranges, configurable naming
X / Twitter	Profile media timeline, avatars, duplicate prevention, handle recovery via stable user IDs

Behavior depends on the platform, authentication, rate limits, and managed connector capabilities.

How it works

Stack: Rust · Tauri 2 · React · TypeScript. Metadata lives in local SQLite; media files stay on disk where you can open them directly.

Chrome Companion (optional)
      │  loopback API  (127.0.0.1)
      ▼
React workspace
      │
      ▼
Tauri command bridge
      │
      ▼
Rust application runtime ─── SQLite workspace
      │
      ├── provider queues and scheduler
      ├── internal provider connectors
      └── managed external tools
              │
              ▼
        Media folders on disk

Providers are compiled into the app (not a drop-in plugin ABI). External tools provide extraction; queueing, rules, persistence, and UI stay in NinjaCrawler.

Chrome Companion

The extension in NinjaCrawler.Companion bridges Chrome and the desktop app:

• Detect supported profile tabs and add a selected batch
• Queue sync for the active profile
• Import the signed-in browser account (cookies stay local)
• Download the selected Instagram or TikTok story when the URL/media id is known
• Themes, keyboard shortcuts, and update guidance when a newer Companion is available
• With NinjaCrawler running: Download to AppData stages the ZIP under %LOCALAPPDATA%\NinjaCrawler\Companion, then Reload extension applies it when that folder is the loaded path

The extension talks only to http://127.0.0.1:47219. Session material is stored in NinjaCrawler’s protected session store.

Desktop app releases (vX.Y.Z) co-ship the Companion ZIP from that commit. Companion-only releases (companion-vX.Y.Z) ship extension updates independently; in-app update links use the Companion track.

More detail: Companion README · account import · release packaging.

Requirements

Runtime (end users):

• Windows 10 or Windows 11 (x64)
• Microsoft Edge WebView2 Runtime
• Internet access on first launch (connector download)

Development builds also need:

• Node.js LTS and npm
• Rust stable with the MSVC target
• Visual Studio 2022 Build Tools — Desktop development with C++
• PowerShell 5.1 or newer

Development

git clone https://github.com/JustShinobi/NinjaCrawler.git
cd NinjaCrawler
npm ci
Tools\Dev-Desktop.cmd

Frontend only (native Tauri commands unavailable):

npm run dev

Validation

npm run lint
npm test
npm run build

Full desktop build:

powershell -ExecutionPolicy Bypass -File Tools\Build-NinjaCrawler.ps1 -Configuration Debug

Release build + smoke test before publishing:

powershell -ExecutionPolicy Bypass -File Tools\Build-NinjaCrawler.ps1 -Configuration Release
powershell -ExecutionPolicy Bypass -File Tools\SmokeTest-NinjaCrawler.ps1 -Configuration Release

Artifacts:

src-tauri\target\release\                 # portable exe
src-tauri\target\release\bundle\          # installers

Use -PortableOnly when installers are not required.

Continuous integration and releases

GitHub Actions runs frontend quality on hosted ubuntu-latest and a Windows x64 cross-build for trusted PRs on self-hosted runners.

Pull requests get a merge-method label (merge:squash vs merge:merge-commit). Feature work into develop is squash; promote/release paths use merge commits. See merge policy.

The desktop app and Chrome Companion use independent Release Please tracks:

Track	Version files	Tag
App	package.json, tauri.conf.json, Cargo.toml	vX.Y.Z
Companion	NinjaCrawler.Companion/manifest.json (only Companion paths)	companion-vX.Y.Z

1. Merge Conventional Commits from develop → main.
2. Release Please opens release PR(s) for the track(s) that changed.
3. Merging a release PR creates a draft GitHub Release and dispatches the matching publish workflow.

App release assets: changelog, portable exe, NSIS setup, Companion ZIP from the release tree, SHA-256 sums.

Companion release assets: extension ZIP + checksums (canonical for in-app update links).

Versions below 1.0.0 publish as GitHub prereleases. An existing tag can be republished from the Release workflow.

Local data

What	Default path
App data	%LOCALAPPDATA%\NinjaCrawler\
SQLite DB	%LOCALAPPDATA%\NinjaCrawler\data\ninjacrawler.db
Staged Companion	%LOCALAPPDATA%\NinjaCrawler\Companion\
Media root	%USERPROFILE%\Pictures\NinjaCrawler\

Media and provider paths can be changed in the app. Back up the database and media folders before migrations or destructive maintenance. Authentication material is local-only and must not be committed.

Repository layout

Path	Purpose
src/	React workspace, windows, state, bridge, frontend tests
src-tauri/	Rust backend, SQLite, providers, queues, scheduler
connectors/manifest.json	Pinned connector versions and release assets
NinjaCrawler.Companion/	Chrome Companion extension
assets/	Brand and documentation images
Tools/	Dev, build, smoke-test, and publish scripts
docs/	Architecture, distribution, Companion, merge policy

Additional documentation

• Architecture
• Provider account flow
• Windows distribution
• Companion release packaging
• Companion account import
• Merge policy