v1.7.13 - Token-based crawl + diagnostic error modal

Why miamidesigngroup.com was failing

Your 3846-page site was hitting a wall because every crawl batch POSTed the full URL list (~300KB of JSON). With ~97 batches × 5 concurrent, that's many MB of repeated payload getting sent to your server in parallel. Cloudflare, nginx `client_max_body_size`, or a WAF can reject these bursts with a 4xx/5xx and the plugin was showing a generic "Error scanning images" alert that hid the real cause.

What v1.7.13 does

1. Token-based URL handoff

`crawl_discover` now stores the full URL list server-side under a one-hour transient token. `crawl_batch` requests carry only `{token, offset, batch_size}` — about 100 bytes instead of 300KB. That's a ~3000x payload reduction per batch request.

2. Detailed error modal

When any phase of the scan fails you now get a modal showing:

• Which phase failed: `cache_status`, `discover`, `crawl_batch`, `cache_finalize`, or `scan_final`
• HTTP status + status text
• First 2000 chars of the server response body (often reveals the actual PHP fatal / Cloudflare page / firewall block)
• Human-readable interpretation for common codes (413 payload too large, 502/503/504 server busy, 524 Cloudflare timeout, 500 PHP fatal, 403 WAF, etc.)
• Copy details button so you can paste the exact error into an issue or support ticket

The full jqXHR + response is also logged to the browser console.

3. Fail-tolerant parallel crawl

A single 40-URL batch returning 500 or timing out no longer kills the whole scan. Failed batches are tracked and the scan continues with the remaining ones. The scan only aborts if all batches fail.

Next step for you

1. Install v1.7.13 (either via Check Now or upload the ZIP from this release).
2. Click the `[refresh]` link on the crawl cache badge so it retries with the token-based flow.
3. If a scan still fails, the modal now tells you exactly what — copy the details and send them over.

v1.7.12 - Restore UX redesign + history + abort

What's in this release

Five upgrades driven by your end-to-end restore test on v1.7.11.

1. Restore runs one image at a time

Batch size dropped from 5 to 1. Every single restored image triggers a progress bar move and a UI update — no more waiting 15 seconds to see the bar advance.

2. Redesigned restore progress card

Replaced the tiny hairline bar and tiny gray text with a proper card:

• Purple refresh icon on the left
• Title "Restoring images from backup" + live subtitle
• Big live percentage (e.g. "67%") on the right
• 12px gradient progress bar
• Footer with count ("45 / 250"), a thumbnail preview of the most recently restored image with its filename, and an ID preserved / new ID badge
• Stop button on the right

3. Abort a restore mid-run

Stop button on the restore card. Confirms first, then flips an abort flag so no new batches launch. Anything that was restored before you clicked Stop stays in the Media Library and shows up in the completion modal — clearly labeled as a stopped run.

4. Restore history

Every restore run (normal or stopped) is saved to `wp_options.hozio_restore_history` (last 20 runs). A new panel under the progress card lists each run with:

• Date/time
• Pills: restored count, IDs preserved, remapped, failed, stopped
• View button that re-opens the exact same thumbnail-grid modal the run ended with

Clear-history button wipes the log without touching any actual images.

5. Scan UI fixes

From your screenshots on v1.7.11:

• Stop button no longer explodes — a global `.hozio-empty-state .dashicons` rule was forcing 48px on every nested dashicon, including the Stop button's X. Now scoped to only direct children.
• Clean state "Your Media Library is Clean!" checkmark sits on a proper green circle instead of looking layered/transparent.
• Scanning state refresh icon is colored (indigo) on a light indigo background.
• Progress bar height doubled (6px → 10px) with a subtle shadow so you can actually see it move.

v1.7.11 - Restore UX + ID preservation, Stop button, refresh warning

What's in this release

Four fixes from an end-to-end scan → delete → restore cycle on v1.7.10.

1. Restore preserves original attachment IDs

Previously the restore created a new auto-increment attachment ID and then rewrote every database reference it could find. That works for featured images, WC galleries, JSON-quoted ID fields, and termmeta — but it can never touch hardcoded theme PHP, cached HTML (CDN / page-cache), or plugin tables that store raw integer IDs.

v1.7.11 uses WordPress's built-in `import_id` parameter on `wp_insert_attachment()`, which reclaims the original attachment ID slot whenever it's free. When preservation succeeds, every existing reference keeps working automatically — no rewriting needed. On ID conflict (rare), the old `update_id_references` fallback kicks in.

2. Restore progress bar moves

Batch size dropped from 20 to 5 images so the bar advances every 5-15 seconds instead of appearing frozen for up to a minute per batch. The label now shows the most recently restored filename and an ID status marker:

• `Restoring… 45 / 250 ✓ hero-banner.jpg` (ID preserved)
• `Restoring… 45 / 250 ↻ old-product.jpg` (ID remapped, DB updated)

3. Post-restore modal with thumbnail grid

Replaces the old alert + forced page reload. Shows:

• Total restored count
• Count that kept their original IDs
• Count that got new IDs (DB references updated)
• Failed count (details logged to browser console)
• Thumbnail grid of every restored image with a green-check or orange-arrow badge

You can Close (stay on the current page with current results intact) or Reload.

4. Stop button during scan

The scanning state now has a "Stop" button. Clicking it:

• Prevents any new crawl or DB batches from launching
• Lets in-flight AJAX complete (their results are discarded)
• Returns the UI to the initial state so you can try something different

5. Better refresh warning

Clicking `[refresh]` on the crawl-cache badge now shows a specific dialog:

Clear the frontend-crawl cache?

The next scan will re-crawl 342 pages. Typically takes about 14 seconds and briefly uses extra server resources.

Only refresh if you have added new pages or images since the last crawl.

Verification

1. Scan → delete 5 images → note their IDs. Upload the auto-downloaded ZIP. Modal should show "5 kept their original attachment ID". Media Library should show those exact IDs back.
2. During a fresh crawl, click Stop. Scan UI disappears, no further requests fire.
3. Click `[refresh]` — dialog tells you the exact page count.

v1.7.10 - Crawl speedups + honest progress count

Why your crawl was slow

Three compounding issues:

1. PHP session lock. Parallel AJAX requests from the browser were being serialized by the PHP session write lock (opened by WooCommerce or any other plugin that uses `$_SESSION`). Client-side parallelism was effectively capped at 1 for AJAX, even though the browser fired multiple requests.
2. Sub-optimal concurrency. 30-per-batch × 3 parallel = 90 in flight, but the session lock neutralized the 3-parallel part.
3. Honest-number confusion. The progress bar showed "180 of 1228" because the filter runs before crawling but the UI never displayed the pre-filter vs post-filter split clearly.

What v1.7.10 does

• `session_write_close()` at the top of every crawl AJAX handler releases the session lock immediately, letting the browser's 5 parallel batches actually run in parallel.
• Concurrency bumped to 5 client × 40 server = ~200 HTTP fetches in flight.
• Planning phase runs `discover_urls` first and shows exactly what will be crawled:

  "342 pages to crawl (886 of 1228 noise URLs filtered) — starting…"

• Cache persists until you click Refresh. No more 1-hour TTL. After the first successful crawl, every subsequent scan is instant until you explicitly refresh. Badge warns if the cache is more than 7 days old.

Expected speedup

If your site was showing 180 / 1228 after 60s (3 pages/sec), the session-lock fix alone should take you to ~15 pages/sec. With the higher concurrency, ~25-30 pages/sec. That same site should finish in ~45-60 seconds end-to-end instead of several minutes.

Will parallelism overload my server?

Realistically no — it's a one-time burst of ~30-60 seconds, only on your own site, and the cache means you won't trigger it again unless you click Refresh. On strict shared hosts real visitors may see briefly slower pageviews during the crawl, but nothing will break.

v1.7.9 - 5-10x faster crawl (noise filter + parallel batches)

Speedups

Two changes that compound to make the crawl 5-10x faster while staying accurate.

1. URL noise filter

Your sitemap is typically padded with URLs that either duplicate content already crawled or contain no HTML at all. The crawler now skips:

• Paginated archives (`/page/2/`, `/page/3/`, ...) — same post images as page 1
• Attachment pages (`/attachment/slug/`) — just one image each, nothing extra
• Author archives (`/author/name/`) — duplicate of post archives
• Feed URLs (`/feed/`, `/comments/feed/`, `/feed/atom/`, etc.) — XML, not HTML
• Trackback, embed, AMP endpoints — duplicate content
• `?replytocom=` comment anchors

On a typical site this drops ~1200 discovered URLs to ~300-400. The progress label now shows how many were skipped: "Crawling pages… 0 / 342 (886 noise URLs skipped)".

2. Parallel crawl batches from the browser

The JS crawl loop was doing one batch at a time. It now runs 3 batches in parallel, and since each server batch already does 30 concurrent HTTP fetches via curl_multi, total concurrency is now ~90 pages in flight simultaneously.

Expected result

A site that was showing "30 / 1228" (at a pace of roughly 30 pages per 3-4 seconds) should now finish in ~10-15 seconds on a typical 300-page site. The cache still holds the result for an hour, so subsequent scans within the hour remain ~1-2 seconds.

Accuracy unchanged

Every URL pattern in the noise filter is guaranteed to not contain any image reference that isn't already covered by another crawled page. A paginated archive page 2 shows different posts than page 1, but those posts are crawled individually via the post sitemap. An attachment page shows one image that's already covered by its parent post.

v1.7.8 - Never flag favicon / custom logo as unused

Fix

The site icon (favicon) set via Appearance → Customize → Site Identity → Site Icon is stored as a bare integer in the `site_icon` WordPress option. The scan's reference checks only matched JSON-quoted values (`"123"`), not bare integer values (`123`), so the favicon kept showing up as unused.

v1.7.8 explicitly pulls attachment IDs from the WordPress core options that store bare integers:

• `site_icon` (favicon)
• `theme_mods_.custom_logo` (theme custom logo)
• `theme_mods_.header_image_data.attachment_id` (theme header image)
• `theme_mods_.background_image` (theme background image)
• `woocommerce_placeholder_image` (WooCommerce placeholder)

Also added a `hozio_used_attachment_ids_from_options` filter so plugins/themes storing attachment IDs in their own options can register them for detection.

Apply to your site

After installing v1.7.8, click [refresh] on the crawl-cache badge (or wait for the 1-hour cache to expire) to re-run a clean scan. The site icon will no longer appear in the unused list.

v1.7.7 - Unified fast+accurate scan (crawl cache)

The scan that's both fast AND accurate

This release unifies the fast scan and the accurate scan into one, using a persistent crawl cache to get the best of both:

• First scan ever (or after cache expires): crawls your frontend once in ~20-40s with 30 parallel HTTP requests, stores the result in a DB option for 1 hour.
• All scans after that: ~1-2 seconds. The cached crawl data is fed into a single-request DB check and filters out anything that appears on the frontend. Still crawl-accurate.

Why this is finally deterministic

The previous batched DB scan ran preload queries across 6 separate HTTP requests. Any cross-request state (WP transients, object cache entries, WP-cron writes during the scan) could cause different batches to see different DB snapshots — that's where 26↔52 came from.

v1.7.7 runs the entire DB check inside a single PHP request via `scan_final`. There are no transient handoffs, no object-cache races. Same inputs always produce the same outputs.

UI changes

• Cache status badge above the results: "Frontend crawl cached 12m ago (215 pages) [refresh]"
• If no cache: "No crawl cache — first scan will crawl your site (~30s)."
• The separate "Verify all with frontend" button is gone — verification is now automatic (and cached).
• Per-image "Verify" button on each result still works for one-off spot checks.

Files

• `includes/class-frontend-crawler.php`: 1h cache in `hozio_frontend_crawl_cache` option; concurrency 30
• `includes/class-unused-detector.php`: new `scan_final()` one-shot method
• `admin/class-ajax-handler.php`: new actions `hozio_crawl_cache_status`, `hozio_crawl_cache_clear`, `hozio_crawl_cache_finalize`, `hozio_scan_final`
• `admin/views/backups.php`: rewritten scan JS with cache-aware flow + cache badge UI

v1.7.6 - Fix Check Now on object-cached hosts

Fix

The "Check Now" button in Settings was showing stale release data on sites with persistent object caches (Redis, Memcached, W3TC). Even after clicking "Check Now" the plugin would keep reporting an older version as "the latest" because the cache invalidation for `delete_transient()` wasn't propagating inside the same request.

What changed

• `force_update_check()` now purges every cache layer (transient, site transient, object cache) and hits the GitHub API with a cache-busting query parameter and `Cache-Control: no-cache` header.
• The AJAX handler uses the freshly-fetched release object directly, instead of reading back through WordPress's `update_plugins` transient.
• The confirmation message now shows the detected GitHub release version even when you're up to date — so you can confirm the check actually reached the network.
• On failure, you see a specific error message instead of a generic "Failed".

Why this kept biting us

WordPress transients on hosts with object caches have two layers: a persistent store (DB or Redis) and an in-request memory cache. `delete_transient()` invalidates both in most setups, but some caches (especially misconfigured Redis connections or W3TC) keep the in-memory value across the same HTTP request. Any subsequent `get_transient()` in that request returns the stale value. The fix is to never trust a cached read after a force-check — fetch directly from the upstream source.

v1.7.5 - Frontend crawl is opt-in + 10x faster

You control when the crawl runs

v1.7.4 ran the frontend crawl automatically on every scan, which made scans slow. v1.7.5 reverts that: scan is DB-only by default (~60 seconds for ~1800 images, same as v1.7.3).

Frontend verification is now a manual action that appears after the scan:

• Bulk: A banner above the results says "Extra safety check (optional)" with a Verify all with frontend crawl button. Click it when you want to eliminate false positives before deleting. This crawls every public page, cross-references with the candidates, and removes any image found on the frontend. Results are tallied at the end (N rescued, M confirmed unused).
• Per-image: The Verify button on each candidate crawls the frontend for just that one image and shows a clear modal with a pass/fail verdict and the exact pages where the image appears.

Use the bulk verify before a big delete; use the per-image verify when you're unsure about a specific result.

10x faster crawling

Switched from sequential wp_remote_get to curl_multi_exec for parallel HTTP fetches. A 200-page site now crawls in ~30-60 seconds instead of the ~7 minutes v1.7.4 would have taken. Graceful fallback to sequential fetching if curl extension isn't available.

What's unchanged from v1.7.4

• Crawler class (sitemap discovery, HTML extraction) — still used by the opt-in verify
• Stateless batched DB scan — no more Redis/Memcached race condition
• Per-image Verify button modal

Files changed

• admin/views/backups.php — restored fast scan JS, added Verify All button & banner
• includes/class-frontend-crawler.php — parallel curl_multi fetcher with sequential fallback
• hozio-image-optimizer.php — version 1.7.5

v1.7.4 - Frontend crawler + 100% accurate unused detection

Major Feature: Ground-truth unused-image detection

Database-only scanning could never guarantee 100% accuracy because page builders (Elementor, Divi, Oxygen, Bricks, Gutenberg) store image references in proprietary formats that aren't easy to parse reliably. v1.7.4 adds a frontend crawler that fetches every public page on your site and extracts all image URLs from the rendered HTML — the same HTML your visitors see.

An image is flagged unused ONLY if:

1. It does not appear in any rendered frontend page, AND
2. It has no database references

The intersection of these two signals eliminates the false positives that were causing real used images to be flagged for deletion.

Multi-phase scan

When you click "Scan Unused Images" the scan now runs four phases with progressive progress:

1. Discover — find every public URL (Yoast/RankMath/WP core sitemap, or DB fallback)
2. Crawl — fetch each page, extract every image URL from <img>, srcset, data-src, CSS url(), OG meta, JSON-LD
3. DB Scan — same DB reference check as before (featured images, galleries, post content, options, termmeta, JSON-quoted IDs)
4. Cross-reference — an image is flagged unused only if BOTH phases agree it's unused

Any image DB-flagged but found on the frontend is rescued (logged to console) and kept safe.

New Verify button

Each unused result has a Verify button that crawls every public page on your site for that one image and shows a clear modal:

• SAFE TO DELETE (green) — not found anywhere
• DO NOT DELETE (red) — list of pages where it appears, with links
• Full database reference list
• All URL variants checked (thumbnails, scaled versions, etc.)

Permanent fix for inconsistent scan counts

scan_batch() is now fully stateless — no cross-request transients. Object cache races (Redis/Memcached) can no longer produce different counts between consecutive scans.

Files changed

• NEW includes/class-frontend-crawler.php — sitemap parser, page crawler, HTML extractor
• includes/class-unused-detector.php — stateless batching + frontend-filter helper
• admin/class-ajax-handler.php — new actions for crawl, verify, cross-reference
• admin/views/backups.php — multi-phase scan JS, new Verify modal
• hozio-image-optimizer.php — version 1.7.4, loads crawler class

Notes

• First scan on large sites takes longer (~2-5 min for 200+ pages) because we're crawling the frontend. This is the cost of 100% accuracy.
• Images only visible behind login (members-only) won't be detected as "used" by the frontend crawler — the DB scan is the safety net for those.
• Pages that fail to load (timeouts, 404s, redirects) are logged to the browser console so you can review.