Add parallel IMAP fetch and --full option

[linyows]

Summary

• Parallelize IMAP message fetching using multiple threads with independent connections, auto-scaled to CPU core count (max 16)
• Add --full flag to re-fetch all messages ignoring previously fetched history
• Fix progress display by adding ANSI erase-to-end-of-line (\x1b[K) after \r

Benchmark (224 messages)

Version	Time
main (sequential)	1m 53.7s
parallel-fetch	32.8s

~3.5x faster

Test plan

• reports fetch --full re-fetches all messages
• reports fetch incremental fetch works as before
• Progress display overwrites cleanly without leftover characters

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds a parallel IMAP fetching implementation (multi-threaded with multiple IMAP connections), introduces a --full option to re-fetch all messages regardless of prior history, and improves the terminal progress display cleanup.

Changes:

• Add new src/fetch.zig module implementing parallel IMAP fetch + result processing helpers.
• Wire parallel fetch + --full flag into the CLI fetch path and update progress rendering.
• Update C-ABI fetch path to reuse the new fetch module.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
src/root.zig	Exposes the new fetch module from the library root.
src/main.zig	Adds --full flag handling and switches CLI fetch to the new parallel fetch implementation with progress display tweaks.
src/lib.zig	Refactors exported reports_fetch to use the new parallel fetch + shared processing logic.
src/fetch.zig	New module: worker thread orchestration, result collection, parsing/saving reports, and cleanup utilities.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Ignoring allocation errors from new_uids.append(...) can silently skip UIDs and miss reports. Prefer handling the error (e.g., stop processing this account / return a failure code) rather than catch {} so the caller can react appropriately.

[Copilot]

When --full is used, UIDs that are already present in fetched_set will be processed again, and markUidFetched will append duplicates to the .fetched_uids file. Over time this can grow without bound and slow down loadFetchedUids. Consider only calling markUidFetched when the UID was not already in the set (e.g., check contains before writing, or use getOrPut and only write on first insert).

Suggested change

	st.markUidFetched(r.uid);
	fetched_set.put(r.uid, {}) catch {};
	if (!fetched_set.contains(r.uid)) {
	st.markUidFetched(r.uid);
	fetched_set.put(r.uid, {}) catch {};
	}

[Copilot]

worker returns immediately if client.connect() fails, leaving the assigned results entries uninitialized and (when used with the CLI progress loop) preventing progress from ever reaching the total. This can cause the main thread to wait forever and silently skip batches. Consider filling results for all assigned UIDs (with .uid = uid, .data = null) and incrementing progress even on connect failure, or propagating the failure so the caller can abort and clean up.

Suggested change

	client.connect() catch return;
	client.connect() catch {
	for (ctx.uids, 0..) |uid, i| {
	ctx.results[i] = .{
	.uid = uid,
	.data = null,
	};
	if (ctx.progress) |p| {
	_ = p.fetchAdd(1, .monotonic);
	}
	}
	return;
	};

[Copilot]

std.Thread.spawn(...) catch continue; drops the entire batch on spawn failure (offset is still advanced) and also means progress may never reach uids.len, which can deadlock the CLI progress loop. Handle spawn failure by either (a) running worker synchronously for that batch, (b) returning null/error after freeing results, or (c) retrying with fewer workers while ensuring every UID is assigned to exactly one worker.

Suggested change

	job.threads[job.spawned] = std.Thread.spawn(.{}, worker, .{&job.contexts[job.spawned]}) catch continue;
	job.threads[job.spawned] = std.Thread.spawn(.{}, worker, .{&job.contexts[job.spawned]}) catch {
	worker(&job.contexts[job.spawned]);
	offset += this_batch;
	continue;
	};

[Copilot]

startFetch executes allocations and imap.Client operations on multiple threads using the passed-in allocator. In the CLI this is a GeneralPurposeAllocator, which is not thread-safe by default, so this can introduce data races/memory corruption. Use a thread-safe allocator for the worker threads (e.g., wrap with std.heap.ThreadSafeAllocator and pass that allocator consistently to startFetch and freeResults, or allocate per-worker with std.heap.c_allocator).

Suggested change

	var progress = std.atomic.Value(usize).init(0);
	var job = reports.fetch.startFetch(allocator, acct, new_uid_slice, &progress) orelse return .{ .dmarc = 0, .tls = 0 };
	defer reports.fetch.freeResults(allocator, job.results);
	var thread_safe_allocator = std.heap.ThreadSafeAllocator{ .child_allocator = allocator };
	const fetch_allocator = thread_safe_allocator.allocator();
	var progress = std.atomic.Value(usize).init(0);
	var job = reports.fetch.startFetch(fetch_allocator, acct, new_uid_slice, &progress) orelse return .{ .dmarc = 0, .tls = 0 };
	defer reports.fetch.freeResults(fetch_allocator, job.results);

[Copilot]

Ignoring allocation errors from new_uids.append(...) can silently skip UIDs (and thus skip fetching/saving reports) under memory pressure. Instead of catch {}, handle the error explicitly (e.g., abort the fetch for this account with a message / return early) so failures are visible and behavior is deterministic.