feat: add CountEntries to skip decryption for list --detail by moonD4rk · Pull Request #562 · moonD4rk/HackBrowserData

moonD4rk · 2026-04-07T13:57:08Z

Summary

Closes #549

list --detail previously called Extract() for each browser, which performed full extraction (copy files → retrieve master key → decrypt all data) just to count entries via len(). This was wasteful and produced misleading zero counts when the master key was unavailable.

This PR adds a CountEntries() method to the Browser interface that counts entries without decryption:

SQLite categories (password, cookie, history, download, creditcard): SELECT COUNT(*) FROM <table>
JSON categories (bookmark, extension): parse and count elements without decrypting values
LevelDB categories (localStorage, sessionStorage): iterate keys without decoding values

Key changes:

Add CountRows utility to utils/sqliteutil for scalar count queries
Add per-category count* functions alongside existing extract* functions in both browser/chromium/ and browser/firefox/
Add CountEntries() + countCategory() methods to both chromium.Browser and firefox.Browser
Update printDetail() in list.go to use CountEntries() instead of Extract()
SQL count queries defined as constants next to their extract query counterparts
Count functions handle Opera extensions (opsettings) and Yandex browser variants

Test plan

Added count unit tests for all 9 Chromium categories (password, cookie, history, download, creditcard, bookmark, extension, localStorage, sessionStorage)
Added count unit tests for all 7 Firefox categories (password, cookie, history, download, bookmark, extension, localStorage)
Each category tests both populated and empty data scenarios
Extract and count tests share the same test fixtures via setup* functions
All existing tests pass unchanged
go build ./... passes
golangci-lint run passes (no new issues)

list --detail previously called Extract() which performed full decryption just to count entries. Add CountEntries() that uses SELECT COUNT(*) queries, JSON element counting, and LevelDB key iteration — no master key needed. This fixes misleading zero counts when the master key is unavailable and significantly improves performance for the detail listing.

codecov-commenter · 2026-04-07T13:58:39Z

Codecov Report

❌ Patch coverage is 75.87940% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.28%. Comparing base (5f42d4f) to head (7a7a9e9).

Files with missing lines	Patch %	Lines
browser/chromium/chromium.go	70.73%	9 Missing and 3 partials ⚠️
browser/firefox/firefox.go	65.71%	9 Missing and 3 partials ⚠️
browser/chromium/extract_storage.go	73.33%	4 Missing and 4 partials ⚠️
browser/chromium/extract_extension.go	76.00%	3 Missing and 3 partials ⚠️
utils/sqliteutil/query.go	71.42%	2 Missing and 2 partials ⚠️
browser/chromium/extract_bookmark.go	90.00%	1 Missing and 1 partial ⚠️
browser/firefox/extract_extension.go	77.77%	1 Missing and 1 partial ⚠️
browser/firefox/extract_password.go	60.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #562      +/-   ##
==========================================
+ Coverage   72.47%   73.28%   +0.80%     
==========================================
  Files          48       48              
  Lines        1751     1950     +199     
==========================================
+ Hits         1269     1429     +160     
- Misses        364      384      +20     
- Partials      118      137      +19

Flag	Coverage Δ
unittests	`73.28% <75.87%> (+0.80%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Adds a lightweight per-category counting pathway so list --detail can display entry counts without requiring master-key retrieval or decrypting browser data, addressing misleading zero counts and unnecessary work.

Changes:

Introduce Browser.CountEntries() and update list --detail to use it instead of Extract().
Add sqliteutil.CountRows() plus per-category count* helpers for Chromium and Firefox (SQLite/JSON/LevelDB).
Add/extend unit tests to cover count behavior for supported categories across both browser implementations.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
utils/sqliteutil/query.go	Adds `CountRows` helper for scalar `COUNT(*)` queries.
cmd/hack-browser-data/list.go	Switches detail listing to use `CountEntries` counts per category.
browser/browser.go	Extends `Browser` interface with `CountEntries`.
browser/chromium/chromium.go	Implements `CountEntries` and category routing for Chromium-based browsers.
browser/chromium/extract_storage.go	Adds LevelDB-based counting for local/session storage.
browser/chromium/extract_storage_test.go	Adds unit tests for storage counting; refactors fixtures.
browser/chromium/extract_password.go	Adds SQLite password count query + `countPasswords`.
browser/chromium/extract_password_test.go	Adds count tests; refactors DB setup helper.
browser/chromium/extract_history.go	Adds history count query + `countHistories`.
browser/chromium/extract_history_test.go	Adds count tests; refactors DB setup helper.
browser/chromium/extract_extension.go	Adds JSON-based extension counting (incl. Opera key path).
browser/chromium/extract_extension_test.go	Adds count tests; refactors JSON setup helper.
browser/chromium/extract_download.go	Adds download count query + `countDownloads`.
browser/chromium/extract_download_test.go	Adds count tests; refactors DB setup helper.
browser/chromium/extract_creditcard.go	Adds credit-card count query + `countCreditCards`.
browser/chromium/extract_creditcard_test.go	Adds count tests; refactors DB setup helper.
browser/chromium/extract_cookie.go	Adds cookie count query + `countCookies`.
browser/chromium/extract_cookie_test.go	Adds count tests; refactors DB setup helper.
browser/chromium/extract_bookmark.go	Adds JSON bookmark counting via recursive traversal.
browser/chromium/extract_bookmark_test.go	Adds count tests; refactors JSON setup helper.
browser/firefox/firefox.go	Implements `CountEntries` and category routing for Firefox.
browser/firefox/extract_storage.go	Adds SQLite localStorage count query + `countLocalStorage`.
browser/firefox/extract_storage_test.go	Adds localStorage count tests; refactors DB setup helper.
browser/firefox/extract_password.go	Adds JSON-based password counting from `logins` array.
browser/firefox/extract_password_test.go	Adds count tests.
browser/firefox/extract_history.go	Adds history count query + `countHistories`.
browser/firefox/extract_history_test.go	Adds count tests; refactors DB setup helper.
browser/firefox/extract_extension.go	Adds extension counting filtered to `location == app-profile`.
browser/firefox/extract_extension_test.go	Adds count tests; refactors JSON setup helper.
browser/firefox/extract_download.go	Adds download count query + `countDownloads`.
browser/firefox/extract_download_test.go	Adds count tests; refactors DB setup helper.
browser/firefox/extract_cookie.go	Adds cookie count query + `countCookies`.
browser/firefox/extract_cookie_test.go	Adds count tests; refactors DB setup helper.
browser/firefox/extract_bookmark.go	Adds bookmark count query + `countBookmarks`.
browser/firefox/extract_bookmark_test.go	Adds count tests; refactors DB setup helper.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T14:01:12Z

utils/sqliteutil/query.go

+// CountRows runs a scalar count query (e.g. SELECT COUNT(*) FROM ...) and
+// returns the integer result. It reuses the same database-open logic as
+// QuerySQLite (file existence check, optional journal_mode=off).
+func CountRows(dbPath string, journalOff bool, query string) (int, error) {
+	var count int
+	err := QuerySQLite(dbPath, journalOff, query, func(rows *sql.Rows) error {
+		return rows.Scan(&count)
+	})
+	if err != nil {
+		return 0, fmt.Errorf("count rows: %w", err)
+	}
+	return count, nil


CountRows uses QuerySQLite, but QuerySQLite intentionally swallows scanFn errors (logs at debug and continues). That means rows.Scan errors (e.g., type/overflow issues on 32-bit when COUNT(*) exceeds int) will be silently ignored and CountRows may return 0 with nil error, producing misleading counts. Consider implementing CountRows with a fail-fast QueryRow/Scan path (reusing the same file existence + optional PRAGMA logic), and scan into int64 with explicit range checking before converting to int.

Copilot · 2026-04-07T14:01:12Z

browser/chromium/extract_storage.go

+		if bytes.Equal(key, []byte(localStorageVersionKey)) {
+			continue
+		}
+		count++


countLocalStorage currently counts every LevelDB key except VERSION. extractLocalStorage/parseLocalStorageEntry only includes META/METAACCESS and '_' data keys, and skips any unrecognized keys; with the current logic, CountEntries can overcount if the DB contains additional internal keys. To keep list --detail consistent with Extract(), filter using the same key classification (e.g., call parseLocalStorageEntry on iter.Key()/iter.Value() and count only when ok==true).

Suggested change

count++

if _, ok := parseLocalStorageEntry(key, iter.Value()); ok {

count++

}

Add integration tests for CountEntries() method and unit tests for countCategory() dispatch in both chromium_test.go and firefox_test.go, matching the existing Extract/extractCategory test patterns.

Copilot

Pull request overview

Copilot reviewed 39 out of 39 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T14:11:01Z

browser/chromium/extract_storage.go

+func countLocalStorage(path string) (int, error) {
+	if _, err := os.Stat(path); err != nil {
+		return 0, err
+	}
+	db, err := leveldb.OpenFile(path, nil)
+	if err != nil {
+		return 0, err
+	}
+	defer db.Close()
+
+	var count int
+	iter := db.NewIterator(nil, nil)
+	defer iter.Release()
+
+	for iter.Next() {
+		key := iter.Key()
+		if bytes.Equal(key, []byte(localStorageVersionKey)) {
+			continue
+		}
+		count++
+	}
+	return count, iter.Error()


countLocalStorage currently counts all LevelDB keys except VERSION. This can diverge from extractLocalStorage, which ignores unrecognized keys via parseLocalStorageEntry (and only treats METAACCESS:, META:, and '_' data keys as entries). To keep list --detail counts consistent with extracted entry semantics and avoid over-counting future/unknown metadata keys, consider counting only recognized key patterns (METAACCESS:/META:/data prefix) instead of every non-VERSION key.

- CountRows: use db.QueryRow instead of QuerySQLite to fail fast on scan errors instead of silently returning 0. - countLocalStorage: use parseLocalStorageEntry for key classification to match extractLocalStorage filtering (skip unrecognized keys).

Cover normal count, empty table, file not found, and journalOff mode.

Copilot

Pull request overview

Copilot reviewed 40 out of 40 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T14:26:31Z

utils/sqliteutil/query.go

+	var count int
+	if err := db.QueryRow(query).Scan(&count); err != nil {
+		return 0, fmt.Errorf("count rows: %w", err)
+	}


CountRows scans COUNT(*) into an int, but SQLite returns an integer type that can exceed 32-bit limits. On 32-bit builds (or very large profiles) this can overflow silently. Consider scanning into int64 (or uint64) and then converting to int with a bounds check (or return int64 from CountRows).

Copilot AI review requested due to automatic review settings April 7, 2026 13:57

Copilot started reviewing on behalf of moonD4rk April 7, 2026 13:57 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

test: add CountEntries and countCategory tests at browser level

060f489

Add integration tests for CountEntries() method and unit tests for countCategory() dispatch in both chromium_test.go and firefox_test.go, matching the existing Extract/extractCategory test patterns.

moonD4rk requested a review from Copilot April 7, 2026 14:06

Copilot started reviewing on behalf of moonD4rk April 7, 2026 14:07 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

moonD4rk added 2 commits April 7, 2026 22:16

test: add CountRows unit tests

7a7a9e9

Cover normal count, empty table, file not found, and journalOff mode.

moonD4rk requested a review from Copilot April 7, 2026 14:22

Copilot started reviewing on behalf of moonD4rk April 7, 2026 14:23 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

moonD4rk merged commit b3bbc0d into main Apr 7, 2026
13 checks passed

moonD4rk deleted the fix/list-detail-count-without-decryption branch April 7, 2026 14:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add CountEntries to skip decryption for list --detail#562

feat: add CountEntries to skip decryption for list --detail#562
moonD4rk merged 4 commits intomainfrom
fix/list-detail-count-without-decryption

moonD4rk commented Apr 7, 2026

Uh oh!

codecov-commenter commented Apr 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-		count++
+		if _, ok := parseLocalStorageEntry(key, iter.Value()); ok {
+			count++
+		}

Conversation

moonD4rk commented Apr 7, 2026

Summary

Test plan

Uh oh!

codecov-commenter commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented Apr 7, 2026 •

edited

Loading