Skip to content

Release v0.8.0 - #54

Merged
hitalin merged 12 commits into
mainfrom
develop
Aug 3, 2026
Merged

Release v0.8.0#54
hitalin merged 12 commits into
mainfrom
develop

Conversation

@hitalin

@hitalin hitalin commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

変更一覧(v0.7.0 → v0.8.0)

破壊的変更

  • DB スキーマ V6: notes_cache.timeline_type を除去し、所属を note_timelines テーブルへ分離。初回起動時に自動移行(既存のノート・FTS・アカウント・チャットは保持。streaming が壊れたキーで書いた孤児行のみ削除 — 元々どのタイムライン読み出しにも乗っていない行)
  • TimelineType を廃止し TimelineKey に一本化(parse / as_canonical / api_endpoint / ws_channel
  • Database API: cache_notes / cache_noteingest_notes、読み出し系は &TimelineKey を取る、delete_cached_note は account スコープ、get_cached_timeline_before に keyset cursor 引数、remove_membership / clear_timeline / sweep_orphan_notes 新設
  • StreamingManager: subscribe_timeline / subscribe_antenna / subscribe_channel / subscribe_rolesubscribe_notes(TimelineKey) に統合
  • NoteDeckError::Apiapi_code フィールド追加(refactor: エラー型の粒度を細分化する #48
  • daemon: GET /api/{host}/timeline/{tl_type} は Basic キーのみ受理(パラメータ付きキーは 400)、InvalidInput は 500 → 400

新機能・修正

  • リスト TL の WS チャンネル名バグ修正(user-listTimelineuserList)、フォーク TL の kebab→lowerCamel 対応(vmimi-relayvmimiRelayTimeline
  • polling モードで antenna / channel / role / user-list も更新されるように
  • CLI: timeline antenna:{id} 等の prefix 付きキーに対応、notecli cache sweep サブコマンド新設
  • eviction に per_timeline_limit(バケット単位のトリム。チャンク分割 tx で長時間 writer 占有を回避)

更新時の注意

  • daemon を先に停止してから更新すること(稼働中の旧 daemon は実行時 "no such column" になる)
  • V6 適用は 1 回だけ長い(1M 行で 1 分前後・warn ログあり)。一時的に DB サイズの最大 2 倍弱の空きディスクが必要。途中で中断しても DB は無傷(再起動で再試行)
  • V6 適用済み DB は旧バージョンでは開けない。ダウングレードの可能性があるなら事前バックアップを推奨
  • 同一 DB を共有する notetui / notebot は対応版(rev bump 済み)に更新すること

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added support for expanded timeline types, including lists, antennas, channels, roles, clips, mentions, and favorites.
    • Added Hanami note search with date-based pagination.
    • Added a cache cleanup command for removing orphaned cached notes.
    • Added configurable per-timeline cache limits and improved timeline pagination.
  • Bug Fixes

    • Improved authentication and API error reporting with stable error codes.
    • Invalid timeline requests now return clearer client errors.
  • Improvements

    • Updated to version 0.8.0.

hitalin and others added 12 commits August 3, 2026 00:14
文字列ベースの Auth(String) は code が一律 "AUTH" になり、呼び出し側が
回復手段を選べなかった。また Misskey が返す error.code は Api の message
に埋め込まれるだけで、消費者側が文字列を parse して復元していた。

- Auth(String) → Auth(AuthErrorKind): NoToken / MiAuthFailed / MiAuthPending
  / MiAuthMalformed / SessionInvalid / CredentialMissing。code() は
  AUTH_ 接頭辞付きで variant ごとに分かれる
- Api に api_code: Option<String> を追加し、serialize に apiCode として出す。
  request() はサーバー由来の error.code をそのまま載せる
- WebSocket(String) を削除(生成箇所がゼロの死んだ variant)
- Internal(String) を追加。認証と無関係な内部不整合の受け皿とし、
  safe_message() では詳細を伏せる

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
refactor: エラー型の粒度を細分化する
接続 / 購読追加・削除 / 中断・再開 / モード切替に状態遷移のテストが無く、
#700 (ログアウトと購読リトライの競合) の系統が再発しても検知できなかった。

connect() は初回接続に失敗してもハンドルを残すので、到達不能な host を
使えば実サーバーなしで StreamingManager の表 (connections / subscriptions
/ captured_notes) の遷移を通せる。以下を追加した:

- connect の冪等性と、冪等 return 時の実状態 emit (復帰時の取り逃し補正)
- disconnect が切断したアカウントの購読と capture だけを消すこと (#777)
- 中断・再開が metadata を捨てずに active だけを倒すこと、二重操作が no-op
- 他アカウントの購読 ID を渡しても中断・再開できないこと
- モード切替が購読を保ったまま輸送路だけ差し替えること
- note capture が明示的に外すまで残り、空になったらエントリごと消えること

各テストは対応する実装のガードを外すと落ちることを確認済み (9 種の
ミューテーション)。接続なしの購読が NO_CONNECTION になる経路だけは
get_host と send_subscribe の二重ガードで、片方だけ外しても振る舞いは
保たれる。

Refs: #877

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test: ストリーミング接続ライフサイクルの状態遷移を固める
はなみすきー (hanamisskey/misskey) はロールポリシー canSearchNotes が
false で本家 notes/search が常に UNAVAILABLE を返すため、独自エンドポイント
notes/hanamisearch-v1 を叩く search_notes_hanami() を追加する。

このエンドポイントは sinceDate/untilDate を受け付けないが、サーバー側で
sinceId/untilId を idService.parse() で createdAt に戻して比較しているため、
日付を Misskey ID の時刻部 (base36 8 桁) に変換して渡すことで代替する。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat: はなみすきーの notes/hanamisearch-v1 に対応
判定ロジックが呼び出し側にしかない検索のための API を足す。この層は
「FTS で粗く絞って行を読み、述語に渡す」だけで、述語の意味論には関与しない。
notedeck のカラムクエリ (#783 Phase 3) が最初の利用者になる。

既存の search_cached_notes_advanced はユーザーの検索文字列 1 本を前提に
しているため、別関数として追加した。

- fts_literals は AND 結合で FTS5 に押し込む。trigram が成立しない 3 文字未満は
  落とす (押し込むと 0 件になり偽陰性を生む)。押し込めるものが無ければ全件走査
- max_scanned_rows で走査を打ち切り、継続カーソルを返す。巨大キャッシュでも
  応答が返らなくならないようにする
- カーソルは「最後に走査した行」を指す。最後にマッチした行を指すと、その間の
  マッチしない行を再開時に読み直すことになる
- ORDER BY は created_at + note_id の複合。created_at が同値でも順序が定まり、
  カーソル反復で取りこぼしと重複が出ない
- 述語が None を返した行と、note_json として読めなかった行は per-note エラーと
  して件数に計上し、ノートは返さない
- DB ロックはチャンク単位で取り直す。述語の評価はロックの外で行うので、重い
  述語が他の DB 利用者を待たせない

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(db): キャッシュ済みノートの述語注入スキャン API
notes キャッシュを entity (notes_cache) と membership (note_timelines) に
分離し、同一ノートの複数タイムライン所属と REST/streaming のキー不一致
(孤児化) を解消する。

- V6 migration: 壊れキー行 DELETE → note_timelines (WITHOUT ROWID + FK
  CASCADE) 新設 → INSERT SELECT → DROP COLUMN timeline_type → ANALYZE。
  refinery set_grouped(true) で履歴記録と単一 tx 化 (中断安全)
- TimelineKey: タイムラインキーの正本型。parse (splitn(2)・予約語・
  256B・制御文字検証) / canonical / api_endpoint / ws_channel
  (kebab→lowerCamel fallback で vmimi-relay 等のフォーク TL 購読を修正、
  userList チャンネル名バグも解消)。TimelineType は廃止
- ingest_notes: entity upsert + membership upsert の唯一の書込経路。
  streaming / polling も同経路に統一し antenna/channel/role/user-list の
  streaming 受信分が読み出しに乗るようになる
- 読み出し: membership JOIN + keyset cursor (sort_key, note_id) で
  同一時刻多発時のページング前進を保証
- eviction: per_timeline_limit 新設 (チャンク分割 tx で writer 長期占有を
  回避)、remove_membership / clear_timeline / sweep_orphan_notes 追加、
  delete_cached_note を account スコープ化
- 起動フロー: busy_timeout/journal_size_limit 追加、wal_checkpoint
  (TRUNCATE) 2 点 + PRAGMA optimize (CASCADE の stat1 要件)
- daemon: get_timeline を Basic キー allowlist 化、InvalidInput→400
- CLI: timeline の prefix 付きキー対応、cache sweep サブコマンド新設

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(db): ノート実体とタイムライン所属の分離 (V6 note_timelines)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The package moves to version 0.8.0. It introduces typed timeline keys, separates cached note entities from timeline memberships, updates REST and streaming flows, adds structured errors, and provides cache orphan cleanup.

Changes

Timeline cache and API modernization

Layer / File(s) Summary
Typed timeline contract and API integration
src/models.rs, src/api.rs, src/http_server.rs, src/commands/notes.rs
TimelineKey replaces TimelineType. REST, WebSocket, CLI, Hanami search, and pagination logic use the typed key.
Separated cache entities and memberships
migrations/V6__split_note_timeline_membership.sql, src/db.rs
The V6 migration adds note_timelines. Database ingestion, pagination, trimming, clearing, scanning, and orphan cleanup use separate entity and membership records.
Typed streaming subscriptions
src/streaming.rs
Subscriptions store typed targets and derive channels, parameters, timeline keys, reconnect behavior, and note ingestion from those targets.
Structured authentication and API errors
src/error.rs, src/lib.rs, src/api.rs
Authentication errors use typed variants and stable codes. API errors preserve optional server codes. Internal error details are sanitized before frontend output.
Cache sweep command and formatting updates
src/cli.rs, src/commands/mod.rs, src/commands/auth.rs, src/commands/doctor.rs, src/main.rs, Cargo.toml
The CLI adds cache sweep for orphan-note cleanup. The package version changes to 0.8.0. Several existing expressions are reformatted without behavior changes.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant TimelineKey
  participant MisskeyClient
  participant StreamingManager
  participant Database
  CLI->>TimelineKey: parse timeline identifier
  TimelineKey->>MisskeyClient: provide endpoint and parameters
  MisskeyClient-->>Database: ingest REST notes with TimelineKey
  StreamingManager->>TimelineKey: derive typed subscription key
  StreamingManager-->>Database: ingest streaming notes with TimelineKey
  Database-->>CLI: return timeline data or sweep count
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning Hanami search, authentication error restructuring, and other release-wide features are not covered by issue #30. Move unrelated features to separate issues or link additional issues that define their scope.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title identifies the v0.8.0 release, which matches the package version change and the stated release objective.
Linked Issues check ✅ Passed The changes separate note entities from memberships, adopt TimelineKey, unify ingest, and add membership-level trimming and orphan cleanup required by issue #30.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch develop

Comment @coderabbitai help to get the list of available commands.

@hitalin hitalin self-assigned this Aug 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (1)
src/http_server.rs (1)

143-158: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Use safe_message() for the outward error body.

The new status mapping documents why e.to_string() is safe for InvalidInput. The same message field still carries e.to_string() for every other variant, including Api and Keychain, whose Display output comes from upstream text. The coding guidelines require safe_message() for error messages so a token cannot reach a log or a response.

🛡️ Proposed change
         Self {
             status,
             code,
-            message: e.to_string(),
+            message: e.safe_message(),
         }

As per coding guidelines: 「API トークンをログやエラーメッセージに含めず、safe_message() を使用する。」

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/http_server.rs` around lines 143 - 158, Update the From<NoteDeckError>
for ApiError implementation to populate message using e.safe_message() instead
of e.to_string(), while preserving the existing status and code mapping.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/api.rs`:
- Around line 909-945: Update the shared search dispatch used by
commands/notes.rs::run_search and http_server.rs::search_notes so
Hanami/Misskey-compatible servers call ApiClient::search_notes_hanami instead of
search_notes. Reuse the server-software detection mechanism already used by both
callers, preserving the existing search_notes path for other software and
ensuring the selection is centralized rather than duplicated.

In `@src/commands/mod.rs`:
- Around line 44-52: Update the cache sweep branch in run_cli to honor the fmt
OutputFormat parameter: retain the existing human-readable message for the
default format, and emit a structured deletion-count record for
OutputFormat::Json and OutputFormat::Jsonl. Add CLI coverage verifying both
default output and JSON output, including the deleted count.

In `@src/db.rs`:
- Around line 1041-1074: Refactor cleanup_with_eviction and
trim_timelines_chunked so the helper accepts &self rather than a held
&Connection, and acquire the writer lock separately inside each chunk iteration.
Remove the long-lived lock_write guard from cleanup_with_eviction while
preserving TTL handling and deletion totals, ensuring the mutex is released
after every chunk so ingest_notes and other writers can interleave.

In `@src/error.rs`:
- Around line 56-57: Rename the canonical application error enum in error.rs
from NoteDeckError to NotecliError, then migrate all callers, imports, return
types, and references to use NotecliError consistently as the public error
boundary.
- Around line 71-73: Update the HTTP error adapter in http_server.rs,
specifically the response construction around the message field, to use
NoteDeckError::safe_message() instead of e.to_string(). Preserve the existing
status and response handling while ensuring Internal errors return the sanitized
message.

In `@src/models.rs`:
- Around line 513-559: Update TimelineKey::parse so the fallback Basic branch
accepts only non-empty names composed of lowercase ASCII letters, digits, and
hyphens. Reject any other character with NoteDeckError::InvalidInput before
constructing Basic, while preserving the existing prefixed-key parsing and
reserved-name handling.

---

Nitpick comments:
In `@src/http_server.rs`:
- Around line 143-158: Update the From<NoteDeckError> for ApiError
implementation to populate message using e.safe_message() instead of
e.to_string(), while preserving the existing status and code mapping.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b5968b5e-29ee-4db2-a950-f6e26a6cdeca

📥 Commits

Reviewing files that changed from the base of the PR and between e235701 and 114850d.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (15)
  • Cargo.toml
  • migrations/V6__split_note_timeline_membership.sql
  • src/api.rs
  • src/cli.rs
  • src/commands/auth.rs
  • src/commands/doctor.rs
  • src/commands/mod.rs
  • src/commands/notes.rs
  • src/db.rs
  • src/error.rs
  • src/http_server.rs
  • src/lib.rs
  • src/main.rs
  • src/models.rs
  • src/streaming.rs

Comment thread src/api.rs
Comment on lines +909 to +945
/// はなみすきー (hanamisskey/misskey) 独自のノート検索。
///
/// 本家 `notes/search` はロールポリシー `canSearchNotes` で無効化されており、
/// 代わりに `notes/hanamisearch-v1` が開放されている。このエンドポイントは
/// sinceDate/untilDate を受け付けないため、日付は ID の時刻部に変換して渡す。
/// ページング用の sinceId/untilId が明示されていればそちらを優先する
/// (untilId で遡る 2 ページ目以降は日付の上限より必ず古いため)。
pub async fn search_notes_hanami(
&self,
host: &str,
token: &str,
account_id: &str,
query: &str,
options: SearchOptions,
) -> Result<Vec<NormalizedNote>, NoteDeckError> {
let mut params = json!({ "query": query, "limit": options.limit() });
let since_id = options
.since_id
.clone()
.or_else(|| options.since_date.map(misskey_id_at));
let until_id = options
.until_id
.clone()
.or_else(|| options.until_date.map(misskey_id_at));
apply_pagination(&mut params, since_id.as_deref(), until_id.as_deref());
if let Some(ref uid) = options.user_id {
params["userId"] = json!(uid);
}
let data = self
.request(host, token, "notes/hanamisearch-v1", params)
.await?;
let raw: Vec<RawNote> = serde_json::from_value(data)?;
Ok(raw
.into_iter()
.map(|n| n.normalize(account_id, host))
.collect())
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Find callers of search_notes_hanami and how the endpoint is selected.
set -euo pipefail

rg -nP -C5 '\bsearch_notes_hanami\s*\(' --type=rust
# Check for fork/software detection near search dispatch
rg -nP -C4 'hanami' --type=rust

Repository: notedeck-dev/notecli

Length of output: 158


🏁 Script executed:

set -euo pipefail
printf '%s\n' '--- matching files ---'
git ls-files | rg '(^|/)(api|notes)\.rs$|src/commands'

printf '%s\n' '--- search_notes_hanami references ---'
rg -n -C3 'search_notes_hanami|search_notes\s*\(' --glob '*.rs' .

printf '%s\n' '--- endpoint and software detection references ---'
rg -n -C4 'notes/search|hanami|software|version|instance' --glob '*.rs' src

Repository: notedeck-dev/notecli

Length of output: 41397


🏁 Script executed:

set -euo pipefail
printf '%s\n' '--- search dispatch ---'
sed -n '1,105p' src/commands/notes.rs
sed -n '490,525p' src/http_server.rs

printf '%s\n' '--- detection consumers and software branches ---'
rg -n -C3 'ServerDetection|get_server_detection|load_server_detections|software_name|software_repository|account\.software|\.software' src --glob '*.rs'

printf '%s\n' '--- direct-call verifier ---'
python3 - <<'PY'
from pathlib import Path
import re

files = [p for p in Path("src").rglob("*.rs")]
hanami = []
normal = []
software_branches = []
for p in files:
    text = p.read_text()
    for m in re.finditer(r'\bsearch_notes_hanami\s*\(', text):
        line = text.count("\n", 0, m.start()) + 1
        hanami.append((str(p), line, text.splitlines()[line-1].strip()))
    for m in re.finditer(r'\.search_notes\s*\(', text):
        line = text.count("\n", 0, m.start()) + 1
        normal.append((str(p), line, text.splitlines()[line-1].strip()))
    for i, line in enumerate(text.splitlines(), 1):
        if re.search(r'(?i)hanami', line) or re.search(r'software_(name|repository)|account\.software', line):
            software_branches.append((str(p), i, line.strip()))

print("search_notes_hanami references:")
for row in hanami:
    print(row)
print("search_notes call sites:")
for row in normal:
    print(row)
print("Hanami/software-related lines:")
for row in software_branches:
    print(row)
PY

Repository: notedeck-dev/notecli

Length of output: 25051


Route Hanami searches to search_notes_hanami.

src/commands/notes.rs::run_search and src/http_server.rs::search_notes always call search_notes. No production caller selects search_notes_hanami; Hanami instances still use the disabled notes/search endpoint. Add shared server-software dispatch.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/api.rs` around lines 909 - 945, Update the shared search dispatch used by
commands/notes.rs::run_search and http_server.rs::search_notes so
Hanami/Misskey-compatible servers call ApiClient::search_notes_hanami instead of
search_notes. Reuse the server-software detection mechanism already used by both
callers, preserving the existing search_notes path for other software and
ensuring the selection is centralized rather than duplicated.

Comment thread src/commands/mod.rs
Comment on lines +44 to +52
Commands::Cache(cache_cmd) => {
return match cache_cmd {
crate::cli::CacheCommands::Sweep => {
let deleted = db.sweep_orphan_notes()?;
println!("Removed {deleted} orphan note(s) from cache");
Ok(())
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Honor OutputFormat for cache sweep.

run_cli receives fmt, but this branch always prints plain text. When the caller selects OutputFormat::Json or OutputFormat::Jsonl, the command returns invalid machine-readable output. Match fmt and emit a structured deletion count for JSON modes. Add a CLI test for default and JSON output.

Suggested fix
-                    println!("Removed {deleted} orphan note(s) from cache");
+                    match fmt {
+                        OutputFormat::Json | OutputFormat::Jsonl => {
+                            println!(r#"{{"deleted":{deleted}}}"#);
+                        }
+                        _ => println!("Removed {deleted} orphan note(s) from cache"),
+                    }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/mod.rs` around lines 44 - 52, Update the cache sweep branch in
run_cli to honor the fmt OutputFormat parameter: retain the existing
human-readable message for the default format, and emit a structured
deletion-count record for OutputFormat::Json and OutputFormat::Jsonl. Add CLI
coverage verifying both default output and JSON output, including the deleted
count.

Comment thread src/db.rs
Comment on lines 1041 to +1074
pub fn cleanup_with_eviction(&self, config: &EvictionConfig) -> Result<u64, NoteDeckError> {
// どちらも無効なら早期 return (lock も取らない)。
if config.per_account_limit.is_none() && config.ttl_days.is_none() {
// 全フィールド無効なら早期 return (lock も取らない)。
// per_timeline_limit を含む 3 フィールド判定であること (2 フィールド判定だと
// per-timeline のみ設定時にトリムが走らない)。
if config.per_account_limit.is_none()
&& config.ttl_days.is_none()
&& config.per_timeline_limit.is_none()
{
return Ok(0);
}

let conn = self.lock_write()?;
let tx = conn.unchecked_transaction()?;
let mut total_deleted: u64 = 0;

// ① TTL (単一 tx)
if let Some(ttl_days) = config.ttl_days {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
let ttl_cutoff = now - ttl_days * 86_400;
let tx = conn.unchecked_transaction()?;
let n = tx.execute(
"DELETE FROM notes_cache WHERE cached_at < ?1",
params![ttl_cutoff],
)?;
tx.commit()?;
total_deleted += n as u64;
}

// ② per-timeline トリム (チャンク分割 tx)
if let Some(per_timeline_limit) = config.per_timeline_limit {
total_deleted += self.trim_timelines_chunked(&conn, per_timeline_limit)?;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift

Chunked trimming still blocks every in-process writer for the whole run.

cleanup_with_eviction acquires the writer MutexGuard at line 1052 and holds it until the function returns. trim_timelines_chunked receives that &Connection and only splits the SQLite transaction. The Mutex is never released between chunks, so ingest_notes and every other lock_write caller in this process waits for the complete trim, which the comments describe as taking minutes on a first run with 1M rows.

Pass &self into the trim helper and take the writer lock once per chunk. Each chunk then yields the mutex and lets streaming ingestion interleave.

♻️ Proposed restructure
-        let conn = self.lock_write()?;
         let mut total_deleted: u64 = 0;
 
         // ① TTL (単一 tx)
         if let Some(ttl_days) = config.ttl_days {
+            let conn = self.lock_write()?;
             ...
         }
 
         // ② per-timeline トリム (チャンク分割 tx)
         if let Some(per_timeline_limit) = config.per_timeline_limit {
-            total_deleted += self.trim_timelines_chunked(&conn, per_timeline_limit)?;
+            total_deleted += self.trim_timelines_chunked(per_timeline_limit)?;
         }
 
         // ③ per-account hard cap (単一 tx)
         if let Some(per_account_limit) = config.per_account_limit {
+            let conn = self.lock_write()?;
             let tx = conn.unchecked_transaction()?;
-    fn trim_timelines_chunked(
-        &self,
-        conn: &Connection,
-        per_timeline_limit: i64,
-    ) -> Result<u64, NoteDeckError> {
+    fn trim_timelines_chunked(&self, per_timeline_limit: i64) -> Result<u64, NoteDeckError> {
         let mut total: u64 = 0;
         loop {
+            // チャンクごとに writer lock を取り直し、WS ingest を割り込ませる
+            let conn = self.lock_write()?;
             let tx = conn.unchecked_transaction()?;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
pub fn cleanup_with_eviction(&self, config: &EvictionConfig) -> Result<u64, NoteDeckError> {
// どちらも無効なら早期 return (lock も取らない)。
if config.per_account_limit.is_none() && config.ttl_days.is_none() {
// 全フィールド無効なら早期 return (lock も取らない)。
// per_timeline_limit を含む 3 フィールド判定であること (2 フィールド判定だと
// per-timeline のみ設定時にトリムが走らない)。
if config.per_account_limit.is_none()
&& config.ttl_days.is_none()
&& config.per_timeline_limit.is_none()
{
return Ok(0);
}
let conn = self.lock_write()?;
let tx = conn.unchecked_transaction()?;
let mut total_deleted: u64 = 0;
// ① TTL (単一 tx)
if let Some(ttl_days) = config.ttl_days {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
let ttl_cutoff = now - ttl_days * 86_400;
let tx = conn.unchecked_transaction()?;
let n = tx.execute(
"DELETE FROM notes_cache WHERE cached_at < ?1",
params![ttl_cutoff],
)?;
tx.commit()?;
total_deleted += n as u64;
}
// ② per-timeline トリム (チャンク分割 tx)
if let Some(per_timeline_limit) = config.per_timeline_limit {
total_deleted += self.trim_timelines_chunked(&conn, per_timeline_limit)?;
}
pub fn cleanup_with_eviction(&self, config: &EvictionConfig) -> Result<u64, NoteDeckError> {
// 全フィールド無効なら早期 return (lock も取らない)。
// per_timeline_limit を含む 3 フィールド判定であること (2 フィールド判定だと
// per-timeline のみ設定時にトリムが走らない)。
if config.per_account_limit.is_none()
&& config.ttl_days.is_none()
&& config.per_timeline_limit.is_none()
{
return Ok(0);
}
let mut total_deleted: u64 = 0;
// ① TTL (単一 tx)
if let Some(ttl_days) = config.ttl_days {
let conn = self.lock_write()?;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
let ttl_cutoff = now - ttl_days * 86_400;
let tx = conn.unchecked_transaction()?;
let n = tx.execute(
"DELETE FROM notes_cache WHERE cached_at < ?1",
params![ttl_cutoff],
)?;
tx.commit()?;
total_deleted += n as u64;
}
// ② per-timeline トリム (チャンク分割 tx)
if let Some(per_timeline_limit) = config.per_timeline_limit {
total_deleted += self.trim_timelines_chunked(per_timeline_limit)?;
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/db.rs` around lines 1041 - 1074, Refactor cleanup_with_eviction and
trim_timelines_chunked so the helper accepts &self rather than a held
&Connection, and acquire the writer lock separately inside each chunk iteration.
Remove the long-lived lock_write guard from cleanup_with_eviction while
preserving TTL handling and deletion totals, ensuring the mutex is released
after every chunk so ingest_notes and other writers can interleave.

Comment thread src/error.rs
Comment on lines 56 to +57
#[error("{0}")]
Auth(String),

#[error("WebSocket: {0}")]
WebSocket(String),
Auth(AuthErrorKind),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift

Use NotecliError as the canonical public error type.

Lines 56-57 continue to expose NoteDeckError as the application error boundary. Rename the canonical enum in src/error.rs to NotecliError and migrate its callers in this release.

As per coding guidelines, error types must be unified in error.rs as NotecliError.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/error.rs` around lines 56 - 57, Rename the canonical application error
enum in error.rs from NoteDeckError to NotecliError, then migrate all callers,
imports, return types, and references to use NotecliError consistently as the
public error boundary.

Source: Coding guidelines

Comment thread src/error.rs
Comment on lines +71 to +73
/// 起こり得ないはずの内部不整合(ロック汚染、保存直後の読み出し失敗等)。
#[error("Internal error: {0}")]
Internal(String),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Use safe_message() in the HTTP error adapter.

src/http_server.rs Lines 143-159 assigns message: e.to_string(). If NoteDeckError::Internal reaches an HTTP route, the response exposes its internal detail despite Lines 137-140 defining a sanitized message. Replace that conversion with e.safe_message().

Proposed fix
-            message: e.to_string(),
+            message: e.safe_message(),

As per coding guidelines, API error messages must use safe_message() and must not expose API tokens.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/error.rs` around lines 71 - 73, Update the HTTP error adapter in
http_server.rs, specifically the response construction around the message field,
to use NoteDeckError::safe_message() instead of e.to_string(). Preserve the
existing status and response handling while ensuring Internal errors return the
sanitized message.

Source: Coding guidelines

Comment thread src/models.rs
Comment on lines +513 to 559
pub fn parse(s: &str) -> Result<Self, crate::error::NoteDeckError> {
use crate::error::NoteDeckError;
if s.is_empty() {
return Err(NoteDeckError::InvalidInput(
"timeline key must not be empty".to_string(),
));
}
if s.len() > 256 {
return Err(NoteDeckError::InvalidInput(
"timeline key exceeds 256 bytes".to_string(),
));
}
if s.bytes().any(|b| b < 0x20 || b == 0x7F) {
return Err(NoteDeckError::InvalidInput(
"timeline key contains control characters".to_string(),
));
}
if let Some((prefix, id)) = s.split_once(':') {
if id.is_empty() {
return Err(NoteDeckError::InvalidInput(format!(
"timeline key '{prefix}:' has empty id"
)));
}
let id = id.to_string();
match prefix {
"user-list" => Ok(Self::UserList { list_id: id }),
"antenna" => Ok(Self::Antenna { antenna_id: id }),
"channel" => Ok(Self::Channel { channel_id: id }),
"role" => Ok(Self::Role { role_id: id }),
"clip" => Ok(Self::Clip { clip_id: id }),
"user" => Ok(Self::UserNotes { user_id: id }),
_ => Err(NoteDeckError::InvalidInput(format!(
"unknown timeline key prefix '{prefix}'"
))),
}
} else {
match s {
"mentions" => Ok(Self::Mentions),
"specified" => Ok(Self::Specified),
"favorites" => Ok(Self::Favorites),
_ if RESERVED_PREFIXES.contains(&s) => Err(NoteDeckError::InvalidInput(format!(
"bare reserved timeline key '{s}' (id required)"
))),
_ => Ok(Self::Basic(s.to_string())),
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

Restrict the allowed characters for Basic timeline names.

parse rejects only empty input, over-length input, control characters, and unknown prefixes. Every other byte sequence becomes Basic(String). api_endpoint then interpolates that string into an API path: Cow::Owned(format!("notes/{other}-timeline")), and MisskeyClient::api_url inserts the result into {scheme}://{host}/api/{endpoint}.

A caller-supplied name that contains /, ., ?, or # therefore changes the effective request target. For example, notes/../../admin/x? resolves to /api/admin/x with -timeline pushed into the query string. src/http_server.rs get_timeline accepts the path segment and only checks matches!(key, TimelineKey::Basic(_)), so this input passes the allowlist and reaches the endpoint mapping with the account token attached.

Add a charset check in parse so Basic accepts only the shape the endpoint and channel mapping assume (lowercase letters, digits, and -).

🛡️ Proposed validation for `Basic` names
             match s {
                 "mentions" => Ok(Self::Mentions),
                 "specified" => Ok(Self::Specified),
                 "favorites" => Ok(Self::Favorites),
                 _ if RESERVED_PREFIXES.contains(&s) => Err(NoteDeckError::InvalidInput(format!(
                     "bare reserved timeline key '{s}' (id required)"
                 ))),
-                _ => Ok(Self::Basic(s.to_string())),
+                _ if s
+                    .bytes()
+                    .all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-') =>
+                {
+                    Ok(Self::Basic(s.to_string()))
+                }
+                _ => Err(NoteDeckError::InvalidInput(format!(
+                    "invalid basic timeline key '{s}'"
+                ))),
             }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
pub fn parse(s: &str) -> Result<Self, crate::error::NoteDeckError> {
use crate::error::NoteDeckError;
if s.is_empty() {
return Err(NoteDeckError::InvalidInput(
"timeline key must not be empty".to_string(),
));
}
if s.len() > 256 {
return Err(NoteDeckError::InvalidInput(
"timeline key exceeds 256 bytes".to_string(),
));
}
if s.bytes().any(|b| b < 0x20 || b == 0x7F) {
return Err(NoteDeckError::InvalidInput(
"timeline key contains control characters".to_string(),
));
}
if let Some((prefix, id)) = s.split_once(':') {
if id.is_empty() {
return Err(NoteDeckError::InvalidInput(format!(
"timeline key '{prefix}:' has empty id"
)));
}
let id = id.to_string();
match prefix {
"user-list" => Ok(Self::UserList { list_id: id }),
"antenna" => Ok(Self::Antenna { antenna_id: id }),
"channel" => Ok(Self::Channel { channel_id: id }),
"role" => Ok(Self::Role { role_id: id }),
"clip" => Ok(Self::Clip { clip_id: id }),
"user" => Ok(Self::UserNotes { user_id: id }),
_ => Err(NoteDeckError::InvalidInput(format!(
"unknown timeline key prefix '{prefix}'"
))),
}
} else {
match s {
"mentions" => Ok(Self::Mentions),
"specified" => Ok(Self::Specified),
"favorites" => Ok(Self::Favorites),
_ if RESERVED_PREFIXES.contains(&s) => Err(NoteDeckError::InvalidInput(format!(
"bare reserved timeline key '{s}' (id required)"
))),
_ => Ok(Self::Basic(s.to_string())),
}
}
}
pub fn parse(s: &str) -> Result<Self, crate::error::NoteDeckError> {
use crate::error::NoteDeckError;
if s.is_empty() {
return Err(NoteDeckError::InvalidInput(
"timeline key must not be empty".to_string(),
));
}
if s.len() > 256 {
return Err(NoteDeckError::InvalidInput(
"timeline key exceeds 256 bytes".to_string(),
));
}
if s.bytes().any(|b| b < 0x20 || b == 0x7F) {
return Err(NoteDeckError::InvalidInput(
"timeline key contains control characters".to_string(),
));
}
if let Some((prefix, id)) = s.split_once(':') {
if id.is_empty() {
return Err(NoteDeckError::InvalidInput(
format!("timeline key '{prefix}:' has empty id")
));
}
let id = id.to_string();
match prefix {
"user-list" => Ok(Self::UserList { list_id: id }),
"antenna" => Ok(Self::Antenna { antenna_id: id }),
"channel" => Ok(Self::Channel { channel_id: id }),
"role" => Ok(Self::Role { role_id: id }),
"clip" => Ok(Self::Clip { clip_id: id }),
"user" => Ok(Self::UserNotes { user_id: id }),
_ => Err(NoteDeckError::InvalidInput(format!(
"unknown timeline key prefix '{prefix}'"
))),
}
} else {
match s {
"mentions" => Ok(Self::Mentions),
"specified" => Ok(Self::Specified),
"favorites" => Ok(Self::Favorites),
_ if RESERVED_PREFIXES.contains(&s) => Err(NoteDeckError::InvalidInput(format!(
"bare reserved timeline key '{s}' (id required)"
))),
_ if s
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-') =>
{
Ok(Self::Basic(s.to_string()))
}
_ => Err(NoteDeckError::InvalidInput(format!(
"invalid basic timeline key '{s}'"
))),
}
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/models.rs` around lines 513 - 559, Update TimelineKey::parse so the
fallback Basic branch accepts only non-empty names composed of lowercase ASCII
letters, digits, and hyphens. Reject any other character with
NoteDeckError::InvalidInput before constructing Basic, while preserving the
existing prefixed-key parsing and reserved-name handling.

@hitalin
hitalin merged commit b051a7f into main Aug 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

notes キャッシュの実体/タイムライン所属の分離(複数 TL 所属とキー不一致の解消)

1 participant