From 6f2e8bca00049b4dce1fb0f61811751d5a5b2071 Mon Sep 17 00:00:00 2001 From: zensgit <77236085+zensgit@users.noreply.github.com> Date: Thu, 25 Sep 2025 13:33:17 +0800 Subject: [PATCH 1/2] api: add negative auth tests, family default ledger test, superadmin doc, export streaming design draft --- README.md | 19 +++ docs/EXPORT_STREAMING_DESIGN.md | 76 ++++++++++++ .../integration/auth_login_negative_test.rs | 108 ++++++++++++++++++ .../integration/family_default_ledger_test.rs | 49 ++++++++ 4 files changed, 252 insertions(+) create mode 100644 docs/EXPORT_STREAMING_DESIGN.md create mode 100644 jive-api/tests/integration/auth_login_negative_test.rs create mode 100644 jive-api/tests/integration/family_default_ledger_test.rs diff --git a/README.md b/README.md index 85158cc4..a7c1d12b 100644 --- a/README.md +++ b/README.md @@ -176,6 +176,25 @@ curl -s http://localhost:8012/health make db-dev-down ``` +### 超级管理员默认密码说明 + +迁移脚本会为内置超级管理员（`superadmin@jive.money`）设置一个固定 Argon2 哈希，对应初始密码：`SuperAdmin@123`。 + +注意事项： +- 如在本地手动用工具（例如 `cargo run --bin hash_password`）更新了该账号密码，再次重建 / 重新应用迁移（或使用全新数据库）时会回退到默认密码。 +- CI / 新开发环境请按默认密码尝试首次登录后立即修改。 +- 不要在仓库提交真实生产密码；如需变更默认策略，可新增迁移修改哈希值并在安全文档中注明。 + +本地若需要快速验证超级管理员登录，可： + +```bash +curl -s -X POST http://localhost:8012/api/v1/auth/login \ + -H 'Content-Type: application/json' \ + -d '{"email":"superadmin@jive.money","password":"SuperAdmin@123"}' +``` + +返回中包含 `token` 即表示成功；请在生产部署中更换为安全随机密码并限制暴露。 + ## 🧪 本地CI（不占用GitHub Actions分钟） 当你的GitHub Actions分钟不足时，可以使用本地CI脚本模拟CI流程： diff --git a/docs/EXPORT_STREAMING_DESIGN.md b/docs/EXPORT_STREAMING_DESIGN.md new file mode 100644 index 00000000..096ea6cc --- /dev/null +++ b/docs/EXPORT_STREAMING_DESIGN.md @@ -0,0 +1,76 @@ +## Export Streaming Design (Draft) + +Current implementation (GET /transactions/export.csv and POST JSON export) builds the entire CSV payload in memory before responding. This is acceptable for small/medium datasets (< ~5–10k rows) but risks: +- Elevated peak memory usage proportional to row count × serialized width +- Increased latency before first byte (TTFB) for large exports +- Potential timeouts on slow clients / large data + +### Target Goals +- Stream CSV rows incrementally to the client +- Maintain existing endpoint semantics (query params + include_header) +- Preserve authorization and filtering logic +- Avoid loading all rows simultaneously; use DB cursor / chunked fetch +- Keep memory O(chunk_size) instead of O(total_rows) + +### Proposed Approach +1. Introduce an async Stream body (e.g. `axum::body::Body` from a `tokio_stream::wrappers::ReceiverStream`). +2. Acquire a server-side channel (mpsc) or use `async_stream::try_stream!` to yield `Result` chunks. +3. Write header first (conditional on `include_header`). +4. Fetch rows in chunks: `LIMIT $N OFFSET loop*chunk` or preferably a server-side cursor / keyset pagination if ordering stable. +5. Serialize each row to a CSV line and push into the stream; flush periodically (small `Bytes` frames of ~8–32 KB to balance syscall overhead vs latency). +6. Close stream when no more rows; ensure cancellation drops DB cursor. + +### Database Access Pattern +- Option A (simple): repeated `SELECT ... ORDER BY id LIMIT $chunk OFFSET $offset` until fewer than chunk results. + - Pros: trivial to implement. + - Cons: OFFSET penalty grows with large tables. +- Option B (keyset): track last (id, date) composite and use `WHERE (date,id) > ($last_date,$last_id)` ORDER BY (date,id) LIMIT $chunk. + - Pros: stable performance. + - Cons: Requires deterministic ordering and composite index. +- Option C (cursor): Use PostgreSQL declared cursor inside a transaction with `FETCH FORWARD $chunk`. + - Pros: Minimal SQL complexity, effective for very large sets. + - Cons: Keeps transaction open; need timeout/abort on client disconnect. + +Initial recommendation: start with keyset pagination if the transactions table already has suitable indexes (date + id). Fall back to OFFSET if index not present, then iterate. + +### Error Handling +- If an error occurs mid-stream (DB/network), terminate stream and rely on client detecting incomplete CSV (documented). Optionally append a trailing comment line starting with `# ERROR:` for internal tooling (not for production by default). +- Authorization is validated before streaming begins; per-row permissions should already be enforced by the query predicate. + +### Backpressure & Chunk Size +- Default chunk size: 500 rows. +- Tune by measuring latency vs memory: each row ~150 bytes average → 500 rows ≈ 75 KB before encoding to Bytes (still reasonable). +- Emit each chunk as one Bytes frame; header as a separate first frame. + +### Include Header Logic +- If `include_header=false`, skip header frame. +- Otherwise first frame = `b"col1,col2,...\n"`. + +### CSV Writer +- Reuse existing row-to-CSV logic; adapt it to a function returning `String` line without accumulating in Vec. +- Avoid allocation churn: use a `String` buffer with `clear()` per row. + +### Observability +- Add tracing spans: `export.start` (with row_estimate if available), `export.chunk_emitted` (chunk_index, rows_in_chunk), `export.complete` (total_rows, duration_ms). +- Consider a soft limit guard (e.g. if > 200k rows warn user or require async job + presigned URL pattern—out of scope for first iteration). + +### Compatibility +- Existing clients expecting entire body still work; streaming is transparent at HTTP level. +- For very small datasets overhead is negligible (one header + one chunk frame). + +### Future Extensions +1. Async job offload + download token when row count exceeds threshold. +2. Compression: optional `Accept-Encoding: gzip` support via layered body wrapper. +3. Column selection / dynamic schema negotiation. +4. Rate limiting or concurrency caps per user. + +### Minimal Implementation Checklist +- [ ] Refactor current CSV builder into row serializer +- [ ] Add streaming variant behind feature flag `export_stream` (optional) +- [ ] Implement keyset pagination helper +- [ ] Write integration test for large (e.g. 5k rows) export ensuring early first-byte (<500ms on local) and total content hash matches non-stream version +- [ ] Bench memory before/after (heap snapshot or simple RSS sampling) + +--- +Status: Draft for review. Adjust chunk strategy after initial benchmarks. + diff --git a/jive-api/tests/integration/auth_login_negative_test.rs b/jive-api/tests/integration/auth_login_negative_test.rs new file mode 100644 index 00000000..8110c312 --- /dev/null +++ b/jive-api/tests/integration/auth_login_negative_test.rs @@ -0,0 +1,108 @@ +#[cfg(test)] +mod tests { + use axum::{routing::post, Router}; + use http::StatusCode; + use hyper::Body; + use tower::ServiceExt; + + use jive_money_api::handlers::auth::{login, refresh_token}; + + use crate::fixtures::create_test_pool; + + async fn post_json(app: &Router, path: &str, body: serde_json::Value) -> http::Response { + let req = http::Request::builder() + .method("POST") + .uri(path) + .header(http::header::CONTENT_TYPE, "application/json") + .body(Body::from(body.to_string())) + .unwrap(); + app.clone().oneshot(req).await.unwrap() + } + + #[tokio::test] + async fn login_fails_with_wrong_password_bcrypt() { + let pool = create_test_pool().await; + let email = format!("bcrypt_fail_{}@example.com", uuid::Uuid::new_v4()); + let good_plain = "CorrectPass123!"; + let bcrypt_hash = bcrypt::hash(good_plain, bcrypt::DEFAULT_COST).unwrap(); + + sqlx::query( + r#"INSERT INTO users (email, password_hash, name, is_active, created_at, updated_at) + VALUES ($1,$2,$3,true,NOW(),NOW())"#, + ) + .bind(&email) + .bind(&bcrypt_hash) + .bind("Bcrypt Fail") + .execute(&pool) + .await + .expect("insert bcrypt user"); + + let app = Router::new() + .route("/api/v1/auth/login", post(login)) + .with_state(pool.clone()); + + // Wrong password + let resp = post_json(&app, "/api/v1/auth/login", serde_json::json!({ + "email": email, + "password": "BadPass999!", + })).await; + assert_eq!(resp.status(), StatusCode::UNAUTHORIZED); + + // Cleanup + sqlx::query("DELETE FROM users WHERE LOWER(email)=LOWER($1)") + .bind(&email) + .execute(&pool) + .await + .ok(); + } + + #[tokio::test] + async fn refresh_fails_for_inactive_user() { + let pool = create_test_pool().await; + let email = format!("inactive_refresh_{}@example.com", uuid::Uuid::new_v4()); + + // Create inactive user (argon2) + let salt = argon2::password_hash::SaltString::generate(&mut argon2::password_hash::rand_core::OsRng); + let argon2 = argon2::Argon2::default(); + let hash = argon2 + .hash_password("InactivePass123!".as_bytes(), &salt) + .unwrap() + .to_string(); + let user_id: uuid::Uuid = uuid::Uuid::new_v4(); + sqlx::query( + r#"INSERT INTO users (id, email, password_hash, name, is_active, created_at, updated_at) + VALUES ($1,$2,$3,$4,false,NOW(),NOW())"#, + ) + .bind(user_id) + .bind(&email) + .bind(&hash) + .bind("Inactive Refresh") + .execute(&pool) + .await + .expect("insert inactive user"); + + // Generate a JWT manually to simulate prior login (even though user inactive now) + let token = jive_money_api::auth::generate_jwt(user_id, None).unwrap(); + + let app = Router::new() + .route("/api/v1/auth/refresh", post(refresh_token)) + .with_state(pool.clone()); + + // Attempt refresh + let req = http::Request::builder() + .method("POST") + .uri("/api/v1/auth/refresh") + .header("Authorization", format!("Bearer {}", token)) + .body(Body::empty()) + .unwrap(); + let resp = app.clone().oneshot(req).await.unwrap(); + assert_eq!(resp.status(), StatusCode::FORBIDDEN); + + sqlx::query("DELETE FROM users WHERE id = $1") + .bind(user_id) + .execute(&pool) + .await + .ok(); + } +} + diff --git a/jive-api/tests/integration/family_default_ledger_test.rs b/jive-api/tests/integration/family_default_ledger_test.rs new file mode 100644 index 00000000..80322850 --- /dev/null +++ b/jive-api/tests/integration/family_default_ledger_test.rs @@ -0,0 +1,49 @@ +#[cfg(test)] +mod tests { + use jive_money_api::services::{auth_service::{AuthService, RegisterRequest}, FamilyService}; + use crate::fixtures::create_test_pool; + + #[tokio::test] + async fn family_creation_sets_default_ledger() { + let pool = create_test_pool().await; + let auth = AuthService::new(pool.clone()); + let email = format!("family_def_{}@example.com", uuid::Uuid::new_v4()); + let uc = auth.register_with_family(RegisterRequest { + email: email.clone(), + password: "FamilyDef123!".to_string(), + name: Some("Family Owner".to_string()), + username: None, + }).await.expect("register user"); + + let user_id = uc.user_id; + let family_id = uc.current_family_id.expect("family id"); + + // Query ledger(s) + #[derive(sqlx::FromRow, Debug)] + struct LedgerRow { id: uuid::Uuid, family_id: uuid::Uuid, is_default: Option, created_by: Option, name: String } + let ledgers = sqlx::query_as::<_, LedgerRow>( + "SELECT id, family_id, is_default, created_by, name FROM ledgers WHERE family_id = $1" + ) + .bind(family_id) + .fetch_all(&pool).await.expect("fetch ledgers"); + + assert_eq!(ledgers.len(), 1, "exactly one default ledger expected"); + let ledger = &ledgers[0]; + assert_eq!(ledger.family_id, family_id); + assert_eq!(ledger.is_default.unwrap_or(false), true, "ledger should be default"); + assert_eq!(ledger.created_by.unwrap(), user_id, "created_by should be owner user_id"); + assert_eq!(ledger.name, "默认账本"); + + // Also ensure service context can fetch families list for sanity + let fam_service = FamilyService::new(pool.clone()); + let families = fam_service.get_user_families(user_id).await.expect("user families"); + assert_eq!(families.len(), 1); + + sqlx::query("DELETE FROM users WHERE id = $1") + .bind(user_id) + .execute(&pool) + .await + .ok(); + } +} + From c707c4d8e0dd786e6a9f360e6ac7b82e0eb8e692 Mon Sep 17 00:00:00 2001 From: zensgit <77236085+zensgit@users.noreply.github.com> Date: Thu, 25 Sep 2025 13:40:45 +0800 Subject: [PATCH 2/2] docs: reconcile superadmin password baseline (admin123 vs SuperAdmin@123) --- README.md | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index a7c1d12b..f270fc26 100644 --- a/README.md +++ b/README.md @@ -178,22 +178,37 @@ make db-dev-down ### 超级管理员默认密码说明 -迁移脚本会为内置超级管理员（`superadmin@jive.money`）设置一个固定 Argon2 哈希，对应初始密码：`SuperAdmin@123`。 +仓库历史存在两个默认密码基线： -注意事项： -- 如在本地手动用工具（例如 `cargo run --bin hash_password`）更新了该账号密码，再次重建 / 重新应用迁移（或使用全新数据库）时会回退到默认密码。 -- CI / 新开发环境请按默认密码尝试首次登录后立即修改。 -- 不要在仓库提交真实生产密码；如需变更默认策略，可新增迁移修改哈希值并在安全文档中注明。 +| 密码 | 出现来源 | 当前优先级 | +|------|----------|------------| +| `admin123` | 早期迁移：`005_create_superadmin.sql` / `006_update_superadmin_password.sql` / `016_fix_families_member_count_and_superadmin.sql` | 旧（可能仍在本地旧库残留） | +| `SuperAdmin@123` | 后续迁移：`009_create_superadmin_user.sql` 与补偿脚本 | 新（建议统一） | + +实际生效取决于“最后一次在你的数据库中执行成功的迁移顺序”。如果你基于较新的全量迁移（包含 009 及之后）初始化数据库，默认应为 `SuperAdmin@123`（Argon2）。如果本地数据库较早创建，仍可能是 `admin123`（bcrypt 或 Argon2）。 -本地若需要快速验证超级管理员登录，可： +判定与处理建议： +1. 直接尝试两次登录（先 `SuperAdmin@123`，再 `admin123`）。 +2. 若均失败，可在本地用工具重置： + ```bash + cargo run -p jive-money-api --bin hash_password -- SuperAdmin@123 + # 得到哈希后： + psql "$DATABASE_URL" -c "UPDATE users SET password_hash='' WHERE LOWER(email)='superadmin@jive.money';" + ``` +3. 重置后立即登录并修改为你的本地私有密码（不要提交哈希）。 +注意事项： +- 重新“干净”初始化数据库（删除数据卷 / 新建数据库）后会再次回到迁移脚本指定的默认值。 +- 请勿将生产环境实际超级管理员密码写入仓库或日志。 +- 如果团队决定最终统一为 `SuperAdmin@123` 以外的基线，请新增新的迁移并在此表格中更新来源说明。 + +快速登录测试（假设使用新基线）： ```bash curl -s -X POST http://localhost:8012/api/v1/auth/login \ -H 'Content-Type: application/json' \ -d '{"email":"superadmin@jive.money","password":"SuperAdmin@123"}' ``` - -返回中包含 `token` 即表示成功；请在生产部署中更换为安全随机密码并限制暴露。 +若返回 JSON 含 `token` 字段表示成功。生产中请务必改成强随机密码并限制暴露。 ## 🧪 本地CI（不占用GitHub Actions分钟）