Skip to content

(feat): PSDB Layout 2 #266

Open
Adit2607 wants to merge 19 commits intodevelopfrom
poc/fs_layout
Open

(feat): PSDB Layout 2 #266
Adit2607 wants to merge 19 commits intodevelopfrom
poc/fs_layout

Conversation

@Adit2607
Copy link
Contributor

@Adit2607 Adit2607 commented Jan 7, 2026

What type of PR is this?

  • 🍕 Feature
  • 🐛 Bug Fix
  • ⚙️ Config Change
  • 📄 Template Change
  • 📝 CHANGELOG/README Update
  • 🎨 Linting
  • 🧑‍💻 Code Refactor
  • 🔥 Performance Improvements
  • ✅ Test
  • 🔁 Version Bump
  • ⏩ Revert

📂 Modules Affected

  • horizon (Real-time systems / networking)
  • online-feature-store (Feature serving infra)
  • trufflebox-ui (Admin panel / UI)
  • infra (Docker, CI/CD, GCP/AWS setup)
  • docs (Documentation updates)
  • Other: ___________

Added tests

  • 👍 Yes
  • 🙅 No
  • 🙋 Need Help!
  • 👍 Manual
  • 📝 Unit Tests
  • 🔗 API/Integration Tests

Key Requirement Doc

Downstream Impact

  • No downstream impact

Additional Information

Updated by ReadyToReviewBot for @shubhamk-meesho · Edit form

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as outdated.

@shubhamk-meesho shubhamk-meesho changed the title POC FS LAYOUT (feat): PSDB Layout 2 Mar 15, 2026
shubhamk-meesho and others added 3 commits March 16, 2026 12:13
- Fix bool vector serialization: include partial byte in buffer when there are unfilled bits
- Fix type mismatch in system.go: change FP32Vector default decoder from FP16Vector to Float32Vector
- Add bitmap bounds checking: validate bitmap indices before access in deserialized_psdb_layout2.go
  - Check bounds in countSetBitsBefore() to prevent panic on undersized bitmaps
  - Check bounds in skipStringVectorsInDense() before bitmap access
  - Check bounds in GetNumericVectorFeature() and GetBoolVectorFeature() bitmap iteration

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@Meesho Meesho deleted a comment from coderabbitai bot Mar 16, 2026
shubhamk-meesho and others added 2 commits March 16, 2026 15:49
- LLD_TEMPLATE_FOR_WORD.txt: Word-compatible template with 686 lines
- WORD_IMPORT_GUIDE.txt: Step-by-step import instructions
- lld_doc_generation_prompt.md: Comprehensive LLM prompt with 12 sections
- LLD_TEMPLATE.md: Markdown version for GitHub documentation
- COMMIT_ANALYSIS.md: Detailed analysis of 14 commits
- README.md: Quick start guide for all usage workflows
- INDEX.txt: Master navigation index
- GENERATION_SUMMARY.txt: Feature highlights

Covers PSDB Layout V2 & V3 bitmap-based sparse feature encoding.
Ready for Word document creation or LLM-based generation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
coderabbitai[bot]

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

shubhamk-meesho and others added 2 commits March 16, 2026 17:06
- psdb_shadow_compare.go: return pooled PSDB to pool on Build() error (pool leak fix)
- system/system.go: fix GetFP32Vector using FP16 decoder for FP32 defaults
- perm_storage_datablock_v3.go: fix sparse bool vector partial byte truncation, add setupBoolDtypeLastIdx; make setupHeadersLayout2 idempotent (overwrite instead of append on repeat calls)
- cache_storage_datablock_v2.go: add uint16 overflow guard for payloads > 64KB
- perm_storage_datablock_v2.go: fix comment (byte 8 bit 3 → byte 9 bit 0)
- deserialized_psdb_v3.go: add bounds checks for pos in GetStringVectorFeature and GetBoolVectorFeature
- layout_comparison_test.go: guard divide-by-zero in Conclusion block; use safe type assertion for DeserializedPSDBLayout2; remove hardcoded rationale numbers
- redis.go: preserve lifetime-TTL (ExpiryAt==0) PSDBs when building Redis CSDBs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cache_storage_datablock_v2_test.go: replace hardcoded 3-byte defaults with ddb.GetDataType().Size() for all GetNumericScalarFeature calls (14 sites)
- deserialized_psdb_v2_test.go: same fix for d.GetDataType().Size() (12 sites)
- handler/feature/persist.go: fix metric tags to use colon-separated format ("key:value") matching metric.TagAsString() convention
- deserialized_psdb_v3.go: add comments explaining dual bit-ordering in GetBoolVectorFeature (bitmap=LSB-first, dense payload=MSB-first)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@Meesho Meesho deleted a comment from coderabbitai bot Mar 17, 2026
@Meesho Meesho deleted a comment from readytoreview bot Mar 17, 2026
@Meesho Meesho deleted a comment from readytoreview bot Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants