fix: return user_metadata in list/search responses

[oniani1]

What kind of change does this PR introduce?

Bug fix - resolves #759

What is the current behavior?

storage.list() and list-v2 only return system metadata (size, mimetype, eTag) for each object. Custom user_metadata set during upload is omitted, forcing users to make N additional .info() calls to retrieve it.

What is the new behavior?

All list/search endpoints now include user_metadata in their response. Files return their stored user_metadata; folders return null.

Changes

SQL migration (0059-search-functions-user-metadata.sql):

• Drops and recreates four search functions with user_metadata jsonb added to their RETURNS TABLE and SELECT queries:
  • storage.list_objects_with_delimiter (used by v2 name-sorted listing)
  • storage.search (used by v1 listing, both name and non-name sorting paths)
  • storage.search_v2 (router that delegates to the above)
  • storage.search_by_timestamp (used by v2 timestamp-sorted listing)
• DROP FUNCTION IF EXISTS is required before each CREATE OR REPLACE because PostgreSQL cannot change a function's return type in-place
• Folders emit NULL for user_metadata in all code paths

TypeScript (knex.ts):

• Added 'user_metadata' to the select array in listObjectsV2 for the non-delimiter direct query path (line 383)

Test (object.test.ts):

• Uploads a file with custom metadata, calls list, asserts user_metadata is present on files and null on folders

Verification

• Ran all 59 migrations against a fresh PostgreSQL 15 instance
• Confirmed all 4 functions return user_metadata via direct SQL queries
• Tested all code paths: name sorting, timestamp sorting, non-name sorting, and direct delimiter listing

Additional context

• The Obj schema and route handlers already supported user_metadata -- the data just never left the database layer
• user_metadata column has existed on storage.objects since migration 0025
• .info() correctly returns it; only the search/list functions were missing it
• Backward compatible: normalizeColumns already strips user_metadata for tenants below migration 25, and migrations run sequentially so the column always exists before this migration
• No S3 protocol changes needed -- S3 ListObjects spec doesn't include custom metadata

[oniani1]

Hey, just wanted to bump this one -- the underlying issue has a good amount of user interest and the current workaround (calling .info() per file) adds unnecessary round trips. Happy to adjust anything if needed.

[coveralls]

Coverage Report for CI Build 24613523692

Coverage remained the same at 68.127%

Details

• Coverage remained the same as the base build.
• Patch coverage: No coverable lines changed in this PR.
• No coverage regressions found.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	8935
Covered Lines:	6423
Line Coverage:	71.89%
Relevant Branches:	4876
Covered Branches:	2986
Branch Coverage:	61.24%
Branches in Coverage %:	Yes
Coverage Strength:	405.63 hits per line

---

💛 - Coveralls

[oniani1]

Dug into why CI stays red after the rebase. The tests themselves pass (635/635), but the Verify migration idempotency step fails because 0059 drops and recreates search, list_objects_with_delimiter, search_v2, and search_by_timestamp with user_metadata added to the return type. Once 0059 has run, the idempotency test can't re-apply migration 10 (search-files-search-function), since that migration tries to CREATE OR REPLACE the same 8-param signature with the old return type and PG rejects the return type change.

Two ways I can see to reshape this:

1. Add new function names (search_with_metadata, etc.) and leave the existing functions untouched, switching callers over
2. Drop the SQL-function path here and surface user_metadata only through the direct knex query branch, skipping the functions

Happy to go whichever direction fits the codebase conventions better, or if there's a third preferred pattern, let me know.