Skip to content

feat: adapt migration schema calls to query-lib value objects - #222

Open
abnegate wants to merge 15 commits into
mainfrom
feat-query-lib
Open

feat: adapt migration schema calls to query-lib value objects#222
abnegate wants to merge 15 commits into
mainfrom
feat-query-lib

Conversation

@abnegate

@abnegate abnegate commented Aug 21, 2026

Copy link
Copy Markdown
Member

Required by appwrite/appwrite#11649 and appwrite-labs/cloud#5410. Depends on utopia-php/database#823.

Why this approach

Database::createCollection() takes a Collection only. Destination Appwrite now wraps collection id/attributes/indexes (and named permissions/documentSecurity) in new Collection(...).

Composer pins utopia-php/database to dev-feat-query-lib as 7.0.0 (808f90bf).

Verified

  • Pint on the Collection wrap

Not verified

  • CI on f2de906 after the Collection-only wrap

Published 2.0 still used Database::VAR_* and positional createAttribute/createIndex, which feat-query-lib removed. Rebase onto main and pass Attribute/Index/Relationship VOs plus ColumnType/IndexType so Appwrite can pin this branch as 2.0.0.
Appwrite stores ColumnType::BigInteger as biginteger. CSV export
resolved that as an unsupported column type and wrote no rows.
createDocument can return an empty Mongo sequence while a subsequent
getDocument has the ObjectId. Creating database_{seq} from the create
return left table import looking up a collection that did not exist.
Appwrite main added huggingface as a project OAuth2 provider. Without
an allow-list entry, Appwrite-to-Appwrite migrations fail on that
provider even when the rest of the transfer succeeded.
Keep query-lib APIs and the huggingface PROVIDERS allow-list already on main.
Appwrite #11649 locks database at 5719edd. Staying on e593b78 would
only prove the schema VO calls against an older query-lib surface.
@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR updates migration schema operations to use query-lib and database value objects while adapting source normalization, destination creation, and tests to the revised APIs. The follow-up changes also complete failed-database recovery under the default duplicate policy.

  • Wraps collection, attribute, relationship, and index schema calls in typed value objects.
  • Converts schema constants and capability checks to query-lib enums and database capabilities.
  • Normalizes the new big-integer representation and updates source readers.
  • Marks metadata reload failures as failed and allows default-policy retries to repair them.
  • Updates dependency resolution and test bootstrapping for the new database branch.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported failed-database retry defect is resolved: failed metadata now enters overwrite recovery even under the default duplicate policy, recreates a missing backing collection, and transitions to ready only after successful completion.

Important Files Changed

Filename Overview
src/Migration/Destinations/Appwrite.php Adapts destination schema creation to typed value objects and correctly completes the previously reported failed-database recovery path.
src/Migration/Resources/Database/Column.php Normalizes query-lib's biginteger schema value to the migration model's existing bigint representation.
src/Migration/Sources/Appwrite.php Updates relationship filtering to use the new schema and relation-side enum values.
src/Migration/Sources/Appwrite/Reader/Database.php Adapts database schema responses to the new typed attribute and index representations.
composer.json Selects the database development branch required for the new schema value-object APIs and adds its VCS repositories.
tests/Migration/Unit/Destinations/AppwriteDatabaseStatusTest.php Covers metadata reload failure and successful recovery under the default duplicate policy.

Reviews (9): Last reviewed commit: "(refactor): use Schema\Order and wrap Co..." | Re-trigger Greptile

utopia-php/database feat-query-lib keys silenced events with
Coroutine::getCid(). The CI image is vanilla PHP, so Memory-adapter
tests fatalled before they could run.
createDocument can persist a row whose subsequent getDocument is empty.
That throw sat outside the failed-status handler, so a later skip could
flip the unusable database to ready without a backing collection.
@abnegate

Copy link
Copy Markdown
Member Author

Addressed the reload-failure finding.

createDocument can persist a _databases row whose immediately following getDocument is empty (the Mongo sequence miss this branch already reloads for). That throw sat outside the markDatabaseFailed catch, so the document stayed provisioning and a later spec-matching skip could flip it to ready with no backing collection.

Reload + createCollection now share that catch. testReloadFailureMarksTheDatabaseFailed fails without the wrap (provisioning) and passes with it (failed).

Also stubbed Swoole\Coroutine::getCid() in the PHPUnit bootstrap so the Memory-adapter suite can run on the CI image, which has no Swoole extension. utopia-php/database feat-query-lib keys silenced events with it.

@greptile-apps review

Asterisk wildcards on utopia-php packages are replaced with
equivalent caret constraints so Composer ranges stay consistent.
Keep composer.json and composer.lock in sync so `composer validate`
passes, and pin utopia-php/database to the current query-lib HEAD.
@abnegate

Copy link
Copy Markdown
Member Author

@greptileai review

@abnegate

Copy link
Copy Markdown
Member Author

@greptile-apps review

Force re-review of HEAD 7a3a60e. Description updated for factories and caret lock refresh.

Database::createCollection no longer accepts a string id.
Database::checkAttribute now requires Attribute. Build schema models from the resource key so metadata document IDs are not used as attribute keys.
Appwrite E2E migrations failed because checkAttribute now requires
Attribute, and the destination still handed it a metadata Document.
Comment thread src/Migration/Destinations/Appwrite.php
A reload failure leaves a metadata document in `failed` with no backing
collection. Recovery only ran when onDuplicate was not Fail, so the
default policy retried createDocument against the existing ID and
stranded the database.
@abnegate

Copy link
Copy Markdown
Member Author

@greptileai review

Index types already used IndexType; column direction was still a raw
ASC string. Collection constructors with multiple named params were
also jammed on one line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant