Skip to content

[IndexedDB API] Support IDBGetAllOptions in IDBIndex::getAllKeys()#61317

Merged
webkit-commit-queue merged 1 commit intoWebKit:mainfrom
RupinMittal:eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys
Mar 27, 2026
Merged

[IndexedDB API] Support IDBGetAllOptions in IDBIndex::getAllKeys()#61317
webkit-commit-queue merged 1 commit intoWebKit:mainfrom
RupinMittal:eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys

Conversation

@RupinMittal
Copy link
Copy Markdown
Contributor

@RupinMittal RupinMittal commented Mar 25, 2026

b682bbe

[IndexedDB API] Support IDBGetAllOptions in IDBIndex::getAllKeys()
https://bugs.webkit.org/show_bug.cgi?id=310700
rdar://173311682

Reviewed by Sihui Liu.

The Indexed API Spec has modified IDBIndex's getAllKeys() function to be able to
take in IDBGetAllOptions (https://w3c.github.io/IndexedDB/#dom-idbindex-getallkeys).

This means that getAllKeys() now supports specifiying an IDBCursorDirection.

This patch adds support for this.

The first step is to enable getAllKeys() to take in IDBGetAllOptions. We follow:
https://w3c.github.io/IndexedDB/#create-a-request-to-retrieve-multiple-items.

Currently the overload of IDBIndex::getAllKeys which handles non IDBKeyRange
inputs only handles inputs that are a valid IDBKey. IDBGetAllOptions is not.
So follow the spec's steps and update the logic to be:
1. If the input "is a potentially valid key range", convert it to a keyRange
   (The current algorithm ends here)
2. If we support IDBGetAllOptions, convert it to an IDBGetAllOptions
3. If that's successful, use the query, count, direction from it
   (Need to discern if query is empty, IDBKeyRange, or a regular key)

Since this input parsing must occur after other checks (implemented in
doGetAllKeys) and the input can now contain a count and direction in addition to
a KeyRange, we change doGetAllKeys to take in a new ParsedGetAllQueryOrOptions
object that contains all of these items. In the case where we do have an
IDBGetAllOptions object with a count specified, this count will be used.

The second step is to ensure that the functions later on which actaully return
the records (SQLiteIDBBackingStore::getAllIndexRecords()) receive and use this
direction.

IDBIndex::getAllKeys() will pass this direction to IDBIndex::doGetAllKeys() which
passes it to IDBTransaction::requestGetAllIndexRecords(). This will capture the
direction in an IDBGetAllRecordsData object. Via this object, the direction is
plumbed all the way down to SQLiteIDBBackingStore::getAllIndexRecords().

This function uses the SQLiteIDBCursor returned by maybeOpenBackingStoreCursor()
to iterate the records. Currently it's hardcoded to use "next". So we update it
to use the direction from the IDBGetAllRecordsData object. This cursor already
knows how to deal with the direction from there.

* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any.serviceworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any.sharedworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any.worker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any.serviceworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any.sharedworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any.worker-expected.txt:
* Source/WebCore/Modules/indexeddb/IDBIndex.cpp:
(WebCore::IDBIndex::doGetAll):

Now that IDBTransaction::requestGetAllIndexRecords() takes in a direction,
continue hardcoding "next" for now so behavior is unchanged here.

(WebCore::IDBIndex::doGetAllKeys):
(WebCore::IDBIndex::getAllKeys):

Modify logic to support IDBGetAllOptions based on these steps:
https://w3c.github.io/IndexedDB/#create-a-request-to-retrieve-multiple-items

* Source/WebCore/Modules/indexeddb/IDBIndex.h:
* Source/WebCore/Modules/indexeddb/IDBIndex.idl:
* Source/WebCore/Modules/indexeddb/IDBTransaction.cpp:
(WebCore::IDBTransaction::requestGetAllObjectStoreRecords):
(WebCore::IDBTransaction::requestGetAllIndexRecords):
* Source/WebCore/Modules/indexeddb/IDBTransaction.h:
* Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
(WebCore::IDBServer::SQLiteIDBBackingStore::getAllIndexRecords):
* Source/WebCore/Modules/indexeddb/server/SQLiteIDBCursor.cpp:
(WebCore::IDBServer::SQLiteIDBCursor::maybeCreateBackingStoreCursor):

Take a in a cursor direction and give it to SQLiteIDBCursor.

(WebCore::IDBServer::SQLiteIDBCursor::SQLiteIDBCursor):

Use the passed in cursor direction instead of always using "next".

* Source/WebCore/Modules/indexeddb/server/SQLiteIDBCursor.h:
* Source/WebCore/Modules/indexeddb/server/SQLiteIDBTransaction.cpp:
(WebCore::IDBServer::SQLiteIDBTransaction::maybeOpenBackingStoreCursor):

Take in a cursor direction and give it to SQLiteIDBCursor::maybeCreateBackingStoreCursor.

* Source/WebCore/Modules/indexeddb/server/SQLiteIDBTransaction.h:
* Source/WebCore/Modules/indexeddb/shared/IDBGetAllRecordsData.cpp:
(WebCore::IDBGetAllRecordsData::isolatedCopy const):
(WebCore::IDBGetAllRecordsData::loggingString const):
* Source/WebCore/Modules/indexeddb/shared/IDBGetAllRecordsData.h:

Store a cursor direction.

* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:

Canonical link: https://commits.webkit.org/310051@main

73402cd

Misc iOS, visionOS, tvOS & watchOS macOS Linux Windows Apple Internal
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 win ✅ 🛠 ios-apple
✅ 🧪 bindings ✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug ✅ 🧪 wpe-wk2 🧪 win-tests loading 🛠 mac-apple
✅ 🧪 webkitperl ✅ 🧪 ios-wk2 ✅ 🧪 api-mac ✅ 🧪 api-wpe ⏳ 🛠 vision-apple
✅ 🧪 ios-wk2-wpt ✅ 🧪 api-mac-debug ✅ 🛠 gtk3-libwebrtc
loading 🛠 🧪 jsc ✅ 🧪 api-ios ✅ 🧪 mac-wk1 ✅ 🛠 gtk
✅ 🛠 ios-safer-cpp ❌ 🧪 mac-wk2 ✅ 🧪 gtk-wk2
✅ 🛠 vision ✅ 🧪 mac-AS-debug-wk2 ✅ 🧪 api-gtk
✅ 🛠 vision-sim ✅ 🧪 mac-wk2-stress ✅ 🛠 playstation
✅ 🛠 🧪 unsafe-merge ✅ 🧪 vision-wk2 ✅ 🧪 mac-intel-wk2
✅ 🛠 tv ✅ 🛠 mac-safer-cpp
✅ 🛠 tv-sim
✅ 🛠 watch
✅ 🛠 watch-sim

@RupinMittal RupinMittal self-assigned this Mar 25, 2026
@RupinMittal RupinMittal added the New Bugs Unclassified bugs are placed in this component until the correct component can be determined. label Mar 25, 2026
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 25, 2026
@RupinMittal RupinMittal removed the merging-blocked Applied to prevent a change from being merged label Mar 25, 2026
@RupinMittal RupinMittal force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from 4f8bde4 to d0cd40a Compare March 25, 2026 10:35
@webkit-ews-buildbot
Copy link
Copy Markdown
Collaborator

macOS Safer C++ Build #88607 (d0cd40a)

❌ Found 1 failing file with 3 issues. Please address these issues before landing. See WebKit Guidelines for Safer C++ Programming.
(cc @rniwa)

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 25, 2026
@webkit-ews-buildbot
Copy link
Copy Markdown
Collaborator

iOS Safer C++ Build #6479 (d0cd40a)

❌ Found 1 failing file with 3 issues. Please address these issues before landing. See WebKit Guidelines for Safer C++ Programming.
(cc @rniwa)

@RupinMittal RupinMittal removed the merging-blocked Applied to prevent a change from being merged label Mar 25, 2026
@RupinMittal RupinMittal force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from d0cd40a to 1df6f68 Compare March 25, 2026 18:12
@RupinMittal RupinMittal force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from 1df6f68 to 435a92d Compare March 25, 2026 21:09
@RupinMittal
Copy link
Copy Markdown
Contributor Author

RupinMittal commented Mar 25, 2026

Do we also need to add support for direction in MemoryIndex::getAllRecords? If so how would we test it? (I believe MemoryIndex is used in private mode? Do we have a way to run WPT layout tests in private mode?).

I didn't do it here because that doesn't already have logic to support direction and would be complex and too much for this patch. Can do in a follow up if it's needed.

@RupinMittal RupinMittal marked this pull request as ready for review March 25, 2026 21:13
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 25, 2026
@RupinMittal RupinMittal removed the merging-blocked Applied to prevent a change from being merged label Mar 25, 2026
@RupinMittal RupinMittal force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from 435a92d to 0577eab Compare March 25, 2026 23:57
@RupinMittal RupinMittal requested a review from szewai March 26, 2026 00:00
@RupinMittal RupinMittal force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from 0577eab to 1fa8ff2 Compare March 26, 2026 00:52
@RupinMittal RupinMittal force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from 1fa8ff2 to 73402cd Compare March 26, 2026 22:48
@RupinMittal RupinMittal requested a review from szewai March 26, 2026 22:54
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Mar 27, 2026
@RupinMittal RupinMittal added unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing and removed merging-blocked Applied to prevent a change from being merged labels Mar 27, 2026
https://bugs.webkit.org/show_bug.cgi?id=310700
rdar://173311682

Reviewed by Sihui Liu.

The Indexed API Spec has modified IDBIndex's getAllKeys() function to be able to
take in IDBGetAllOptions (https://w3c.github.io/IndexedDB/#dom-idbindex-getallkeys).

This means that getAllKeys() now supports specifiying an IDBCursorDirection.

This patch adds support for this.

The first step is to enable getAllKeys() to take in IDBGetAllOptions. We follow:
https://w3c.github.io/IndexedDB/#create-a-request-to-retrieve-multiple-items.

Currently the overload of IDBIndex::getAllKeys which handles non IDBKeyRange
inputs only handles inputs that are a valid IDBKey. IDBGetAllOptions is not.
So follow the spec's steps and update the logic to be:
1. If the input "is a potentially valid key range", convert it to a keyRange
   (The current algorithm ends here)
2. If we support IDBGetAllOptions, convert it to an IDBGetAllOptions
3. If that's successful, use the query, count, direction from it
   (Need to discern if query is empty, IDBKeyRange, or a regular key)

Since this input parsing must occur after other checks (implemented in
doGetAllKeys) and the input can now contain a count and direction in addition to
a KeyRange, we change doGetAllKeys to take in a new ParsedGetAllQueryOrOptions
object that contains all of these items. In the case where we do have an
IDBGetAllOptions object with a count specified, this count will be used.

The second step is to ensure that the functions later on which actaully return
the records (SQLiteIDBBackingStore::getAllIndexRecords()) receive and use this
direction.

IDBIndex::getAllKeys() will pass this direction to IDBIndex::doGetAllKeys() which
passes it to IDBTransaction::requestGetAllIndexRecords(). This will capture the
direction in an IDBGetAllRecordsData object. Via this object, the direction is
plumbed all the way down to SQLiteIDBBackingStore::getAllIndexRecords().

This function uses the SQLiteIDBCursor returned by maybeOpenBackingStoreCursor()
to iterate the records. Currently it's hardcoded to use "next". So we update it
to use the direction from the IDBGetAllRecordsData object. This cursor already
knows how to deal with the direction from there.

* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any.serviceworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any.sharedworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/idbindex_getAllKeys-options.any.worker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any.serviceworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any.sharedworker-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.any.worker-expected.txt:
* Source/WebCore/Modules/indexeddb/IDBIndex.cpp:
(WebCore::IDBIndex::doGetAll):

Now that IDBTransaction::requestGetAllIndexRecords() takes in a direction,
continue hardcoding "next" for now so behavior is unchanged here.

(WebCore::IDBIndex::doGetAllKeys):
(WebCore::IDBIndex::getAllKeys):

Modify logic to support IDBGetAllOptions based on these steps:
https://w3c.github.io/IndexedDB/#create-a-request-to-retrieve-multiple-items

* Source/WebCore/Modules/indexeddb/IDBIndex.h:
* Source/WebCore/Modules/indexeddb/IDBIndex.idl:
* Source/WebCore/Modules/indexeddb/IDBTransaction.cpp:
(WebCore::IDBTransaction::requestGetAllObjectStoreRecords):
(WebCore::IDBTransaction::requestGetAllIndexRecords):
* Source/WebCore/Modules/indexeddb/IDBTransaction.h:
* Source/WebCore/Modules/indexeddb/server/SQLiteIDBBackingStore.cpp:
(WebCore::IDBServer::SQLiteIDBBackingStore::getAllIndexRecords):
* Source/WebCore/Modules/indexeddb/server/SQLiteIDBCursor.cpp:
(WebCore::IDBServer::SQLiteIDBCursor::maybeCreateBackingStoreCursor):

Take a in a cursor direction and give it to SQLiteIDBCursor.

(WebCore::IDBServer::SQLiteIDBCursor::SQLiteIDBCursor):

Use the passed in cursor direction instead of always using "next".

* Source/WebCore/Modules/indexeddb/server/SQLiteIDBCursor.h:
* Source/WebCore/Modules/indexeddb/server/SQLiteIDBTransaction.cpp:
(WebCore::IDBServer::SQLiteIDBTransaction::maybeOpenBackingStoreCursor):

Take in a cursor direction and give it to SQLiteIDBCursor::maybeCreateBackingStoreCursor.

* Source/WebCore/Modules/indexeddb/server/SQLiteIDBTransaction.h:
* Source/WebCore/Modules/indexeddb/shared/IDBGetAllRecordsData.cpp:
(WebCore::IDBGetAllRecordsData::isolatedCopy const):
(WebCore::IDBGetAllRecordsData::loggingString const):
* Source/WebCore/Modules/indexeddb/shared/IDBGetAllRecordsData.h:

Store a cursor direction.

* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:

Canonical link: https://commits.webkit.org/310051@main
@webkit-commit-queue webkit-commit-queue force-pushed the eng/IndexedDBSupportIDBGetAllOptionsInIDBIndexGetAllKeys branch from 73402cd to b682bbe Compare March 27, 2026 06:06
@webkit-commit-queue
Copy link
Copy Markdown
Collaborator

Committed 310051@main (b682bbe): https://commits.webkit.org/310051@main

Reviewed commits have been landed. Closing PR #61317 and removing active labels.

@webkit-commit-queue webkit-commit-queue merged commit b682bbe into WebKit:main Mar 27, 2026
@webkit-commit-queue webkit-commit-queue removed the unsafe-merge-queue Applied to send a pull request to merge-queue, but skip building and testing label Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

New Bugs Unclassified bugs are placed in this component until the correct component can be determined.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants