Description
This issue is automatically created based on existing pull request: #39603: Remove limit in IdListBuilder
There should be no limit on the IdListBuilder since it is chunked in: vendor/magento/module-sales/Model/ResourceModel/Grid.php
:
public function refreshBySchedule()
{
$lastUpdatedAt = null;
$notSyncedIds = $this->notSyncedDataProvider->getIds($this->mainTableName, $this->gridTableName);
foreach (array_chunk($notSyncedIds, self::BATCH_SIZE) as $bunch) {
$select = $this->getGridOriginSelect()->where($this->mainTableName . '.entity_id IN (?)', $bunch);
Description (*)
The limit in the select is removed so it will fetch all results and later it will be chuncked.
Related Pull Requests
Manual testing scenarios (*)
- Make sure you have more than 100 orders
- Make sure your
sales_order_grid
is also filled with this orders - Truncate
sales_order_grid
- Make sure async indexing of grids is true
- run
magerun2 sys:cron:run sales_grid_order_async_insert
vendor/magento/module-sales/Model/ResourceModel/Grid.php
public function refreshBySchedule()
{
$lastUpdatedAt = null;
$notSyncedIds = $this->notSyncedDataProvider->getIds($this->mainTableName, $this->gridTableName);
foreach (array_chunk($notSyncedIds, self::BATCH_SIZE) as $bunch) {
$select = $this->getGridOriginSelect()->where($this->mainTableName . '.entity_id IN (?)', $bunch);
The code above will be run and fetch $notSyncedIds
and chunck it in batches. The provider used is: vendor/magento/module-sales/Model/ResourceModel/Provider/UpdatedIdListProvider.php
and does this:
public function getIds($mainTableName, $gridTableName)
{
$mainTableName = $this->resourceConnection->getTableName($mainTableName);
$gridTableName = $this->resourceConnection->getTableName($gridTableName);
$select = $this->idListQueryBuilder->build($mainTableName, $gridTableName);
return $this->getConnection()->fetchAll($select, [], \Zend_Db::FETCH_COLUMN);
and builds a select with the idListQueryBuilder
:
public function build(string $mainTableName, string $gridTableName): Select
{
$select = $this->getConnection()->select()
->from(['main_table' => $mainTableName], ['main_table.entity_id'])
->joinLeft(
['grid_table' => $this->resourceConnection->getTableName($gridTableName)],
'main_table.entity_id = grid_table.entity_id',
[]
);
$select->where('grid_table.entity_id IS NULL');
$select->limit(Grid::BATCH_SIZE); // should be removed
foreach ($this->additionalGridTables as $table) {
$select->joinLeft(
[$table => $table],
sprintf(
'%s.%s = %s.%s',
'main_table',
'entity_id',
$table,
'entity_id'
),
[]
)
->where($table . '.entity_id IS NULL');
}
return $select;
}
In here there is limit: $select->limit(Grid::BATCH_SIZE);
the same as the chunk batch size in refreshBySchedule
. This looks like a bug to me, because you will never get more items than 100. Now this will probably only occur when the sales_order_grid
is truncated or more than 100 orders are placed between the cron execution.
Questions or comments
Eventually the grid will be filled, because the cron is run every minute. But it takes a lot of time if you have a lot of orders.
Contribution checklist (*)
- Pull request has a meaningful description of its purposeAll commits are accompanied by meaningful commit messagesAll new or changed code is covered with unit/integration tests (if applicable)README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an updateAll automated tests passed successfully (all builds are green)
Activity
hostep commentedon Feb 18, 2025
This is a duplicate of #39602 BTW.
engcom-Bravo commentedon Feb 28, 2025
Closing this as duplicate of #39602