Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IBX-719: Introduced BatchIterator #211

Merged
merged 10 commits into from
Jul 15, 2021
Merged

IBX-719: Introduced BatchIterator #211

merged 10 commits into from
Jul 15, 2021

Conversation

adamwojs
Copy link
Member

@adamwojs adamwojs commented Jul 6, 2021

Question Answer
JIRA issue https://issues.ibexa.co/browse/IBX-719
Type feature
Target eZ Platform version v3.3
BC breaks no
Doc needed yes

BatchIterator allows iterate over result sets (e.g. content objects, locations) which cannot be loaded into memory at once.

Examples from existing code base:

$query = new URLQuery();
$query->filter = new Criterion\VisibleOnly();
$query->sortClauses = [
new SortClause\URL(),
];
$query->offset = 0;
$query->limit = $limit;
$totalCount = $this->getTotalCount();
$progress = new ProgressBar($output, $totalCount);
$progress->start();
while ($query->offset < $totalCount) {
$this->urlChecker->check($query);
$progress->advance(min($limit, $totalCount - $query->offset));
$query->offset += $limit;
}
$progress->finish();

while ($query->offset <= $totalCount) {
$results = $this->searchService->findContent($query);
/** @var \eZ\Publish\API\Repository\Values\Content\Search\SearchHit $hit */
foreach ($results->searchHits as $hit) {
$this->resize($output, $hit, $imageFieldIdentifier, $filter);
$progressBar->advance();
}
$query->offset += $iterationCount;
}

Usage

$filter = new Filter();
// ... add criteria, sort clauses etc.

$iterator = new BatchIterator(new ContentFilteringAdapter($contentService, $filter));
foreach ($iterator as $content) {
   // process content object
}

Available adapters

All built-in adapters are places under \eZ\Publish\API\Repository\Iterator\BatchIteratorAdapter\ namespace.

Adapter Method
ContentFilteringAdapter \eZ\Publish\API\Repository\ContentService::find
ContentInfoSearchAdapter \eZ\Publish\API\Repository\SearchService::findContentInfo
ContentSearchAdapter \eZ\Publish\API\Repository\SearchService::findContent
LocationFilteringAdapter \eZ\Publish\API\Repository\LocationService::find
LocationSearchAdapter \eZ\Publish\API\Repository\SearchService::findLocations

Checklist:

  • Provided PR description.
  • Tested the solution manually.
  • Provided automated test coverage.
  • Checked that target branch is set correctly (master for features, the oldest supported for bugs).
  • Ran PHP CS Fixer for new PHP code (use $ composer fix-cs).
  • Asked for a review (ping @ezsystems/php-dev-team).

@juskora juskora added Doc needed The changes require some documentation and removed Doc needed The changes require some documentation labels Jul 6, 2021
@adamwojs adamwojs force-pushed the batch_iterators branch 2 times, most recently from cc55031 to 26341fc Compare July 6, 2021 18:23
@adamwojs adamwojs marked this pull request as ready for review July 6, 2021 18:23
@adamwojs adamwojs requested review from a team and Steveb-p July 6, 2021 18:23
@adamwojs
Copy link
Member Author

adamwojs commented Jul 6, 2021

PR updated according to code review suggestions.

@adamwojs adamwojs self-assigned this Jul 6, 2021
@adamwojs adamwojs added DX Feature New feature or request Doc needed The changes require some documentation labels Jul 6, 2021
Copy link
Contributor

@Steveb-p Steveb-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added language phpdocs based on usage found in PR.

👍

@adamwojs adamwojs requested a review from a team July 8, 2021 10:42
@Steveb-p
Copy link
Contributor

Steveb-p commented Jul 8, 2021

I think sonar code style checks can be discarded: it complains that a local variable has the same name as class property.


final class BatchIterator implements Iterator
{
public const DEFAULT_BATCH_SIZE = 25;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it need to be public?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ViniTou ViniTou requested a review from a team July 9, 2021 14:02
@adamwojs adamwojs requested a review from a team July 12, 2021 12:57
@micszo micszo self-assigned this Jul 14, 2021
@sonarcloud
Copy link

sonarcloud bot commented Jul 14, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@Steveb-p
Copy link
Contributor

Both code smells detected by SonarCloud should probably be marked as "won't fix", since they are about variable that shares the name with property that they are cloned from.

Copy link
Member

@micszo micszo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA Approved on Ibexa Experience 3.3.6-dev with branch.

@micszo micszo removed their assignment Jul 15, 2021
@adamwojs adamwojs merged commit e55da4b into 1.3 Jul 15, 2021
@adamwojs adamwojs deleted the batch_iterators branch July 15, 2021 06:32
@DominikaK DominikaK removed the Doc needed The changes require some documentation label Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DX Feature New feature or request
6 participants