Memory optimizations for integration processing

[themarolt]

Changes proposed ✍️

What

🤖 Generated by Copilot at 3c9d9d8

The pull request adds pagination support to various database repositories and integration services, and refactors the postprocess method of the IntegrationServiceBase class and its subclasses to use the runId from the context object instead of the parameters. The pull request also adds a new utility function processPaginated to simplify the pagination logic. The purpose of these changes is to improve the performance, scalability, and usability of the backend and the frontend when fetching and processing large amounts of data.
​

🤖 Generated by Copilot at 3c9d9d8

Sing, O Muse, of the mighty pull request
That simplified the postprocess method
And added pagination to the queries
Of Integration and Microservice models.

Why

How

🤖 Generated by Copilot at 3c9d9d8

• Add pagination options to findAllActive and findByStatus methods of IntegrationRepository class and update corresponding tests (link, link, link, link, link, link, link)
• Add pagination options to findAllByType method of MicroserviceRepository class and update corresponding tests (link, link, link, link, link, link)
• Add pagination options to findDelayedRuns and findIntegrationsByState methods of IntegrationRunRepository class (link, link, link, link)
• Add pagination, sorting, and filtering options to findByRunId method of IntegrationStreamRepository class and add a new method getNextStreamToProcess (link, link, link, link, link)
• Remove unused parameters from postprocess method of IntegrationServiceBase class and its subclasses and use processPaginated utility and runId context property instead (link, link, link, link, link, link, link, link, link)

Checklist ✅

• Label appropriately with Feature, Improvement, or Bug.
• Add screehshots to the PR description for relevant FE changes
• New backend functionality has been unit-tested.
• API documentation has been updated (if necessary) (see docs on API documentation).
• Quality standards are met.