Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Further performance improvements for job executions (for oracle, other dbs might vary) #5695

Closed
klopfdreh opened this issue Feb 22, 2024 · 0 comments · Fixed by #5700
Closed
Assignees
Labels
area/db Belongs to database
Milestone

Comments

@klopfdreh
Copy link
Contributor

Description:
For Oracle we found out that the performance for job executions sqls can be further improved as they are not fast enought to deliver a proper response for the UI in time. (30 seconds then our gateway cuts off) Note that we already have 1743745 job executions.

Release versions:
SCDF 2.11.2

Custom apps:
N/A

Steps to reproduce:
Just go into the job executions page at the SCDF UI and navigate through the pages.

Screenshots:
image

Additional context:
The issue can be solved by ceating the following indexes.

create index SCDF.SYS_C0098765 on SCDF.BATCH_STEP_EXECUTION("JOB_EXECUTION_ID");
create index SCDF.SYS_C0087654 on SCDF.BOOT3_BATCH_STEP_EXECUTION("JOB_EXECUTION_ID");
create index SCDF.SYS_C0076543 on SCDF.BOOT3_TASK_TASK_BATCH("JOB_EXECUTION_ID");
create index SCDF.SYS_C0065432 on SCDF.BATCH_JOB_EXECUTION("START_TIME");

Also the oracle database suggests a sql_profile for the first page which can not be created via SQL statements.

After those changes the paged job executions can be delivered in ~900ms (for the initial page (1) and all following (2-N))

@github-actions github-actions bot added the status/need-triage Team needs to triage and take a first look label Feb 22, 2024
corneil added a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
@corneil corneil added area/db Belongs to database and removed status/need-triage Team needs to triage and take a first look labels Feb 22, 2024
@corneil corneil self-assigned this Feb 22, 2024
corneil added a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
corneil added a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes spring-cloud#5695
corneil added a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes spring-cloud#5695
corneil added a commit to corneil/spring-cloud-dataflow that referenced this issue Feb 22, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes spring-cloud#5695
corneil added a commit that referenced this issue Feb 23, 2024
… queries when customer has large number of job executions in history. They experienced queries exceeding 180s.

Fixes #5695
@corneil corneil added this to the 2.11.3 milestone May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/db Belongs to database
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants