Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the eagerly loaded job dependencies and env variables to use the default fetch types #1174

Merged
merged 2 commits into from Aug 24, 2022

Conversation

enicloom
Copy link
Contributor

Entity graph loads the named attributes eagerly. This creates a very big list of returned rows when there are multiple layers of OneToMany relationships exist in a single query.

For example, if a single job has 5 job dependencies and 5 job environment variables. The number of returned rows of this entity graph is 155 = 25. This number grows very fast.

This change makes the job_dependencies and job_env_variables to use the default FetchType, which is LAZY. Then in the above example, at most, the number of returned rows is 1 + 5 + 5 if both getJobDependencies() and getJobEnvironmentVariables() are called.

This diff does not make any changes to the cluster, command and application resources since those are mostly safe.

There is a cost of this. By adopting this change, the number of query/db connection needed for getJobSpecification() will be increased to 3 instead of 1. Since this call is associated with every single job launch, it worth to acknowledge the cost.

@enicloom enicloom requested a review from akashdw August 23, 2022 05:06
@coveralls
Copy link

coveralls commented Aug 23, 2022

Coverage Status

Coverage decreased (-0.007%) to 93.916% when pulling 7c2299b on enicloom:lt_simplify_job_specification_graph into abe4f23 on Netflix:master.

@enicloom enicloom merged commit 124919e into Netflix:master Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants