Fetch service plan visibilities with eager loading

[johha]

Previously the visibility was fetched per service plan. This could cause a high number of db queries when e.g. calling GET /v3/service_plans with a large page size.
This can be improved by:

• Eager loading the service_plan_visibilities to reduce the number of db queries
• Adding a new index for column service_plan_id on postgres to reduce the query time. Mysql automatically creates this index due to the foreign key relationship.

Both changes significantly increases the performance for v3/service_plans when the number of service plans with visibility type organization is high.

Performance measurements - WITHOUT index

100_000 service_plans & 1_000_000 service_plan_visibilities

Parameters

• public plans: 0
• number of visible orgs per plan: 10
• number of org scoped service_plans: 100_000

Results for GET v3/service_plans?page_size=5000

• new implementation: 5.1s (average over 10 requests)
• old implementation: 23.69s (average over 10 requests)

10 service_plans & 1_000_000 service_plan_visibilities

Parameters

• public plans: 0
• number of visible orgs per plan: 100_000
• number of org scoped service_plans: 10

Results for GET v3/service_plans?page_size=5000

• new implementation: 0.15s(average over 10 requests)
• old implementation: 0.14s (average over 10 requests)

100_000 service_plans & 0 service_plan_visibilities

• public plans: 100_000
• number of visible orgs per plan: 0
• number of org scoped service_plans: 0

Results for GET v3/service_plans?page_size=5000

• new implementation: 0.67s (average over 10 requests)
• old implementation: 0.66s (average over 10 requests)

Performance measurements - WITH service_plan_id index

Depending on the table size and possible values of entities in service_plans and service_plan_visibilities eager loading is actually slower than the current implementation.
This is caused by a large sequential scan in the database. To overcome this we introduced a index on column service_plan_id in table service_plan_visibilities

30_000 service_plans & 6_000_000 service_plan_visibilities

Parameters

• public plans: 0
• number of visible orgs per plan: 200
• number of org scoped service_plans: 30_000

Results for GET v3/service_plans?page_size=5000

• new implementation with index: 0.6s (average over 10 requests)
• old implementation with index: 10s (average over 10 requests)
• new implementation wo index: 25s (average over 10 requests)
• old implementation wo index: 14s (average over 10 requests)

• I have reviewed the contributing guide

• I have viewed, signed, and submitted the Contributor License Agreement

• I have made this pull request to the main branch

• I have run all the unit tests using bundle exec rake

• I have run CF Acceptance Tests