Skip to content

⚡ Performance Optimization: Resolve N+1 Query in export_to_sql_by_form#835

Merged
fderuiter merged 3 commits intomainfrom
perf-optimize-sql-export-4238338433989229310
Apr 28, 2026
Merged

⚡ Performance Optimization: Resolve N+1 Query in export_to_sql_by_form#835
fderuiter merged 3 commits intomainfrom
perf-optimize-sql-export-4238338433989229310

Conversation

@fderuiter
Copy link
Copy Markdown
Owner

💡 What:
Optimized the export_to_sql_by_form function in src/imednet/integrations/export.py to eliminate an N+1 query pattern. Instead of making a remote API call for each form to fetch its variables, the function now fetches all variables for the study in a single bulk request and groups them by form_id locally using a dictionary.

🎯 Why:
Previously, the loop executed sdk.variables.list(..., formId=form.form_id) for every form in the forms list. For studies with many forms, this resulted in significant I/O overhead and slow export performance. Bulk-fetching variables reduces the number of API calls from N+1 to 1 (plus the initial forms list call).

📊 Measured Improvement:

  • Baseline: 5 API calls to sdk.variables.list for 5 forms.
  • Optimized: 1 API call to sdk.variables.list for any number of forms.
  • Change: Significant reduction in network latency and API overhead, especially for large studies.

Verified with a reproduction script that confirmed the call count reduction and ensured functionality remains intact. Static analysis with mypy passed successfully.


PR created automatically by Jules for task 4238338433989229310 started by @fderuiter

- Bulk-fetch all variables for the study once before the loop
- Map variables by form_id for O(1) local lookup
- Eliminates N+1 API call pattern in export_to_sql_by_form

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@fderuiter fderuiter marked this pull request as ready for review April 28, 2026 01:19
fderuiter and others added 2 commits April 28, 2026 17:06
- Bulk-fetch all variables for the study once before the loop
- Map variables by form_id for O(1) local lookup
- Eliminates N+1 API call pattern in export_to_sql_by_form
- Updated unit tests to reflect optimized API call count and variable attributes

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@fderuiter fderuiter merged commit c93337b into main Apr 28, 2026
4 checks passed
@fderuiter fderuiter deleted the perf-optimize-sql-export-4238338433989229310 branch April 28, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant