Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(recorder): Suggest index for optimizing query #26433

Merged
merged 5 commits into from
May 14, 2024

Conversation

ankush
Copy link
Member

@ankush ankush commented May 14, 2024

Recorder can now suggest indexes that might help speed up a particular recording.

As of now this is based on simple heuristics that developers also follow:

  1. parse query and get table names
  2. Parse query and get potential indexable columns.
  3. Check cardinality of each potential column.
  4. Recommend highest one.
  5. Do it for each query, sort by most "helpful" indexes and recommend them.

This works like 80% of the time where problem is a missing index. Hire experts for other 20% 😄

docs https://frappeframework.com/docs/user/en/profiling#query-optimization-using-recorder

image

Query is most important yet it was only taking 2 cols, unreadable.
This reverts commit eadfe86fd834f43925956fa6d759127aaf363441.
@ankush ankush requested review from a team and akhilnarang and removed request for a team May 14, 2024 10:29
@ankush ankush marked this pull request as draft May 14, 2024 10:29
@github-actions github-actions bot added the add-test-cases Add test case to validate fix or enhancement label May 14, 2024
@ankush ankush force-pushed the query_optimzier branch 2 times, most recently from 8783cf3 to 8527b92 Compare May 14, 2024 13:02
@ankush ankush marked this pull request as ready for review May 14, 2024 13:04
@ankush ankush enabled auto-merge May 14, 2024 13:05
@ankush ankush added backport version-15-hotfix Backport the PR to v15 and removed add-test-cases Add test case to validate fix or enhancement labels May 14, 2024
@ankush ankush merged commit 6fb6c40 into frappe:develop May 14, 2024
23 of 24 checks passed
@ankush ankush deleted the query_optimzier branch May 14, 2024 13:38
ankush added a commit that referenced this pull request May 16, 2024
…26435)

* fix(DX): fix up column widths in recorder grid

Query is most important yet it was only taking 2 cols, unreadable.

(cherry picked from commit aba8d4e)

# Conflicts:
#	frappe/core/doctype/recorder_query/recorder_query.json

* fix: Avoid erasing recorder during clear_cache

This reverts commit eadfe86fd834f43925956fa6d759127aaf363441.

(cherry picked from commit 16c8a30)

* fix(UX): colour duration to highlight bad rows

(cherry picked from commit 49f8f4f)

* feat(recorder): Suggest query index

(cherry picked from commit 4e251e9)

# Conflicts:
#	frappe/core/doctype/recorder/recorder.py

* feat: Add identified index from UI

(cherry picked from commit b169f87)

# Conflicts:
#	frappe/commands/site.py

---------

Co-authored-by: Ankush Menat <ankush@frappe.io>
frappe-pr-bot pushed a commit that referenced this pull request May 22, 2024
# [15.28.0](v15.27.0...v15.28.0) (2024-05-22)

### Bug Fixes

* "Not assigned" filter ([#26468](#26468)) ([#26470](#26470)) ([ee6545b](ee6545b))
* compute modal title server side ([#26192](#26192)) ([3b787ac](3b787ac))
* condition of event participants ([#26300](#26300)) ([#26456](#26456)) ([31be0b9](31be0b9))
* default tree view ([07883a7](07883a7))
* Don't fiddle with child table indexes ([#26450](#26450)) ([#26453](#26453)) ([7e0d734](7e0d734))
* **DX:** Don't run CI if there are no tests (backport [#26457](#26457)) ([#26466](#26466)) ([72c693e](72c693e))
* enable fetch_from for Time fieldtype ([#25266](#25266)) ([#26471](#26471)) ([#26483](#26483)) ([f1267ca](f1267ca))
* **grid_row:** don't crash when undefined ([116ba1c](116ba1c))
* preserve checked items after a search ([befcce1](befcce1))

### Features

* **recorder:** Suggest index for optimizing query (backport [#26433](#26433)) ([#26435](#26435)) ([4b780cd](4b780cd))

### Performance Improvements

* num2words, babel, gettext, sentry imports ([#26475](#26475)) ([#26477](#26477)) ([e79e5f9](e79e5f9))
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport version-15-hotfix Backport the PR to v15
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant