Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(recorder): Suggest index for optimizing query (backport #26433) #26435

Merged
merged 7 commits into from
May 16, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented May 14, 2024

Recorder can now suggest indexes that might help speed up a particular recording.

As of now this is based on simple heuristics that developers also follow:

  1. parse query and get table names
  2. Parse query and get potential indexable columns.
  3. Check cardinality of each potential column.
  4. Recommend highest one.

Works like 80% of the time where problem is a missing index. Hire experts for other use cases 馃槃

docs https://frappeframework.com/docs/user/en/profiling#query-optimization-using-recorder


This is an automatic backport of pull request #26433 done by Mergify.

Query is most important yet it was only taking 2 cols, unreadable.

(cherry picked from commit aba8d4e)

# Conflicts:
#	frappe/core/doctype/recorder_query/recorder_query.json
This reverts commit eadfe86fd834f43925956fa6d759127aaf363441.

(cherry picked from commit 16c8a30)
(cherry picked from commit 4e251e9)

# Conflicts:
#	frappe/core/doctype/recorder/recorder.py
(cherry picked from commit b169f87)

# Conflicts:
#	frappe/commands/site.py
@mergify mergify bot added the conflicts label May 14, 2024
@mergify mergify bot requested review from a team and ankush and removed request for a team May 14, 2024 13:39
@mergify mergify bot assigned ankush May 14, 2024
Copy link
Contributor Author

mergify bot commented May 14, 2024

Cherry-pick of aba8d4e has failed:

On branch mergify/bp/version-15-hotfix/pr-26433
Your branch is up to date with 'origin/version-15-hotfix'.

You are currently cherry-picking commit aba8d4e5b5.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   frappe/core/doctype/recorder_query/recorder_query.json

no changes added to commit (use "git add" and/or "git commit -a")

Cherry-pick of 4e251e9 has failed:

On branch mergify/bp/version-15-hotfix/pr-26433
Your branch is ahead of 'origin/version-15-hotfix' by 3 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 4e251e9b0b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	deleted:    cypress/integration/recorder.js
	new file:   frappe/core/doctype/recorder/db_optimizer.py
	modified:   frappe/core/doctype/recorder/recorder.js
	modified:   frappe/core/doctype/recorder/recorder.json
	new file:   frappe/core/doctype/recorder_suggested_index/__init__.py
	new file:   frappe/core/doctype/recorder_suggested_index/recorder_suggested_index.json
	new file:   frappe/core/doctype/recorder_suggested_index/recorder_suggested_index.py

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   frappe/core/doctype/recorder/recorder.py

Cherry-pick of b169f87 has failed:

On branch mergify/bp/version-15-hotfix/pr-26433
Your branch is ahead of 'origin/version-15-hotfix' by 4 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit b169f8780a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   frappe/core/doctype/recorder/recorder.js
	modified:   frappe/core/doctype/recorder/recorder.py
	modified:   frappe/core/doctype/recorder/test_recorder.py
	modified:   frappe/core/doctype/recorder_suggested_index/recorder_suggested_index.json
	modified:   frappe/tests/test_commands.py

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   frappe/commands/site.py

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@ankush ankush enabled auto-merge (squash) May 15, 2024 05:05
@ankush ankush merged commit 4b780cd into version-15-hotfix May 16, 2024
16 of 18 checks passed
@ankush ankush deleted the mergify/bp/version-15-hotfix/pr-26433 branch May 16, 2024 12:04
frappe-pr-bot pushed a commit that referenced this pull request May 22, 2024
# [15.28.0](v15.27.0...v15.28.0) (2024-05-22)

### Bug Fixes

* "Not assigned" filter ([#26468](#26468)) ([#26470](#26470)) ([ee6545b](ee6545b))
* compute modal title server side ([#26192](#26192)) ([3b787ac](3b787ac))
* condition of event participants ([#26300](#26300)) ([#26456](#26456)) ([31be0b9](31be0b9))
* default tree view ([07883a7](07883a7))
* Don't fiddle with child table indexes ([#26450](#26450)) ([#26453](#26453)) ([7e0d734](7e0d734))
* **DX:** Don't run CI if there are no tests (backport [#26457](#26457)) ([#26466](#26466)) ([72c693e](72c693e))
* enable fetch_from for Time fieldtype ([#25266](#25266)) ([#26471](#26471)) ([#26483](#26483)) ([f1267ca](f1267ca))
* **grid_row:** don't crash when undefined ([116ba1c](116ba1c))
* preserve checked items after a search ([befcce1](befcce1))

### Features

* **recorder:** Suggest index for optimizing query (backport [#26433](#26433)) ([#26435](#26435)) ([4b780cd](4b780cd))

### Performance Improvements

* num2words, babel, gettext, sentry imports ([#26475](#26475)) ([#26477](#26477)) ([e79e5f9](e79e5f9))
@frappe-pr-bot
Copy link
Collaborator

馃帀 This PR is included in version 15.28.0 馃帀

The release is available on GitHub release

Your semantic-release bot 馃摝馃殌

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants