Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PR: Profile in the IPython console to gain access to variables and get a result in case of cancellation #15372

Open
wants to merge 43 commits into
base: master
Choose a base branch
from
Open

PR: Profile in the IPython console to gain access to variables and get a result in case of cancellation #15372

wants to merge 43 commits into from
+1,698 −1,199

Conversation

impact27
Copy link
Contributor

@impact27 impact27 commented Apr 21, 2021 •
edited by ccordoba12
Loading

Description of Changes

Add profile_file and profile_cell functions to the shell with the same signature as runcell/debugcell and runfile/debugfile

This solves a few issues I have with profiling, including:

Issue(s) Resolved

Fixes #2378

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:

@pep8speaks
Copy link

pep8speaks commented Apr 21, 2021 •
edited
Loading

Hello @impact27! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 206:80: E501 line too long (95 > 79 characters)
Line 207:80: E501 line too long (89 > 79 characters)
Line 208:80: E501 line too long (98 > 79 characters)
Line 349:80: E501 line too long (89 > 79 characters)

Comment last updated at 2024-05-30 20:26:08 UTC

@impact27 impact27 changed the base branch from master to 5.x April 21, 2021 12:07
@impact27 impact27 changed the title PR: Profile script PR: Profile in the ipython console to gain access to variables and get a result in case of cancellation Oct 8, 2021
@ccordoba12
Copy link
Member

Hey @impact27, please post an animated gif/video of the new behavior.

@impact27
Copy link
Contributor Author

impact27 commented Oct 10, 2021 •
edited
Loading

plot

List of improvements:

  • Access to local variables in the console
  • Can compare several runs by profiling in different consoles
  • Can profile in the same environment as the code is run
  • Can get a result even if the run is interrupted (For example if the code reaches an infinite loop)
  • Adds %profile magic to profile a function or short line of code
  • Can profile a single cell
  • Can profile an unsaved file

@impact27
Copy link
Contributor Author

plot

@impact27 impact27 mentioned this pull request Nov 24, 2021
Merged
3 tasks
@impact27 impact27 mentioned this pull request Dec 29, 2021
Open
10 tasks
@impact27 impact27 mentioned this pull request Jun 11, 2022
Closed
@mrclary
Copy link
Contributor

mrclary commented Jun 11, 2022

@impact27, so this replaces the existing "Run profiler" menu item with "Profile file" and "Profile cell", correct? And the heavy lifting is moved to spyder-kernels so that the profiler is always run in an existing IPython Console environment (inheriting its os.environ, etc.), correct?

If the above is correct, then I agree that this PR should resolve #18010 and obviate PR #18031.

@impact27
Copy link
Contributor Author

@impact27, so this replaces the existing "Run profiler" menu item with "Profile file" and "Profile cell", correct? And the heavy lifting is moved to spyder-kernels so that the profiler is always run in an existing IPython Console environment (inheriting its os.environ, etc.), correct?

If the above is correct, then I agree that this PR should resolve #18010 and obviate PR #18031.

Yes exactly, the profiling would happen in the spyder-kernel, so there would be no difference with runfile and debugfile

@impact27 impact27 mentioned this pull request Jun 12, 2022
Merged
@mrclary
Copy link
Contributor

mrclary commented Jun 12, 2022

@impact27 and @ccordoba12, when will this PR be ready for review? Should this be included in 5.3.2 milestone?

@impact27 impact27 mentioned this pull request Jun 12, 2022
Merged
3 tasks
@impact27 impact27 changed the base branch from 5.x to master June 12, 2022 17:13
@impact27
Copy link
Contributor Author

This is ready for review, assuming the tests pass when they are fixed

@ccordoba12
Copy link
Member

@impact27, the plan for this is to merge first PR #17467, which has a significant refactoring about how things are run in Spyder, then PR #11186 because @andfoy and @steff456 need it to continue their work to move the editor to the new API, then this one.

@ccordoba12 ccordoba12 added this to the v6.0alpha1 milestone Jun 22, 2022
@andfoy
Copy link
Member

andfoy commented Jun 22, 2022

If the IPythonConsole will profile the files, we would prefer to use the viewer run API instead.

@impact27 impact27 force-pushed the profile_script branch 3 times, most recently from 075e4cc to cbbb9e9 Compare June 25, 2022 07:18
Quentin Peter added 4 commits August 5, 2023 07:57
git subrepo clone (merge) --branch=improve_namespace --force https://…
89f9538 
…github.com/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "e5f76b33f"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "e5f76b33f"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
git subrepo clone (merge) --branch=improve_namespace --force https://…
83883b0 
…github.com/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "9e8773c81"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "9e8773c81"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
Merge remote-tracking branch 'upstream/master' into profile_script
3fbac2b
temps folder on linux
ab3b70c
@ccordoba12 ccordoba12 modified the milestones: v6.0alpha2, v6.0alpha3 Aug 15, 2023
@ccordoba12
Copy link
Member

@impact27, please update this PR to see if we can merge it next.

@ccordoba12 ccordoba12 modified the milestones: v6.0alpha3, v6.0beta1 Dec 14, 2023
@ccordoba12 ccordoba12 modified the milestones: v6.0alpha4, v6.0beta1 Feb 6, 2024
Quentin Peter added 2 commits March 12, 2024 03:04
Merge branch 'master' into profile_script
d480379
git subrepo clone (merge) --branch=improve_namespace --force https://…
e6aece7 
…github.com/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "920e94057"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "920e94057"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
@ccordoba12 ccordoba12 modified the milestones: v6.0alpha5, v6.0beta1 Mar 12, 2024
Quentin Peter added 3 commits March 14, 2024 06:39
Merge branch 'master' into profile_script
bf4d6fa
git subrepo clone (merge) --branch=improve_namespace --force https://…
42c5622 
…github.com/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "af3fe0a6e"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "af3fe0a6e"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
Merge remote-tracking branch 'upstream/master' into profile_script
6e66329
@impact27 impact27 mentioned this pull request Mar 17, 2024
Open
git subrepo clone --branch=improve_namespace --force https://github.c…
d3d79ac 
…om/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "281be2ba8"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "281be2ba8"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
@ccordoba12 ccordoba12 modified the milestones: v6.0beta1, v6.1.0 May 6, 2024
Quentin Peter added 5 commits May 23, 2024 15:36
Merge remote-tracking branch 'upstream/master' into profile_script
8532ccd
git subrepo clone (merge) --branch=improve_namespace --force https://…
e22b6f9 
…github.com/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "9acc3dd22"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "9acc3dd22"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
fix test
e41455f
Merge branch 'master' into profile_script
413f619
git subrepo clone (merge) --branch=improve_namespace --force https://…
2b8343a 
…github.com/impact27/spyder-kernels.git external-deps/spyder-kernels

subrepo:
  subdir:   "external-deps/spyder-kernels"
  merged:   "53b9706d9"
upstream:
  origin:   "https://github.com/impact27/spyder-kernels.git"
  branch:   "improve_namespace"
  commit:   "53b9706d9"
git-subrepo:
  version:  "0.4.5"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "aa416e4"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ccordoba12 ccordoba12 ccordoba12 left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@impact27 impact27

Labels
component:Profiler type:Enhancement
Projects
None yet
Milestone
v6.1.0
Development

Successfully merging this pull request may close these issues.

Spyder profiler uses wrong python when using virtualenv
7 participants
@impact27 @pep8speaks @ccordoba12 @mrclary @andfoy @dalthviz @CAM-Gerlach