Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kernel finding refactor #4995

Merged
merged 95 commits into from
Mar 9, 2021
Merged

Kernel finding refactor #4995

merged 95 commits into from
Mar 9, 2021

Conversation

rchiodo
Copy link
Contributor

@rchiodo rchiodo commented Mar 2, 2021

Rework how we find kernels.

Two new classes:

  • LocalKernelFinder - (based on old KernelFinder class) that is used to find kernels on a local machine. It looks for kernel.json files in all the known jupyter locations and interpreter locations
  • RemoteKernelFinder - used to find kernels based on a jupyter connection. Asks jupyter for the kernel list and the active session list

Both of these classes then use the same algorithm to find matches for a notebook's metadata and/or an interpreter. (Algorithm was moved to a separate file)

Unit tests were made to test the different possible conditions that we have for finding kernels.

All other classes dealing with finding kernels were changed to use these classses. Quick picks for kernels just use these classes and then wrap the result in a quick pick item.

@rchiodo rchiodo marked this pull request as ready for review March 9, 2021 00:14
@rchiodo rchiodo requested a review from a team as a code owner March 9, 2021 00:14
@rchiodo rchiodo changed the title Draft: Kernel finding refactor Kernel finding refactor Mar 9, 2021
@traceDecorators.verbose('Find kernel spec')
@captureTelemetry(Telemetry.KernelFinderPerf)
public async findKernel(
resource: Resource,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this should be optional

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leave this as is. undefined is supported value.

This way developer needs to make a decision and explicitly pass the value in, instead of accidentally forgetting it.
Fyi, we have had many issues where these (uri) haven't been passed on when they are optional.

}

// See if the version is the same
if (interpreter && interpreter.version && spec && spec.name && nbMetadataLanguage === PYTHON_LANGUAGE) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this resolves #4941. I was about to take a look at this but figured that I should wait until the refactor. Looks like it should match on a python3 spec name to prefer a python3 interpreter over plain python.

Copy link
Member

@IanMatthewHuff IanMatthewHuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a while to get through. Looks like a solid improvement. Cleans up some of our messiest code areas. Had a few suggestions, mostly on some areas where I think that more logging (gated on your new env var) might be helpful.

@rchiodo rchiodo merged commit 2d899c2 into main Mar 9, 2021
@rchiodo rchiodo deleted the rchiodo/refactor_kernel_finding branch March 9, 2021 18:30
DonJayamanne pushed a commit that referenced this pull request Mar 23, 2021
* New idea for kernel finding

* Getting some more of the tests running

* Move interpreters down to the finder level

* Partially building

* Change when extensions load

* Fix some merge problems

* Fix everything except the kernel selector tests

* Remove some more code

* Remove some more methods

* REname build steps

* Get everything building

* Outline for tests for kernel finders

* First local kernel finder test

* More tests and handling kernelspecs from interpreters

* Fix build errors

* List kernels passing

* Get all local kernel finder tests to pass

* Add remote tests

* Rework unit tests for jupyterKernelService

* kernel service unit tests passing

* Fix sub command unit tests

* Fix other tests and eliminate unnecessary test for selections

* Fix execution unit tests

* Fix display of kernels

* Remove dupes that were registered

* Turn on linting

* Format documents that were missed

* Fix build problems

* Code review comments

* Add telemetry for kernel retry

* Add in cancel token and remove unnecessary code

* Fix having no interpreter

* Fix realpath on Unix

* More kernel test fixes

* More review feedback and eliminate jupyter tests from functional. They do not run anymore because of usage of file system.

* More tests to eliminate

* More tests to skip

* Put launch.json back

* Fix build error

* Fix build error

* Fix sorting for interpreter based kernels

* Use endswith instead of includes

* Need user writable location for jupyter kernels

* Update comment

* Update linter

* Put back skip for tests

* Notebook metadata not required for a match

* Add more logging

* More logging and disabling more jupyter tests

* Put back getRealPath function. Behavior of nodejs func is different and causes crashes

* Skip real path for mac/linux

* Fix unit tests and output more information on failures

* Make sure true kernel name is in the notebook

* Get remote working

* Fix linter

* Fix problem when no kernel entry in notebook

* Fix smoke test and add more logging for vscode tests

* More logging and some waiting for a kernel to actually switch

* Put open file back

* More logging and change kernelspec name to be valid path

* Add more logging

* Fix id to be unique

* Update comment to force another run

* Update linter

* Use the unique id to compare kernels for equality

* Fix remote id problems

* More logging and more precise on matches for switch

* More logging and try using label

* Try more logging

* Fix prettier

* Add a bunch more logging

* Switch doesn't seem to happen before execute

* Track kernel data in the document before starting the notebook

* More logging

* Disable failing tests on jupyter

* Fix bug with changing kernel

* Skip install when non raw

* Figure out correct kernel names

* Resolve name during test

* Wrong kernel name

* Use includes in kernel search too

* Try interpreter path instead

* Fix tryGetRealPath to actually wait for the result

* More tracing and other review feedback

* FIx comment on remoteKernelFinder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants