From eb94178c503c1a7f15f058c12e7e29297c27367e Mon Sep 17 00:00:00 2001 From: Don Jayamanne Date: Mon, 24 Feb 2020 13:44:14 -0800 Subject: [PATCH] Ds custom editor squashed (#1543) * Setup usage of the new API * Implement base custom editor support (#9812) * Setup usage of the new API * Partial ideas * Idea for splitting * Idea in place * Get all of the tests to build * Enable proposed api * Still not working but setting more options. * Get web views to load * Fix save and save as to use VS code commands * Fix unit tests * Fix a number of functional tests * REmove autoSave tests as not needed anymore * Add news entry * Review comments * Code review comments * Update PR validation list * Fix nyc compiler problems. * Try fixing the toggle markdown test * Ids of new cells must be provided to actions (#9833) * ids of new cells must be provided to actions * Comments * Oops * Use new filesystem in DataScience functional tests (#9839) * Fix merge problem * Run tests against insiders (#9872) * Run tests against insiders * Disable noisy logging * Restore changes * Ensure we have a base type for all action payloads (#9840) For #9340 Ensures all actions have a base type, to pass information such whether the action was invoked to sync state for multiple editors (pointing to same file) in same session or across sessions. This PR only adds the ability to pass such new information to all actions. * Remove root level flags used to track cell selection and focus (#9882) For #9340 To be merged after #9840 Basically I got rid of the root level state info that keeps track of the cell that's selected and focused. This required us to keep the two in sync along with the state at the cell level. When syncing information between multiple editors this got messy. removing this removes the issues where things could go out of sync, i.e. two ways to storing the same thing. Now we get selected information from the cells directly - single source of truth. * Synchronize contents in multiple editors for same nb file (#9886) For #9340 Note This PR adds the infrastructure necessary to sync edits & the like across editors for the same file Code has been commented out to ensure some remaining bits work properly Waiting on some of the new changes from @rchiodo from his undo PR. Remaining changes Ensuring we do not end up in a crazy loop that will cause one editor to propagate changes to the other, then that one back to this & so on Sync cell execution (currently creation of new cells isn't sync'ed across) Find a bullet proof solution for syncing focused states (this messes up everything - we don't need this in syncing editors, but if required for live share, then we need to ensure focus states work correctly - depending on whether liveshare even needs such a feature?) * Address code review comments * Fix test * Fix linter * Remove unused property * More changes * Run tests against insiders (#9919) * We have code in extension that displays a dialog (preventing extension from loading) in insiders version of VSC. * Run tests against insiders * Restore env variable * Implement undo support for the custom editor (#9946) * Partially working undo/redo * Concept created using update message/command * Put the storage object in the global map * Look for more data during undo/redo * New idea with reverse edit and disable undo/redo in monaco * Potentially working with tracking edits * Fix completions for interactive * Fix unit tests * First working functional test of undo * More testing of undo * More tests * Close suggestions on undo * Fix other functional tests * Fix undo misses * Fix focus changes on undo * Fixup after merge * Add news entry * Put back launch.json * Get rid of asyncness of sendCellsToWebView * Add a comment about why we remove \r * Prevent swapCells from doing anything if same cell * Fix changed event for clear * Actually remove all state from editor class. Try to fix undo on save * Fix func test in DS failing due to FS issues (#9995) * Fix dirty state not matching save button state (#9993) * Fix dirty state not matching save button state * Fix modify to update UI too Make a 'save' point for the undo count. Looks like this is what the editor is doing. * Minor changes to dispatchers (action creators will dispatch multiple actions) (#9920) * Fixes * Remove cyclic messages * Misc * Address code review comments * Oops * More oops * More fixes * Address code review comments * Address code review comments * Fix tests * Retry flaky test 3 times. * Fixes * Fix problem with undoing entire stack (#10018) * Refactor how messages are posted to reducers and to the extension (#10019) Rather than having a developer call create and then arg.queue, this replaces that with one function. Hides the intricacies of how something is posted or queued. Also allows us to add additional logic to control what messages can be posted (for synchronization) instead of developer having to do this in every reducer. Simple harmless refactor * Complete refactoring of creation and queueing of actions (#10036) * Some how I had not included some of the other remaining arg.queueAction into the original PR #10019 * This PR merely brings them from (I had these in the same branch, but failed to include them). * Same as #10019, just a refactor to use a common function to create an action and dispatch it in one step. & removed some of the old syncing stuff. * Prettier * Get everything building again * Fix source maps and undoing the first cell * Fix order of cells inserted (#10229) Cells weren't inserted into the model in the right order. * Insert cell using `a` * Insert cell using `b` * Save NB and reopen * All the new cells are at the very bottom Added https://github.com/microsoft/vscode-python/issues/10230 and moved into backlog and assigned to ds_editor epic. I think in light of this issue it must be done. Else it'll be bad for users to report this. * Fixes to ds custom editor (#10257) * Fix more ds functional tests (#10260) * Eureka * misc * Merge master into ds custom (#10261) * fixing a typo in CONTRIBUTING.md (#10044) * fix for button backgrounds (#10234) * add kernel connection sys info string (#10236) * Fix to return env variables of interpreter that is not current interpreter (#10251) For #10250 * Bug fix * Support jupyter output in the remote scenario too (#10241) * Add some output for remote situations * Add news entry * Skip flaky FS test (#10244) Merely disables test for #10240 * Switch experiments settings scope to machine (#10237) * Switch experiments settings scope to machine * Add news item * Show quickfixes for launch.json (#10245) * Added implementation * Added tests * Moved into providers * Add code actions vscode mock and convert .test.ts into unit.test.ts * Rename folders * Dispose registered services * News entry * Added unit tests * Rename core service * Reduce the number of build agents used by CI. (#10221) * Drop extra jobs from the PR-validation pipeline. This reduces the number of build agents we are using most frequently, without sacrificing much coverage. Note that this relies on 2 things: * sufficient unit test coverage * manual "full CI" runs when there may be OS-specific concerns With this change: * all tests get run on 3.x on linux (including smoke) * for 2.7 only the unit and functional tests are run (and only on linux) * on Windows and OSX, only functional and "single workspace" tests are run The total number of test jobs (and hence agents) drops from 13 to 8. * Drop extra jobs from the PR-merge pipeline. This change relies on the nightly CI run covering the full matrix. The total number of test jobs (and hence agents) drops from 39 to 24. * Limit # of parallel jobs in each job matrix in the PR-merge and nightly pipelines. This reduces the max number of agents used by the PR-merge pipeline to 8 (from 24). For the nightly pipeline it goes down from 84 to 12. Both will take longer to complete, but that shouldn't be a major problem. * Add a "manual build" CI pipeline for faster turnaround. * Do not test internal tools on lower than Python 3.7. * Drop the temporary workaround jobs. * (again) Do not test internal tools on lower than Python 3.7. * Improve the perf of functional tests running with real jupyter (#10242) * Improve the perf of subsequent tests by caching interpreters * Add back the nightly flake * Remove coverage * Add news entry * Use a static map to allow promise to be cleared on new interpreters (as it was before) Co-authored-by: Panpan Lin Co-authored-by: Ian Huff Co-authored-by: Rich Chiodo Co-authored-by: Karthik Nadig Co-authored-by: Kartik Raj Co-authored-by: Eric Snow * Fix startup and shutdown tests (#10284) * Merge master again into ds/custom_editor (#10295) * fixing a typo in CONTRIBUTING.md (#10044) * fix for button backgrounds (#10234) * add kernel connection sys info string (#10236) * Fix to return env variables of interpreter that is not current interpreter (#10251) For #10250 * Bug fix * Support jupyter output in the remote scenario too (#10241) * Add some output for remote situations * Add news entry * Skip flaky FS test (#10244) Merely disables test for #10240 * Switch experiments settings scope to machine (#10237) * Switch experiments settings scope to machine * Add news item * Show quickfixes for launch.json (#10245) * Added implementation * Added tests * Moved into providers * Add code actions vscode mock and convert .test.ts into unit.test.ts * Rename folders * Dispose registered services * News entry * Added unit tests * Rename core service * Reduce the number of build agents used by CI. (#10221) * Drop extra jobs from the PR-validation pipeline. This reduces the number of build agents we are using most frequently, without sacrificing much coverage. Note that this relies on 2 things: * sufficient unit test coverage * manual "full CI" runs when there may be OS-specific concerns With this change: * all tests get run on 3.x on linux (including smoke) * for 2.7 only the unit and functional tests are run (and only on linux) * on Windows and OSX, only functional and "single workspace" tests are run The total number of test jobs (and hence agents) drops from 13 to 8. * Drop extra jobs from the PR-merge pipeline. This change relies on the nightly CI run covering the full matrix. The total number of test jobs (and hence agents) drops from 39 to 24. * Limit # of parallel jobs in each job matrix in the PR-merge and nightly pipelines. This reduces the max number of agents used by the PR-merge pipeline to 8 (from 24). For the nightly pipeline it goes down from 84 to 12. Both will take longer to complete, but that shouldn't be a major problem. * Add a "manual build" CI pipeline for faster turnaround. * Do not test internal tools on lower than Python 3.7. * Drop the temporary workaround jobs. * (again) Do not test internal tools on lower than Python 3.7. * Improve the perf of functional tests running with real jupyter (#10242) * Improve the perf of subsequent tests by caching interpreters * Add back the nightly flake * Remove coverage * Add news entry * Use a static map to allow promise to be cleared on new interpreters (as it was before) * Merge release branch into master (#10258) * Davidkutu/ds interactive window focus fix (#9895) * Fixed the focus on the interactive window when pressing ctrl +2 * the new prop 'focusPending' needs to change in order to give focus * added functional test * In test: force focus to buttons to take it away from * finished the functional test * updated changelog * deleted bug#.md * Fix variables on mac and linux (#9897) (#9903) * Fix regexes to work on other platforms * Fix data frame viewer * Fix linter problems Co-authored-by: Don Jayamanne Co-authored-by: Don Jayamanne * Fix focusing of cells when navigating cells using up/down arrow (#9885) (#9908) For #9884 * Disable webHostNotebook experiment and fix name of exp (#9894) (#9907) Disable an experiment Fix name of an existing experiment * Davidkutu/ds scrapbook render fix (#9912) * check if we support mimetypes before loading html to avoid crashing * Silently skip rendering of the scapbook mime types * added news file * updated changelog and deleted news file * Disable the terminal activation experiment (#9937) Disable until next release of VSC (for hidden terminals). Disabled because a terminal gets displayed to the user and then hidden (doesn't happen in insiders, we will need to wait for their new API to hide terms) * Notebook should use kernel from metadata (#9936) For #9927 * Notebook should use kernel from metadata * Fix formatting * Port server cache fix to release (#9932) * Port variable restart fix (#9938) * Fix variable explorer when restarting a kernel (#9931) * Make sure to clear variable list on reset * Add news entry * Update changelog * Update changelog * Do not override kernels info in other notebooks (#9958) For #9951, #9950, #9949 * Do not override kernels info in other notebooks * Fixes * Disables the use of a terminal to activate an environment (#9967) * Disables the use of a terminal to activate an environment * Fix tests * Fix linter * Port server name fix to release (#9970) * When starting Jupyter activate conda env (#9972) * If interpreter is provided use that to check if is conda environment * Fix tests For #9973 Currently conda isn't getting activated when auto starting Jupyter server. * Porting K8 fix to release (#9987) * Update version and changelog date for release (#10022) * Ignore FS errors from stat() in some cases. (#10032) (for #9901) The earlier FS code ignored errors in a couple of situations. Recent changes inadvertently stopped ignoring them. This change restores the earlier behavior (and adds logging of the errors). This is a backport of #10021. * Update chang log (#10041) * Install jupyter instead of installing kernel spec (#10080) (#10083) * Install jupyter instead of installing kernel spec For #10071 * Fix for zh-tw loading issue. (#10085) (#10089) * Update version and changelog for point release. (#10091) * Update version and changelog for point release. * Add note to change log. * Port fixes for second point release. (#10146) * Better messaging on notebook fail (#10056) * Install jupyter instead of installing kernel spec (#10080) * Install jupyter instead of installing kernel spec For #10071 * Eliminate variable value when computing data frame info (#10081) * Fix ndarray types to be viewable again (#10093) * Eliminate variable value when computing data frame info * Fix ndarrays to work again Add test to make sure we don't regress this again * Rchiodo/kernel telemetry (#10115) * Add duration to select local/remote kernel * Add notebook language telemetry * Add news entries * #9883 telemetry * News entry * Another spot for kernel spec failure * Add telemetry on product install * Fix install telemetry * Undo launch.json change * Handle other cases * Better way to handle case * Wrong event for jupyter install * Fix unit tests * Clear variables when restarting regardless if visible or not (#10117) * Use different method for checking if kernelspec is available (#10114) * Use different method for checking if kernelspec is available * Fix unit tests * More logging for kernelspec problems (#10132) * More logging for kernelspec problems * Actually capture the exception on the new code * Not actually using output if first exception still there. * Actually only return output on one of the expected calls. * Fix nightly flake * Check our saved jupyter interpreters before allowing any of them to be used as active interpreters (#10113) * Update changelog and package.json * Missing part of changelog * Fix tests and linter problems Co-authored-by: Ian Huff Co-authored-by: Don Jayamanne * Cherry pick more fixes for feb point2 (#10183) * Ensure DS auto start code is non-blocking (#10171) * Perf fixes * Add news entry * Include interperter name in message (#10174) For #10071 Fix to ensure interpreter is included in message, here's the actual message seen by the user. Error: Jupyter cannot be started. Error attempting to locate jupyter: 'Kernelspec' module not installed in the selected interpreter ({0}). Note the {0} * Ignore display name when searching interpreters (#10175) Partial fix for #10173 Comparing against path of interpreter alone is sufficient. Figured there's no harm in trying to minimise occurrences of dup kernels (basically just compare against python executable path, as thats sufficient and always accurate) Display name: Can change from version to version of Python extension (i.e. it shouldn't have been used as a unique identifier) Display name can change after extension loads and more information about interpreter is available. * Track cold/warm times to execute notebook cells (#10180) For #10176 * fix SVG mimetype name (#10181) (#10185) * Update change log * Some more ds telemetry (#10218) * Add telemetry to capture perceived time taken to start jupyter and run cell (#10217) * Add telemetry * Only for nb For #10098 For #10212 * Fix change log * Fix linter * Update date in change log * Point release with fix for Issue 10250 (#10254) * Fix to return env variables of interpreter that is not current interpreter (#10251) For #10250 * Bug fix * Update change log * Fix date * Oops * Fix linter * Clean up * More clean up * Some more text fixes Co-authored-by: David Kutugata Co-authored-by: Rich Chiodo Co-authored-by: Don Jayamanne Co-authored-by: Ian Huff Co-authored-by: Eric Snow Co-authored-by: Panpan Lin Co-authored-by: Ian Huff Co-authored-by: Rich Chiodo Co-authored-by: Karthik Nadig Co-authored-by: Kartik Raj Co-authored-by: Eric Snow Co-authored-by: David Kutugata * Restore old webview provider * Ooops * Misc * Fix typos and linter issues * Save and save as * Fix test * More fixes * Add undo/redo Co-authored-by: Rich Chiodo Co-authored-by: Panpan Lin Co-authored-by: Ian Huff Co-authored-by: Karthik Nadig Co-authored-by: Kartik Raj Co-authored-by: Eric Snow Co-authored-by: David Kutugata --- .vscode/launch.json | 8 +- .../webpack.datascience-ui.config.builder.js | 2 +- experiments.json | 2 - gulpfile.js | 206 +++- news/1 Enhancements/9255.md | 1 + news/1 Enhancements/9821.md | 1 + news/2 Fixes/10250.md | 1 + package.datascience-ui.dependencies.json | 442 ++++----- package.json | 6 +- package.nls.tr.json | 2 +- settings.json | 59 ++ src/client/common/application/commands.ts | 5 + .../common/application/customEditorService.ts | 35 + src/client/common/application/types.ts | 107 +++ .../common/application/webPanels/webPanel.ts | 34 +- src/client/common/serviceRegistry.ts | 3 + src/client/datascience/constants.ts | 3 + src/client/datascience/data-viewing/types.ts | 24 +- .../intellisense/intellisenseDocument.ts | 119 ++- .../intellisense/intellisenseProvider.ts | 233 +++-- .../interactive-common/interactiveBase.ts | 64 +- .../interactiveWindowTypes.ts | 175 +++- .../interactive-common/synchronization.ts | 219 +++++ .../datascience/interactive-common/types.ts | 34 + .../interactive-ipynb/autoSaveService.ts | 152 --- .../interactive-ipynb/nativeEditor.ts | 909 +++--------------- .../nativeEditorCommandListener.ts | 7 +- .../nativeEditorOldWebView.ts | 286 ++++++ .../interactive-ipynb/nativeEditorProvider.ts | 449 +++++---- .../nativeEditorProviderOld.ts | 262 +++++ .../interactive-ipynb/nativeEditorStorage.ts | 614 ++++++++++++ .../interactive-window/interactiveWindow.ts | 27 +- .../interactiveWindowCommandListener.ts | 4 +- .../datascience/jupyter/jupyterExporter.ts | 68 +- src/client/datascience/messages.ts | 20 +- src/client/datascience/serviceRegistry.ts | 20 +- src/client/datascience/types.ts | 29 +- src/client/datascience/webViewHost.ts | 9 +- src/client/telemetry/index.ts | 4 + .../nosetest/services/parserService.ts | 1 - .../data-explorer/mainPanel.tsx | 2 +- .../history-react/interactiveCell.tsx | 25 +- .../history-react/redux/actions.ts | 177 ++-- .../history-react/redux/mapping.ts | 89 +- .../history-react/redux/reducers/creation.ts | 53 +- .../history-react/redux/reducers/effects.ts | 20 +- .../history-react/redux/reducers/execution.ts | 32 +- .../history-react/redux/reducers/index.ts | 57 +- .../interactive-common/cellInput.tsx | 19 +- .../interactive-common/code.tsx | 22 +- .../interactive-common/editor.tsx | 111 +-- .../interactive-common/mainState.ts | 39 +- .../interactive-common/markdown.tsx | 20 +- .../interactive-common/redux/helpers.ts | 134 +++ .../interactive-common/redux/postOffice.ts | 139 +-- .../redux/reducers/commonEffects.ts | 46 +- .../redux/reducers/helpers.ts | 31 +- .../redux/reducers/kernel.ts | 46 +- .../redux/reducers/monaco.ts | 83 +- .../redux/reducers/transfer.ts | 251 +++-- .../redux/reducers/types.ts | 118 ++- .../redux/reducers/variables.ts | 118 ++- .../interactive-common/redux/store.ts | 107 ++- .../native-editor/nativeCell.tsx | 33 +- .../native-editor/nativeEditor.tsx | 31 +- .../native-editor/redux/actions.ts | 244 +++-- .../native-editor/redux/mapping.ts | 104 +- .../native-editor/redux/reducers/creation.ts | 356 +++++-- .../native-editor/redux/reducers/effects.ts | 129 +-- .../native-editor/redux/reducers/execution.ts | 171 ++-- .../native-editor/redux/reducers/index.ts | 67 +- .../native-editor/redux/reducers/movement.ts | 105 +- src/datascience-ui/plot/mainPanel.tsx | 2 +- .../react-common/monacoEditor.tsx | 105 +- .../react-common/monacoHelpers.ts | 113 +++ src/datascience-ui/react-common/postOffice.ts | 50 - src/datascience-ui/react-common/reduxUtils.ts | 10 +- .../languageServerFolderService.unit.test.ts | 1 + .../activation/serviceRegistry.unit.test.ts | 1 + .../insidersExtensionService.unit.test.ts | 6 + .../installer.invalidPath.unit.test.ts | 1 - .../common/installer/installer.unit.test.ts | 2 +- src/test/common/process/logger.unit.test.ts | 1 - .../datascience/dataScienceIocContainer.ts | 58 +- .../datascience/intellisense.unit.test.ts | 271 +++++- .../nativeEditor.unit.test.ts | 679 ------------- .../nativeEditorProvider.unit.test.ts | 169 +--- .../nativeEditorStorage.unit.test.ts | 555 +++++++++++ ...terSubCommandExecutionService.unit.test.ts | 1 + .../kernels/kernelSwitcher.unit.test.ts | 2 +- .../datascience/mockCustomEditorService.ts | 134 +++ src/test/datascience/mockFileSystem.ts | 23 + .../nativeEditor.functional.test.tsx | 634 ++++-------- .../datascience/nativeEditorTestHelpers.tsx | 13 +- src/test/datascience/testHelpers.tsx | 20 +- .../datascience/testNativeEditorProvider.ts | 27 +- src/test/debuggerTest.ts | 3 +- .../format/extension.onTypeFormat.test.ts | 1 + src/test/multiRootTest.ts | 5 +- src/test/refactor/rename.test.ts | 1 + src/test/serviceRegistry.ts | 2 +- src/test/standardTest.ts | 5 +- src/test/testing/pytest/pytest.run.test.ts | 1 + .../pytest/pytest.testMessageService.test.ts | 1 + .../testing/unittest/unittest.run.test.ts | 1 + tsconfig.datascience-ui.json | 20 +- tsconfig.extension.json | 12 +- tsconfig.json | 6 +- tsfmt.json | 30 +- tslint.json | 36 +- types/vscode.proposed.d.ts | 119 +++ 111 files changed, 6211 insertions(+), 4535 deletions(-) create mode 100644 news/1 Enhancements/9255.md create mode 100644 news/1 Enhancements/9821.md create mode 100644 news/2 Fixes/10250.md create mode 100644 settings.json create mode 100644 src/client/common/application/customEditorService.ts create mode 100644 src/client/datascience/interactive-common/synchronization.ts create mode 100644 src/client/datascience/interactive-common/types.ts delete mode 100644 src/client/datascience/interactive-ipynb/autoSaveService.ts create mode 100644 src/client/datascience/interactive-ipynb/nativeEditorOldWebView.ts create mode 100644 src/client/datascience/interactive-ipynb/nativeEditorProviderOld.ts create mode 100644 src/client/datascience/interactive-ipynb/nativeEditorStorage.ts create mode 100644 src/datascience-ui/interactive-common/redux/helpers.ts create mode 100644 src/datascience-ui/react-common/monacoHelpers.ts delete mode 100644 src/test/datascience/interactive-ipynb/nativeEditor.unit.test.ts create mode 100644 src/test/datascience/interactive-ipynb/nativeEditorStorage.unit.test.ts create mode 100644 src/test/datascience/mockCustomEditorService.ts create mode 100644 src/test/datascience/mockFileSystem.ts create mode 100644 types/vscode.proposed.d.ts diff --git a/.vscode/launch.json b/.vscode/launch.json index 39f2c4888690..9499c23a2b95 100644 --- a/.vscode/launch.json +++ b/.vscode/launch.json @@ -8,7 +8,9 @@ "request": "launch", "runtimeExecutable": "${execPath}", "args": [ - "--extensionDevelopmentPath=${workspaceFolder}" + "--extensionDevelopmentPath=${workspaceFolder}", + "--enable-proposed-api", + "ms-python.python" ], "stopOnEntry": false, "smartStep": true, @@ -16,10 +18,10 @@ "outFiles": [ "${workspaceFolder}/out/**/*" ], - "preLaunchTask": "Compile", + // "preLaunchTask": "Compile", "skipFiles": [ "/**" - ] + ], }, { "name": "Extension inside container", diff --git a/build/webpack/webpack.datascience-ui.config.builder.js b/build/webpack/webpack.datascience-ui.config.builder.js index b050d403295f..928db7271cd9 100644 --- a/build/webpack/webpack.datascience-ui.config.builder.js +++ b/build/webpack/webpack.datascience-ui.config.builder.js @@ -100,7 +100,7 @@ function buildConfiguration(isNotebook) { chunkFilename: `[name].bundle.js` }, mode: 'development', // Leave as is, we'll need to see stack traces when there are errors. - devtool: 'source-map', + devtool: isProdBuild ? 'source-map' : 'inline-source-map', optimization: { minimize: isProdBuild, minimizer: isProdBuild ? [new TerserPlugin({ sourceMap: true })] : [], diff --git a/experiments.json b/experiments.json index ecd95ff6bac1..966b18080041 100644 --- a/experiments.json +++ b/experiments.json @@ -1,4 +1,3 @@ - [ { "name": "AlwaysDisplayTestExplorer - experiment", @@ -115,4 +114,3 @@ "max": 100 } ] - diff --git a/gulpfile.js b/gulpfile.js index 608838f2bfb1..ba4b5a6b2b8e 100644 --- a/gulpfile.js +++ b/gulpfile.js @@ -117,8 +117,32 @@ gulp.task('check-datascience-dependencies', () => checkDatascienceDependencies() const webpackEnv = { NODE_OPTIONS: '--max_old_space_size=9096' }; async function buildDataScienceUI() { - await spawnAsync('npm', ['run', 'webpack', '--', '--config', './build/webpack/webpack.datascience-ui-notebooks.config.js', '--mode', 'production'], webpackEnv); - await spawnAsync('npm', ['run', 'webpack', '--', '--config', './build/webpack/webpack.datascience-ui-viewers.config.js', '--mode', 'production'], webpackEnv); + await spawnAsync( + 'npm', + [ + 'run', + 'webpack', + '--', + '--config', + './build/webpack/webpack.datascience-ui-notebooks.config.js', + '--mode', + 'production' + ], + webpackEnv + ); + await spawnAsync( + 'npm', + [ + 'run', + 'webpack', + '--', + '--config', + './build/webpack/webpack.datascience-ui-viewers.config.js', + '--mode', + 'production' + ], + webpackEnv + ); } gulp.task('compile-webviews', async () => { @@ -127,17 +151,37 @@ gulp.task('compile-webviews', async () => { gulp.task('webpack', async () => { // Build node_modules. - await buildWebPack('production', ['--config', './build/webpack/webpack.extension.dependencies.config.js'], webpackEnv); + await buildWebPack( + 'production', + ['--config', './build/webpack/webpack.extension.dependencies.config.js'], + webpackEnv + ); // Build DS stuff (separately as it uses far too much memory and slows down CI). // Individually is faster on CI. - await buildWebPack('production', ['--config', './build/webpack/webpack.datascience-ui-notebooks.config.js'], webpackEnv); - await buildWebPack('production', ['--config', './build/webpack/webpack.datascience-ui-viewers.config.js'], webpackEnv); + await buildWebPack( + 'production', + ['--config', './build/webpack/webpack.datascience-ui-notebooks.config.js'], + webpackEnv + ); + await buildWebPack( + 'production', + ['--config', './build/webpack/webpack.datascience-ui-viewers.config.js'], + webpackEnv + ); // Run both in parallel, for faster process on CI. // Yes, console would print output from both, that's ok, we have a faster CI. // If things fail, we can run locally separately. if (isCI) { - const buildExtension = buildWebPack('extension', ['--config', './build/webpack/webpack.extension.config.js'], webpackEnv); - const buildDebugAdapter = buildWebPack('debugAdapter', ['--config', './build/webpack/webpack.debugadapter.config.js'], webpackEnv); + const buildExtension = buildWebPack( + 'extension', + ['--config', './build/webpack/webpack.extension.config.js'], + webpackEnv + ); + const buildDebugAdapter = buildWebPack( + 'debugAdapter', + ['--config', './build/webpack/webpack.debugadapter.config.js'], + webpackEnv + ); await Promise.all([buildExtension, buildDebugAdapter]); } else { await buildWebPack('extension', ['--config', './build/webpack/webpack.extension.config.js'], webpackEnv); @@ -157,8 +201,12 @@ async function updateBuildNumber(args) { // Change version number const versionParts = packageJson['version'].split('.'); - const buildNumberPortion = versionParts.length > 2 ? versionParts[2].replace(/(\d+)/, args.buildNumber) : args.buildNumber; - const newVersion = versionParts.length > 1 ? `${versionParts[0]}.${versionParts[1]}.${buildNumberPortion}` : packageJson['version']; + const buildNumberPortion = + versionParts.length > 2 ? versionParts[2].replace(/(\d+)/, args.buildNumber) : args.buildNumber; + const newVersion = + versionParts.length > 1 + ? `${versionParts[0]}.${versionParts[1]}.${buildNumberPortion}` + : packageJson['version']; packageJson['version'] = newVersion; // Write back to the package json @@ -167,7 +215,10 @@ async function updateBuildNumber(args) { // Update the changelog.md if we are told to (this should happen on the release branch) if (args.updateChangelog) { const changeLogContents = await fsExtra.readFile('CHANGELOG.md', 'utf-8'); - const fixedContents = changeLogContents.replace(/##\s*(\d+)\.(\d+)\.(\d+)\s*\(/, `## $1.$2.${buildNumberPortion} (`); + const fixedContents = changeLogContents.replace( + /##\s*(\d+)\.(\d+)\.(\d+)\s*\(/, + `## $1.$2.${buildNumberPortion} (` + ); // Write back to changelog.md await fsExtra.writeFile('CHANGELOG.md', fixedContents, 'utf-8'); @@ -180,7 +231,11 @@ async function updateBuildNumber(args) { async function buildWebPack(webpackConfigName, args, env) { // Remember to perform a case insensitive search. const allowedWarnings = getAllowedWarningsForWebPack(webpackConfigName).map(item => item.toLowerCase()); - const stdOut = await spawnAsync('npm', ['run', 'webpack', '--', ...args, ...['--mode', 'production', '--devtool', 'source-map']], env); + const stdOut = await spawnAsync( + 'npm', + ['run', 'webpack', '--', ...args, ...['--mode', 'production', '--devtool', 'source-map']], + env + ); const stdOutLines = stdOut .split(os.EOL) .map(item => item.trim()) @@ -188,13 +243,20 @@ async function buildWebPack(webpackConfigName, args, env) { // Remember to perform a case insensitive search. const warnings = stdOutLines .filter(item => item.startsWith('WARNING in ')) - .filter(item => allowedWarnings.findIndex(allowedWarning => item.toLowerCase().startsWith(allowedWarning.toLowerCase())) == -1); + .filter( + item => + allowedWarnings.findIndex(allowedWarning => + item.toLowerCase().startsWith(allowedWarning.toLowerCase()) + ) == -1 + ); const errors = stdOutLines.some(item => item.startsWith('ERROR in')); if (errors) { throw new Error(`Errors in ${webpackConfigName}, \n${warnings.join(', ')}\n\n${stdOut}`); } if (warnings.length > 0) { - throw new Error(`Warnings in ${webpackConfigName}, Check gulpfile.js to see if the warning should be allowed., \n\n${stdOut}`); + throw new Error( + `Warnings in ${webpackConfigName}, Check gulpfile.js to see if the warning should be allowed., \n\n${stdOut}` + ); } } function getAllowedWarningsForWebPack(buildConfig) { @@ -282,7 +344,16 @@ gulp.task('installPythonRequirements', async () => { // See https://github.com/microsoft/vscode-python/issues/7136 gulp.task('installNewPtvsd', async () => { // Install dependencies needed for 'install_ptvsd.py' - const depsArgs = ['-m', 'pip', '--disable-pip-version-check', 'install', '-t', './pythonFiles/lib/temp', '-r', './build/debugger-install-requirements.txt']; + const depsArgs = [ + '-m', + 'pip', + '--disable-pip-version-check', + 'install', + '-t', + './pythonFiles/lib/temp', + '-r', + './build/debugger-install-requirements.txt' + ]; const successWithWheelsDeps = await spawnAsync(process.env.CI_PYTHON_PATH || 'python3', depsArgs, undefined, true) .then(() => true) .catch(ex => { @@ -290,8 +361,12 @@ gulp.task('installNewPtvsd', async () => { return false; }); if (!successWithWheelsDeps) { - console.info("Failed to install dependencies need by 'install_ptvsd.py' using 'python3', attempting to install using 'python'"); - await spawnAsync('python', depsArgs).catch(ex => console.error("Failed to install dependencies need by 'install_ptvsd.py' using 'python'", ex)); + console.info( + "Failed to install dependencies need by 'install_ptvsd.py' using 'python3', attempting to install using 'python'" + ); + await spawnAsync('python', depsArgs).catch(ex => + console.error("Failed to install dependencies need by 'install_ptvsd.py' using 'python'", ex) + ); } // Install new PTVSD with wheels for python 3.7 @@ -305,7 +380,9 @@ gulp.task('installNewPtvsd', async () => { }); if (!successWithWheels) { console.info("Failed to install new PTVSD wheels using 'python3', attempting to install using 'python'"); - await spawnAsync('python', wheelsArgs, wheelsEnv).catch(ex => console.error("Failed to install PTVSD 5.0 wheels using 'python'", ex)); + await spawnAsync('python', wheelsArgs, wheelsEnv).catch(ex => + console.error("Failed to install PTVSD 5.0 wheels using 'python'", ex) + ); } rmrf.sync('./pythonFiles/lib/temp'); @@ -333,8 +410,12 @@ gulp.task('installNewPtvsd', async () => { return false; }); if (!successWithoutWheels) { - console.info("Failed to install source only version of new PTVSD using 'python3', attempting to install using 'python'"); - await spawnAsync('python', args).catch(ex => console.error("Failed to install source only PTVSD 5.0 using 'python'", ex)); + console.info( + "Failed to install source only version of new PTVSD using 'python3', attempting to install using 'python'" + ); + await spawnAsync('python', args).catch(ex => + console.error("Failed to install source only PTVSD 5.0 using 'python'", ex) + ); } }); @@ -386,7 +467,9 @@ function uploadExtension(uploadBlobName) { } gulp.task('uploadDeveloperExtension', () => uploadExtension('ms-python-insiders.vsix')); -gulp.task('uploadReleaseExtension', () => uploadExtension(`ms-python-${process.env.TRAVIS_BRANCH || process.env.BUILD_SOURCEBRANCHNAME}.vsix`)); +gulp.task('uploadReleaseExtension', () => + uploadExtension(`ms-python-${process.env.TRAVIS_BRANCH || process.env.BUILD_SOURCEBRANCHNAME}.vsix`) +); function spawnAsync(command, args, env, rejectOnStdErr = false) { env = env || {}; @@ -473,10 +556,14 @@ async function checkDatascienceDependencies() { if (name.includes('loader') && (name.includes('?') || name.includes('!'))) { return; } - const matchedModules = modulesInPackageLock.filter(dependency => dependency === moduleName2 || dependency === moduleName1 || dependency === name); + const matchedModules = modulesInPackageLock.filter( + dependency => dependency === moduleName2 || dependency === moduleName1 || dependency === name + ); switch (matchedModules.length) { case 0: - throwAndLogError(`Dependency not found in package-lock.json, Dependency = '${name}, ${moduleName1}, ${moduleName2}'`); + throwAndLogError( + `Dependency not found in package-lock.json, Dependency = '${name}, ${moduleName1}, ${moduleName2}'` + ); break; case 1: break; @@ -523,7 +610,9 @@ async function checkDatascienceDependencies() { let nameWithoutNodeModules = name.substring('/node_modules'.length); // Special case expose-loader. if (nameWithoutNodeModules.startsWith('/expose-loader')) { - nameWithoutNodeModules = nameWithoutNodeModules.substring(nameWithoutNodeModules.indexOf('/node_modules') + '/node_modules'.length); + nameWithoutNodeModules = nameWithoutNodeModules.substring( + nameWithoutNodeModules.indexOf('/node_modules') + '/node_modules'.length + ); } let moduleName1 = nameWithoutNodeModules.split('/')[1]; @@ -555,7 +644,9 @@ async function checkDatascienceDependencies() { let nameWithoutNodeModules = name.substring('/node_modules'.length); // Special case expose-loader. if (nameWithoutNodeModules.startsWith('/expose-loader')) { - nameWithoutNodeModules = nameWithoutNodeModules.substring(nameWithoutNodeModules.indexOf('/node_modules') + '/node_modules'.length); + nameWithoutNodeModules = nameWithoutNodeModules.substring( + nameWithoutNodeModules.indexOf('/node_modules') + '/node_modules'.length + ); } let moduleName1 = nameWithoutNodeModules.split('/')[1]; @@ -573,7 +664,9 @@ async function checkDatascienceDependencies() { * @param {*} reasons */ function processReasons(reasons) { - reasons = (reasons || []).map(reason => reason.userRequest).filter(item => typeof item === 'string' && !item.startsWith('.')); + reasons = (reasons || []) + .map(reason => reason.userRequest) + .filter(item => typeof item === 'string' && !item.startsWith('.')); reasons.forEach(item => processOriginOrReason(item)); } @@ -582,10 +675,18 @@ async function checkDatascienceDependencies() { const errorMessages = []; if (newModules.size > 0) { - errorMessages.push(`Add the untracked dependencies '${Array.from(newModules.values()).join(', ')}' to ${existingModulesFileName}`); + errorMessages.push( + `Add the untracked dependencies '${Array.from(newModules.values()).join( + ', ' + )}' to ${existingModulesFileName}` + ); } if (existingModulesCopy.size > 0) { - errorMessages.push(`Remove the unused '${Array.from(existingModulesCopy.values()).join(', ')}' dependencies from ${existingModulesFileName}`); + errorMessages.push( + `Remove the unused '${Array.from(existingModulesCopy.values()).join( + ', ' + )}' dependencies from ${existingModulesFileName}` + ); } if (errorMessages.length > 0) { throwAndLogError(errorMessages.join('\n')); @@ -604,9 +705,16 @@ function hasNativeDependencies() { } const dependencies = JSON.parse(spawn.sync('npm', ['ls', '--json', '--prod']).stdout.toString()); const jsonProperties = Object.keys(flat.flatten(dependencies)); - nativeDependencies = _.flatMap(nativeDependencies, item => path.dirname(item.substring(item.indexOf('node_modules') + 'node_modules'.length)).split(path.sep)) + nativeDependencies = _.flatMap(nativeDependencies, item => + path.dirname(item.substring(item.indexOf('node_modules') + 'node_modules'.length)).split(path.sep) + ) .filter(item => item.length > 0) - .filter(item => jsonProperties.findIndex(flattenedDependency => flattenedDependency.endsWith(`dependencies.${item}.version`)) >= 0); + .filter( + item => + jsonProperties.findIndex(flattenedDependency => + flattenedDependency.endsWith(`dependencies.${item}.version`) + ) >= 0 + ); if (nativeDependencies.length > 0) { console.error('Native dependencies detected', nativeDependencies); return true; @@ -655,7 +763,11 @@ const hygiene = (options, done) => { return done(); } const fileListToProcess = options.mode === 'compile' ? undefined : getFileListToProcess(options); - if (Array.isArray(fileListToProcess) && fileListToProcess !== all && fileListToProcess.filter(item => item.endsWith('.ts')).length === 0) { + if ( + Array.isArray(fileListToProcess) && + fileListToProcess !== all && + fileListToProcess.filter(item => item.endsWith('.ts')).length === 0 + ) { return done(); } @@ -674,7 +786,12 @@ const hygiene = (options, done) => { } else if (/^(\s\s\s\s)+.*/.test(line)) { // Good indent. } else if (/^[\t]+.*/.test(line)) { - console.error(file.relative + '(' + (i + 1) + ',1): Bad whitespace indentation (use 4 spaces instead of tabs or other)'); + console.error( + file.relative + + '(' + + (i + 1) + + ',1): Bad whitespace indentation (use 4 spaces instead of tabs or other)' + ); errorCount++; } }); @@ -724,10 +841,13 @@ const hygiene = (options, done) => { const name = failure.name || failure.fileName; const position = failure.startPosition; const line = position.lineAndCharacter ? position.lineAndCharacter.line : position.line; - const character = position.lineAndCharacter ? position.lineAndCharacter.character : position.character; + const character = position.lineAndCharacter + ? position.lineAndCharacter.character + : position.character; // Output in format similar to tslint for the linter to pickup. - const message = `ERROR: (${failure.ruleName}) ${relative(__dirname, name)}[${line + 1}, ${character + 1}]: ${failure.failure}`; + const message = `ERROR: (${failure.ruleName}) ${relative(__dirname, name)}[${line + + 1}, ${character + 1}]: ${failure.failure}`; if (reportedLinterFailures.indexOf(message) === -1) { console.error(message); reportedLinterFailures.push(message); @@ -844,11 +964,16 @@ const hygiene = (options, done) => { .pipe( es.through(null, function() { if (errorCount > 0) { - const errorMessage = `Hygiene failed with errors 👎 . Check 'gulpfile.js' (completed in ${new Date().getTime() - started}ms).`; + const errorMessage = `Hygiene failed with errors 👎 . Check 'gulpfile.js' (completed in ${new Date().getTime() - + started}ms).`; console.error(colors.red(errorMessage)); exitHandler(options); } else { - console.log(colors.green(`Hygiene passed with 0 errors 👍 (completed in ${new Date().getTime() - started}ms).`)); + console.log( + colors.green( + `Hygiene passed with 0 errors 👍 (completed in ${new Date().getTime() - started}ms).` + ) + ); } // Reset error counter. errorCount = 0; @@ -963,12 +1088,19 @@ function getModifiedFilesSync() { } const repo = process.env.TRAVIS_REPO_SLUG || getAzureDevOpsVarValue('Build.Repository.Name'); - const originOrUpstream = repo.toUpperCase() === 'MICROSOFT/VSCODE-PYTHON' || repo.toUpperCase() === 'VSCODE-PYTHON-DATASCIENCE/VSCODE-PYTHON' ? 'origin' : 'upstream'; + const originOrUpstream = + repo.toUpperCase() === 'MICROSOFT/VSCODE-PYTHON' || + repo.toUpperCase() === 'VSCODE-PYTHON-DATASCIENCE/VSCODE-PYTHON' + ? 'origin' + : 'upstream'; // If on CI, get a list of modified files comparing against // PR branch and master of current (assumed 'origin') repo. try { - cp.execSync(`git remote set-branches --add ${originOrUpstream} master`, { encoding: 'utf8', cwd: __dirname }); + cp.execSync(`git remote set-branches --add ${originOrUpstream} master`, { + encoding: 'utf8', + cwd: __dirname + }); cp.execSync('git fetch', { encoding: 'utf8', cwd: __dirname }); } catch (ex) { return []; diff --git a/news/1 Enhancements/9255.md b/news/1 Enhancements/9255.md new file mode 100644 index 000000000000..be69ac64944d --- /dev/null +++ b/news/1 Enhancements/9255.md @@ -0,0 +1 @@ +Implement VS code's custom editor for opening notebooks. \ No newline at end of file diff --git a/news/1 Enhancements/9821.md b/news/1 Enhancements/9821.md new file mode 100644 index 000000000000..0e7967ac65ba --- /dev/null +++ b/news/1 Enhancements/9821.md @@ -0,0 +1 @@ +Add undo/redo support to notebooks. \ No newline at end of file diff --git a/news/2 Fixes/10250.md b/news/2 Fixes/10250.md new file mode 100644 index 000000000000..e3ed4fd36c48 --- /dev/null +++ b/news/2 Fixes/10250.md @@ -0,0 +1 @@ +Ensure to correctly return env variables of the activated interpreter, when dealing with non-workspace interpreters. diff --git a/package.datascience-ui.dependencies.json b/package.datascience-ui.dependencies.json index e39e035098b1..7eba4d7fb8cf 100644 --- a/package.datascience-ui.dependencies.json +++ b/package.datascience-ui.dependencies.json @@ -1,247 +1,247 @@ [ - "@babel/runtime", - "@babel/runtime-corejs2", - "@emotion/hash", - "@emotion/memoize", - "@emotion/stylis", - "@emotion/unitless", - "@loadable/component", - "@mapbox/polylabel", - "@nteract/markdown", - "@nteract/mathjax", - "@nteract/octicons", - "@nteract/transform-dataresource", - "@nteract/transform-geojson", - "@nteract/transform-model-debug", - "@nteract/transform-plotly", - "@nteract/transform-vdom", - "@nteract/transform-vega", - "@nteract/transforms", - "@blueprintjs/core", - "@blueprintjs/icons", - "@blueprintjs/select", - "@icons/material", - "@nteract/styled-blueprintjsx", - "@nteract/vega-embed-v2", - "@nteract/vega-embed-v3", - "anser", - "ansi-regex", - "ansi-to-html", - "ansi-to-react", - "babel-polyfill", - "babel-runtime", - "bail", - "base16", - "bintrees", - "bootstrap-less", - "base64-js", - "create-react-context", - "character-entities-legacy", - "character-reference-invalid", - "classnames", - "clsx", - "collapse-white-space", - "create-emotion", - "css-loader", - "canvas", - "dom4", - "d3-array", - "d3-bboxCollide", - "d3-brush", - "d3-chord", - "d3-collection", - "d3-color", - "d3-contour", - "d3-dispatch", - "d3-drag", - "d3-ease", - "d3-force", - "d3-format", - "d3-glyphedge", - "d3-hexbin", - "d3-hierarchy", - "d3-interpolate", - "d3-path", - "d3-polygon", - "d3-quadtree", - "d3-sankey-circular", - "d3-scale", - "d3-selection", - "d3-shape", - "d3-time-format", - "d3-time", - "d3-timer", - "d3-transition", - "d3-voronoi", - "dom-helpers", - "d3-cloud", - "d3-delaunay", - "d3-dsv", - "d3-geo", - "d3-request", - "d3-scale-chromatic", - "d3", - "datalib", + "@babel/runtime", + "@babel/runtime-corejs2", + "@emotion/hash", + "@emotion/memoize", + "@emotion/stylis", + "@emotion/unitless", + "@loadable/component", + "@mapbox/polylabel", + "@nteract/markdown", + "@nteract/mathjax", + "@nteract/octicons", + "@nteract/transform-dataresource", + "@nteract/transform-geojson", + "@nteract/transform-model-debug", + "@nteract/transform-plotly", + "@nteract/transform-vdom", + "@nteract/transform-vega", + "@nteract/transforms", + "@blueprintjs/core", + "@blueprintjs/icons", + "@blueprintjs/select", + "@icons/material", + "@nteract/styled-blueprintjsx", + "@nteract/vega-embed-v2", + "@nteract/vega-embed-v3", + "anser", + "ansi-regex", + "ansi-to-html", + "ansi-to-react", + "babel-polyfill", + "babel-runtime", + "bail", + "base16", + "bintrees", + "bootstrap-less", + "base64-js", + "create-react-context", + "character-entities-legacy", + "character-reference-invalid", + "classnames", + "clsx", + "collapse-white-space", + "create-emotion", + "css-loader", + "canvas", + "dom4", + "d3-array", + "d3-bboxCollide", + "d3-brush", + "d3-chord", + "d3-collection", + "d3-color", + "d3-contour", + "d3-dispatch", + "d3-drag", + "d3-ease", + "d3-force", + "d3-format", + "d3-glyphedge", + "d3-hexbin", + "d3-hierarchy", + "d3-interpolate", + "d3-path", + "d3-polygon", + "d3-quadtree", + "d3-sankey-circular", + "d3-scale", + "d3-selection", + "d3-shape", + "d3-time-format", + "d3-time", + "d3-timer", + "d3-transition", + "d3-voronoi", + "dom-helpers", + "d3-cloud", + "d3-delaunay", + "d3-dsv", + "d3-geo", + "d3-request", + "d3-scale-chromatic", + "d3", + "datalib", "delaunator", "define-properties", - "emotion", - "entities", - "escape-carriage", - "extend", - "fast-deep-equal", - "fast-plist", + "emotion", + "entities", + "escape-carriage", + "extend", + "fast-deep-equal", + "fast-plist", "fast-json-stable-stringify", "function-bind", "gud", "has", "hoist-non-react-statics", - "inherits", + "inherits", "invariant", - "is-alphabetical", - "is-alphanumerical", + "is-alphabetical", + "is-alphanumerical", "is-arguments", "is-buffer", "is-date-object", - "is-decimal", - "is-hexadecimal", + "is-decimal", + "is-hexadecimal", "is-plain-obj", "is-regex", - "is-whitespace-character", - "is-word-character", - "ieee754", - "isarray", - "json2csv", - "json-stable-stringify", - "json-stringify-pretty-compact", - "jsonify", - "labella", - "leaflet", - "linear-layout-vector", - "lodash.curry", - "lodash.flow", - "lodash", - "lru-cache", - "martinez-polygon-clipping", - "markdown-escapes", - "material-colors", - "mdast-add-list-metadata", - "monaco-editor", - "monaco-editor-textmate", - "monaco-textmate", - "numeral", - "node-libs-browser", + "is-whitespace-character", + "is-word-character", + "ieee754", + "isarray", + "json2csv", + "json-stable-stringify", + "json-stringify-pretty-compact", + "jsonify", + "labella", + "leaflet", + "linear-layout-vector", + "lodash.curry", + "lodash.flow", + "lodash", + "lru-cache", + "martinez-polygon-clipping", + "markdown-escapes", + "material-colors", + "mdast-add-list-metadata", + "monaco-editor", + "monaco-editor-textmate", + "monaco-textmate", + "numeral", + "node-libs-browser", "object-assign", "object-is", "object-keys", - "onigasm", - "popper.js", - "parse-entities", - "path-browserify", + "onigasm", + "popper.js", + "parse-entities", + "path-browserify", "polygon-offset", "plotly.js-dist", - "process", - "prop-types", - "pseudomap", - "pure-color", - "react-is", - "react-popper", - "react-transition-group", - "regexp.prototype.flags", - "regression", - "replace-ext", - "resize-observer-polyfill", - "react-annotation", - "react-base16-styling", - "react-color", - "react-data-grid", - "react-dom", - "react-hot-loader", - "react-json-tree", - "react-lifecycles-compat", + "process", + "prop-types", + "pseudomap", + "pure-color", + "react-is", + "react-popper", + "react-transition-group", + "regexp.prototype.flags", + "regression", + "replace-ext", + "resize-observer-polyfill", + "react-annotation", + "react-base16-styling", + "react-color", + "react-data-grid", + "react-dom", + "react-hot-loader", + "react-json-tree", + "react-lifecycles-compat", "react-markdown", "react-redux", - "react-svg-pan-zoom", - "react-svgmt", - "react-table-hoc-fixed-columns", - "react-table", - "react-virtualized", + "react-svg-pan-zoom", + "react-svgmt", + "react-table-hoc-fixed-columns", + "react-table", + "react-virtualized", "react", "reactcss", "redux-logger", "redux", - "remark-parse", - "repeat-string", - "roughjs-es5", - "scheduler", - "semiotic-mark", - "semiotic", - "setimmediate", - "slickgrid", - "state-toggle", - "string-hash", - "style-loader", - "styled-jsx", - "stylis-rule-sheet", - "svg-inline-react", - "svg-path-bounding-box", - "svgpath", - "semver", - "timers-browserify", - "tinycolor2", - "tinyqueue", - "transformation-matrix", - "trim-trailing-lines", - "trim", - "trough", - "topojson-client", - "tslib", - "unherit", - "unified", - "uniqid", - "unist-util-is", - "unist-util-remove-position", - "unist-util-stringify-position", - "unist-util-visit-parents", - "unist-util-visit", - "util", - "uuid", - "vfile-location", - "vfile-message", - "vfile", - "viz-annotation", - "vega-canvas", - "vega-crossfilter", - "vega-dataflow", - "vega-embed", - "vega-encode", - "vega-event-selector", - "vega-expression", - "vega-force", - "vega-functions", - "vega-geo", - "vega-hierarchy", - "vega-lite", - "vega-loader", - "vega-parser", - "vega-projection", - "vega-regression", - "vega-runtime", - "vega-scale", - "vega-scenegraph", - "vega-schema-url-parser", - "vega-selections", - "vega-statistics", - "vega-themes", - "vega-tooltip", - "vega-transforms", - "vega-util", - "vega-view-transforms", - "vega-view", - "vega-voronoi", - "vega-wordcloud", - "vega", - "warning", - "x-is-string", - "xtend", - "yallist" + "remark-parse", + "repeat-string", + "roughjs-es5", + "scheduler", + "semiotic-mark", + "semiotic", + "setimmediate", + "slickgrid", + "state-toggle", + "string-hash", + "style-loader", + "styled-jsx", + "stylis-rule-sheet", + "svg-inline-react", + "svg-path-bounding-box", + "svgpath", + "semver", + "timers-browserify", + "tinycolor2", + "tinyqueue", + "transformation-matrix", + "trim-trailing-lines", + "trim", + "trough", + "topojson-client", + "tslib", + "unherit", + "unified", + "uniqid", + "unist-util-is", + "unist-util-remove-position", + "unist-util-stringify-position", + "unist-util-visit-parents", + "unist-util-visit", + "util", + "uuid", + "vfile-location", + "vfile-message", + "vfile", + "viz-annotation", + "vega-canvas", + "vega-crossfilter", + "vega-dataflow", + "vega-embed", + "vega-encode", + "vega-event-selector", + "vega-expression", + "vega-force", + "vega-functions", + "vega-geo", + "vega-hierarchy", + "vega-lite", + "vega-loader", + "vega-parser", + "vega-projection", + "vega-regression", + "vega-runtime", + "vega-scale", + "vega-scenegraph", + "vega-schema-url-parser", + "vega-selections", + "vega-statistics", + "vega-themes", + "vega-tooltip", + "vega-transforms", + "vega-util", + "vega-view-transforms", + "vega-view", + "vega-voronoi", + "vega-wordcloud", + "vega", + "warning", + "x-is-string", + "xtend", + "yallist" ] diff --git a/package.json b/package.json index ee5942e7c1a2..045435fa0d04 100644 --- a/package.json +++ b/package.json @@ -5,6 +5,7 @@ "version": "2020.3.0-dev", "languageServerVersion": "0.5.30", "publisher": "ms-python", + "enableProposedApi": false, "author": { "name": "Microsoft Corporation" }, @@ -82,6 +83,7 @@ "onCommand:python.datascience.exportfileasnotebook", "onCommand:python.datascience.exportfileandoutputasnotebook", "onCommand:python.datascience.selectJupyterInterpreter", + "onWebviewEditor:NativeEditorProvider.ipynb", "onCommand:python.datascience.selectjupytercommandline", "onCommand:python.enableSourceMapSupport" ], @@ -2832,7 +2834,9 @@ "when": "testsDiscovered" } ] - } + }, + "webviewEditors": [ + ] }, "scripts": { "package": "gulp clean && gulp prePublishBundle && vsce package -o ms-python-insiders.vsix", diff --git a/package.nls.tr.json b/package.nls.tr.json index 6164d873ed3e..0e648bb38fdf 100644 --- a/package.nls.tr.json +++ b/package.nls.tr.json @@ -30,4 +30,4 @@ "python.snippet.launch.django.label": "Python: Django", "python.snippet.launch.flask.label": "Python: Flask", "python.snippet.launch.pyramid.label": "Python: Pyramid Uygulaması" - } +} diff --git a/settings.json b/settings.json new file mode 100644 index 000000000000..64e62ac725c8 --- /dev/null +++ b/settings.json @@ -0,0 +1,59 @@ +// Place your settings in this file to overwrite default and user settings. +{ + "files.exclude": { + "out": true, // set this to true to hide the "out" folder with the compiled JS files + "**/*.pyc": true, + ".nyc_output": true, + "obj": true, + "bin": true, + "**/__pycache__": true, + "**/node_modules": true, + ".vscode-test": false, + ".vscode test": false, + "**/.mypy_cache/**": true, + "**/.ropeproject/**": true + }, + "search.exclude": { + "out": true, // set this to false to include "out" folder in search results + "**/node_modules": true, + "coverage": true, + "languageServer*/**": true, + ".vscode-test": true, + ".vscode test": true + }, + "[python]": { + "editor.formatOnSave": true + }, + "[typescript]": { + "editor.formatOnSave": true + }, + "[javascript]": { + "editor.formatOnSave": true + }, + "typescript.tsdk": "./node_modules/typescript/lib", // we want to use the TS server from our node_modules folder to control its version + "tslint.enable": true, + "python.linting.enabled": false, + "python.testing.promptToConfigure": false, + "python.workspaceSymbols.enabled": false, + "python.formatting.provider": "black", + "typescript.preferences.quoteStyle": "single", + "javascript.preferences.quoteStyle": "single", + "typescriptHero.imports.stringQuoteStyle": "'", + "prettier.printWidth": 120, + "prettier.singleQuote": true, + "editor.codeActionsOnSave": { + "source.fixAll.eslint": true, + "source.fixAll.tslint": true + }, + "python.jediEnabled": false, + "python.analysis.logLevel": "Trace", + "python.analysis.downloadChannel": "beta", + "python.linting.pylintEnabled": false, + "python.linting.flake8Enabled": true, + "cucumberautocomplete.skipDocStringsFormat": true, + "python.linting.flake8Args": [ + // Match what black does. + "--max-line-length=88" + ], + "typescript.preferences.importModuleSpecifier": "relative" +} diff --git a/src/client/common/application/commands.ts b/src/client/common/application/commands.ts index 0a23e01c6ef0..041fd2c9946e 100644 --- a/src/client/common/application/commands.ts +++ b/src/client/common/application/commands.ts @@ -88,6 +88,9 @@ export interface ICommandNameArgumentTypeMapping extends ICommandNameWithoutArgu ['python.SelectAndInsertDebugConfiguration']: [TextDocument, Position, CancellationToken]; ['vscode.open']: [Uri]; ['python.viewLanguageServerOutput']: []; + ['vscode.open']: [Uri]; + ['workbench.action.files.saveAs']: [Uri]; + ['workbench.action.files.save']: [Uri]; [Commands.Build_Workspace_Symbols]: [boolean, CancellationToken]; [Commands.Sort_Imports]: [undefined, Uri]; [Commands.Exec_In_Terminal]: [undefined, Uri]; @@ -154,4 +157,6 @@ export interface ICommandNameArgumentTypeMapping extends ICommandNameWithoutArgu [DSCommands.ViewJupyterOutput]: []; [DSCommands.SwitchJupyterKernel]: [INotebook | undefined]; [DSCommands.SelectJupyterCommandLine]: [undefined | Uri]; + [DSCommands.SaveNotebookNonCustomEditor]: [Uri]; + [DSCommands.SaveAsNotebookNonCustomEditor]: [Uri, Uri]; } diff --git a/src/client/common/application/customEditorService.ts b/src/client/common/application/customEditorService.ts new file mode 100644 index 000000000000..11fa0f06fb5c --- /dev/null +++ b/src/client/common/application/customEditorService.ts @@ -0,0 +1,35 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. +'use strict'; +import { inject, injectable } from 'inversify'; +import * as vscode from 'vscode'; + +import { noop } from '../utils/misc'; +import { IApplicationEnvironment, ICommandManager, ICustomEditorService, WebviewCustomEditorProvider } from './types'; + +@injectable() +export class CustomEditorService implements ICustomEditorService { + constructor( + @inject(ICommandManager) private commandManager: ICommandManager, + @inject(IApplicationEnvironment) private readonly appEnv: IApplicationEnvironment + ) {} + + public registerWebviewCustomEditorProvider( + viewType: string, + provider: WebviewCustomEditorProvider, + options?: vscode.WebviewPanelOptions + ): vscode.Disposable { + if (this.appEnv.packageJson.enableProposedApi) { + // tslint:disable-next-line: no-any + return (vscode.window as any).registerWebviewCustomEditorProvider(viewType, provider, options); + } else { + return { dispose: noop }; + } + } + + public async openEditor(file: vscode.Uri): Promise { + if (this.appEnv.packageJson.enableProposedApi) { + await this.commandManager.executeCommand('vscode.open', file); + } + } +} diff --git a/src/client/common/application/types.ts b/src/client/common/application/types.ts index 5cd24940b879..68f39500ccc1 100644 --- a/src/client/common/application/types.ts +++ b/src/client/common/application/types.ts @@ -49,6 +49,8 @@ import { TreeViewOptions, Uri, ViewColumn, + WebviewPanel, + WebviewPanelOptions, WindowState, WorkspaceConfiguration, WorkspaceEdit, @@ -1027,6 +1029,8 @@ export interface IWebPanelOptions { cwd: string; // tslint:disable-next-line: no-any settings?: any; + // Web panel to use if supplied by VS code instead + webViewPanel?: WebviewPanel; } // Wraps the VS Code api for creating a web panel @@ -1090,3 +1094,106 @@ export const IActiveResourceService = Symbol('IActiveResourceService'); export interface IActiveResourceService { getActiveResource(): Resource; } + +// Temporary hack to get the nyc compiler to find these types. vscode.proposed.d.ts doesn't work for some reason. +/** + * Defines the editing functionality of a webview editor. This allows the webview editor to hook into standard + * editor events such as `undo` or `save`. + * + * @param EditType Type of edits. Edit objects must be json serializable. + */ +// tslint:disable-next-line: interface-name +export interface WebviewCustomEditorEditingDelegate { + /** + * Event triggered by extensions to signal to VS Code that an edit has occurred. + */ + readonly onEdit: Event<{ readonly resource: Uri; readonly edit: EditType }>; + /** + * Save a resource. + * + * @param resource Resource being saved. + * + * @return Thenable signaling that the save has completed. + */ + save(resource: Uri): Thenable; + + /** + * Save an existing resource at a new path. + * + * @param resource Resource being saved. + * @param targetResource Location to save to. + * + * @return Thenable signaling that the save has completed. + */ + saveAs(resource: Uri, targetResource: Uri): Thenable; + + /** + * Apply a set of edits. + * + * Note that is not invoked when `onEdit` is called as `onEdit` implies also updating the view to reflect the edit. + * + * @param resource Resource being edited. + * @param edit Array of edits. Sorted from oldest to most recent. + * + * @return Thenable signaling that the change has completed. + */ + applyEdits(resource: Uri, edits: readonly EditType[]): Thenable; + + /** + * Undo a set of edits. + * + * This is triggered when a user undoes an edit or when revert is called on a file. + * + * @param resource Resource being edited. + * @param edit Array of edits. Sorted from most recent to oldest. + * + * @return Thenable signaling that the change has completed. + */ + undoEdits(resource: Uri, edits: readonly EditType[]): Thenable; +} + +// tslint:disable-next-line: interface-name +export interface WebviewCustomEditorProvider { + /** + * Controls the editing functionality of a webview editor. This allows the webview editor to hook into standard + * editor events such as `undo` or `save`. + * + * WebviewEditors that do not have `editingCapability` are considered to be readonly. Users can still interact + * with readonly editors, but these editors will not integrate with VS Code's standard editor functionality. + */ + readonly editingDelegate?: WebviewCustomEditorEditingDelegate; + /** + * Resolve a webview editor for a given resource. + * + * To resolve a webview editor, a provider must fill in its initial html content and hook up all + * the event listeners it is interested it. The provider should also take ownership of the passed in `WebviewPanel`. + * + * @param resource Resource being resolved. + * @param webview Webview being resolved. The provider should take ownership of this webview. + * + * @return Thenable indicating that the webview editor has been resolved. + */ + resolveWebviewEditor(resource: Uri, webview: WebviewPanel): Thenable; +} + +export const ICustomEditorService = Symbol('ICustomEditorService'); +export interface ICustomEditorService { + /** + * Register a new provider for webview editors of a given type. + * + * @param viewType Type of the webview editor provider. + * @param provider Resolves webview editors. + * @param options Content settings for a webview panels the provider is given. + * + * @return Disposable that unregisters the `WebviewCustomEditorProvider`. + */ + registerWebviewCustomEditorProvider( + viewType: string, + provider: WebviewCustomEditorProvider, + options?: WebviewPanelOptions + ): Disposable; + /** + * Opens a file with a custom editor + */ + openEditor(file: Uri): Promise; +} diff --git a/src/client/common/application/webPanels/webPanel.ts b/src/client/common/application/webPanels/webPanel.ts index c0ea7cdcfd97..f64494d71140 100644 --- a/src/client/common/application/webPanels/webPanel.ts +++ b/src/client/common/application/webPanels/webPanel.ts @@ -4,7 +4,7 @@ import '../../extensions'; import * as uuid from 'uuid/v4'; -import { Uri, Webview, WebviewPanel, window } from 'vscode'; +import { Uri, Webview, WebviewOptions, WebviewPanel, window } from 'vscode'; import { Identifiers } from '../../../datascience/constants'; import { InteractiveWindowMessages } from '../../../datascience/interactive-common/interactiveWindowTypes'; @@ -31,18 +31,26 @@ export class WebPanel implements IWebPanel { private token: string | undefined, private options: IWebPanelOptions ) { - this.panel = window.createWebviewPanel( - options.title.toLowerCase().replace(' ', ''), - options.title, - { viewColumn: options.viewColumn, preserveFocus: true }, - { - enableScripts: true, - retainContextWhenHidden: true, - localResourceRoots: [Uri.file(this.options.rootPath), Uri.file(this.options.cwd)], - enableFindWidget: true, - portMapping: port ? [{ webviewPort: RemappedPort, extensionHostPort: port }] : undefined - } - ); + const webViewOptions: WebviewOptions = { + enableScripts: true, + localResourceRoots: [Uri.file(this.options.rootPath), Uri.file(this.options.cwd)], + portMapping: port ? [{ webviewPort: RemappedPort, extensionHostPort: port }] : undefined + }; + if (options.webViewPanel) { + this.panel = options.webViewPanel; + this.panel.webview.options = webViewOptions; + } else { + this.panel = window.createWebviewPanel( + options.title.toLowerCase().replace(' ', ''), + options.title, + { viewColumn: options.viewColumn, preserveFocus: true }, + { + retainContextWhenHidden: true, + enableFindWidget: true, + ...webViewOptions + } + ); + } this.loadPromise = this.load(); } diff --git a/src/client/common/serviceRegistry.ts b/src/client/common/serviceRegistry.ts index 70b36085df98..f0b346f89f56 100644 --- a/src/client/common/serviceRegistry.ts +++ b/src/client/common/serviceRegistry.ts @@ -9,6 +9,7 @@ import { IImportTracker } from '../telemetry/types'; import { ApplicationEnvironment } from './application/applicationEnvironment'; import { ApplicationShell } from './application/applicationShell'; import { CommandManager } from './application/commandManager'; +import { CustomEditorService } from './application/customEditorService'; import { DebugService } from './application/debugService'; import { DebugSessionTelemetry } from './application/debugSessionTelemetry'; import { DocumentManager } from './application/documentManager'; @@ -19,6 +20,7 @@ import { IApplicationEnvironment, IApplicationShell, ICommandManager, + ICustomEditorService, IDebugService, IDocumentManager, ILanguageService, @@ -194,4 +196,5 @@ export function registerTypes(serviceManager: IServiceManager) { IExtensionSingleActivationService, DebugSessionTelemetry ); + serviceManager.addSingleton(ICustomEditorService, CustomEditorService); } diff --git a/src/client/datascience/constants.ts b/src/client/datascience/constants.ts index caad0861e33a..d1457f677916 100644 --- a/src/client/datascience/constants.ts +++ b/src/client/datascience/constants.ts @@ -78,6 +78,8 @@ export namespace Commands { export const ScrollToCell = 'python.datascience.scrolltocell'; export const CreateNewNotebook = 'python.datascience.createnewnotebook'; export const ViewJupyterOutput = 'python.datascience.viewJupyterOutput'; + export const SaveNotebookNonCustomEditor = 'python.datascience.notebookeditor.save'; + export const SaveAsNotebookNonCustomEditor = 'python.datascience.notebookeditor.saveAs'; } export namespace CodeLensCommands { @@ -261,6 +263,7 @@ export enum Telemetry { UserInstalledJupyter = 'DATASCIENCE.USER_INSTALLED_JUPYTER', UserDidNotInstallJupyter = 'DATASCIENCE.USER_DID_NOT_INSTALL_JUPYTER', OpenedInteractiveWindow = 'DATASCIENCE.OPENED_INTERACTIVE', + OpenNotebookFailure = 'DS_INTERNAL.NATIVE.OPEN_NOTEBOOK_FAILURE', FindKernelForLocalConnection = 'DS_INTERNAL.FIND_KERNEL_FOR_LOCAL_CONNECTION', CompletionTimeFromLS = 'DS_INTERNAL.COMPLETION_TIME_FROM_LS', CompletionTimeFromJupyter = 'DS_INTERNAL.COMPLETION_TIME_FROM_JUPYTER', diff --git a/src/client/datascience/data-viewing/types.ts b/src/client/datascience/data-viewing/types.ts index ed1c240c108c..09751f4daa85 100644 --- a/src/client/datascience/data-viewing/types.ts +++ b/src/client/datascience/data-viewing/types.ts @@ -3,7 +3,7 @@ 'use strict'; import { JSONObject } from '@phosphor/coreutils'; -import { CssMessages, IGetCssRequest, IGetCssResponse, SharedMessages } from '../messages'; +import { SharedMessages } from '../messages'; import { IJupyterVariable } from '../types'; export const CellFetchAllLimit = 100000; @@ -40,15 +40,13 @@ export interface IGetRowsResponse { } // Map all messages to specific payloads -export class IDataViewerMapping { - public [DataViewerMessages.Started]: never | undefined; - public [DataViewerMessages.UpdateSettings]: string; - public [DataViewerMessages.InitializeData]: IJupyterVariable; - public [DataViewerMessages.GetAllRowsRequest]: never | undefined; - public [DataViewerMessages.GetAllRowsResponse]: JSONObject; - public [DataViewerMessages.GetRowsRequest]: IGetRowsRequest; - public [DataViewerMessages.GetRowsResponse]: IGetRowsResponse; - public [DataViewerMessages.CompletedData]: never | undefined; - public [CssMessages.GetCssRequest]: IGetCssRequest; - public [CssMessages.GetCssResponse]: IGetCssResponse; -} +export type IDataViewerMapping = { + [DataViewerMessages.Started]: never | undefined; + [DataViewerMessages.UpdateSettings]: string; + [DataViewerMessages.InitializeData]: IJupyterVariable; + [DataViewerMessages.GetAllRowsRequest]: never | undefined; + [DataViewerMessages.GetAllRowsResponse]: JSONObject; + [DataViewerMessages.GetRowsRequest]: IGetRowsRequest; + [DataViewerMessages.GetRowsResponse]: IGetRowsResponse; + [DataViewerMessages.CompletedData]: never | undefined; +}; diff --git a/src/client/datascience/interactive-common/intellisense/intellisenseDocument.ts b/src/client/datascience/interactive-common/intellisense/intellisenseDocument.ts index 0f11a95a4e8e..0fd218cbf419 100644 --- a/src/client/datascience/interactive-common/intellisense/intellisenseDocument.ts +++ b/src/client/datascience/interactive-common/intellisense/intellisenseDocument.ts @@ -3,12 +3,12 @@ 'use strict'; import '../../../common/extensions'; -import * as monacoEditor from 'monaco-editor/esm/vs/editor/editor.api'; import { EndOfLine, Position, Range, TextDocument, TextDocumentContentChangeEvent, TextLine, Uri } from 'vscode'; import * as vscodeLanguageClient from 'vscode-languageclient'; import { PYTHON_LANGUAGE } from '../../../common/constants'; import { Identifiers } from '../../constants'; +import { IEditorContentChange } from '../interactiveWindowTypes'; import { DefaultWordPattern, ensureValidWordDefinition, getWordAtText, regExpLeadsToEndlessLoop } from './wordHelper'; class IntellisenseLine implements TextLine { @@ -208,54 +208,57 @@ export class IntellisenseDocument implements TextDocument { } public loadAllCells(cells: { code: string; id: string }[]): TextDocumentContentChangeEvent[] { - let changes: TextDocumentContentChangeEvent[] = []; if (!this.inEditMode) { this.inEditMode = true; - this._version += 1; + return this.reloadCells(cells); + } + return []; + } - // Normalize all of the cells, removing \r and separating each - // with a newline - const normalized = cells.map(c => { - return { - id: c.id, - code: `${c.code.replace(/\r/g, '')}\n` - }; - }); + public reloadCells(cells: { code: string; id: string }[]): TextDocumentContentChangeEvent[] { + this._version += 1; - // Contents are easy, just load all of the code in a row - this._contents = normalized - .map(c => c.code) - .reduce((p, c) => { - return `${p}${c}`; - }); - - // Cell ranges are slightly more complicated - let prev: number = 0; - this._cellRanges = normalized.map(c => { - const result = { - id: c.id, - start: prev, - fullEnd: prev + c.code.length, - currentEnd: prev + c.code.length - }; - prev += c.code.length; - return result; + // Normalize all of the cells, removing \r and separating each + // with a newline + const normalized = cells.map(c => { + return { + id: c.id, + code: `${c.code.replace(/\r/g, '')}\n` + }; + }); + + // Contents are easy, just load all of the code in a row + this._contents = normalized + .map(c => c.code) + .reduce((p, c) => { + return `${p}${c}`; }); - // Then create the lines. - this._lines = this.createLines(); + // Cell ranges are slightly more complicated + let prev: number = 0; + this._cellRanges = normalized.map(c => { + const result = { + id: c.id, + start: prev, + fullEnd: prev + c.code.length, + currentEnd: prev + c.code.length + }; + prev += c.code.length; + return result; + }); - // Return our changes - changes = [ - { - range: this.createSerializableRange(new Position(0, 0), new Position(0, 0)), - rangeOffset: 0, - rangeLength: 0, // Adds are always zero - text: this._contents - } - ]; - } - return changes; + // Then create the lines. + this._lines = this.createLines(); + + // Return our changes + return [ + { + range: this.createSerializableRange(new Position(0, 0), new Position(0, 0)), + rangeOffset: 0, + rangeLength: 0, // Adds are always zero + text: this._contents + } + ]; } public addCell(fullCode: string, currentCode: string, id: string): TextDocumentContentChangeEvent[] { @@ -305,7 +308,28 @@ export class IntellisenseDocument implements TextDocument { ]; } - public insertCell(id: string, code: string, codeCellAbove: string | undefined): TextDocumentContentChangeEvent[] { + public reloadCell(id: string, code: string): TextDocumentContentChangeEvent[] { + this._version += 1; + + // Make sure to put a newline between this code and the next code + const newCode = `${code.replace(/\r/g, '')}\n`; + + // Figure where this goes + const index = this._cellRanges.findIndex(r => r.id === id); + if (index >= 0) { + const start = this.positionAt(this._cellRanges[index].start); + const end = this.positionAt(this._cellRanges[index].currentEnd); + return this.removeRange(newCode, start, end, index); + } + + return []; + } + + public insertCell( + id: string, + code: string, + codeCellAboveOrIndex: string | undefined | number + ): TextDocumentContentChangeEvent[] { // This should only happen once for each cell. this._version += 1; @@ -313,8 +337,8 @@ export class IntellisenseDocument implements TextDocument { const newCode = `${code.replace(/\r/g, '')}\n`; // Figure where this goes - const aboveIndex = this._cellRanges.findIndex(r => r.id === codeCellAbove); - const insertIndex = aboveIndex + 1; + const aboveIndex = this._cellRanges.findIndex(r => r.id === codeCellAboveOrIndex); + const insertIndex = typeof codeCellAboveOrIndex === 'number' ? codeCellAboveOrIndex : aboveIndex + 1; // Compute where we start from. const fromOffset = @@ -374,10 +398,7 @@ export class IntellisenseDocument implements TextDocument { return []; } - public edit( - editorChanges: monacoEditor.editor.IModelContentChange[], - id: string - ): TextDocumentContentChangeEvent[] { + public editCell(editorChanges: IEditorContentChange[], id: string): TextDocumentContentChangeEvent[] { this._version += 1; // Convert the range to local (and remove 1 based) diff --git a/src/client/datascience/interactive-common/intellisense/intellisenseProvider.ts b/src/client/datascience/interactive-common/intellisense/intellisenseProvider.ts index a1156d870361..9a4821ca2057 100644 --- a/src/client/datascience/interactive-common/intellisense/intellisenseProvider.ts +++ b/src/client/datascience/interactive-common/intellisense/intellisenseProvider.ts @@ -30,13 +30,16 @@ import { noop } from '../../../common/utils/misc'; import { HiddenFileFormatString } from '../../../constants'; import { IInterpreterService, PythonInterpreter } from '../../../interpreter/contracts'; import { sendTelemetryWhenDone } from '../../../telemetry'; -import { Identifiers, Settings, Telemetry } from '../../constants'; -import { IInteractiveWindowListener, IInteractiveWindowProvider, IJupyterExecution, INotebook } from '../../types'; +import { Settings, Telemetry } from '../../constants'; +import { + ICell, + IInteractiveWindowListener, + IInteractiveWindowProvider, + IJupyterExecution, + INotebook +} from '../../types'; import { - IAddCell, ICancelIntellisenseRequest, - IEditCell, - IInsertCell, IInteractiveWindowMapping, ILoadAllCells, INotebookIdentity, @@ -44,9 +47,8 @@ import { IProvideCompletionItemsRequest, IProvideHoverRequest, IProvideSignatureHelpRequest, - IRemoveCell, IResolveCompletionItemRequest, - ISwapCells + NotebookModelChange } from '../interactiveWindowTypes'; import { convertStringsToSuggestions, @@ -61,6 +63,9 @@ import { IntellisenseDocument } from './intellisenseDocument'; // tslint:disable:no-any @injectable() export class IntellisenseProvider implements IInteractiveWindowListener { + public get postMessage(): Event<{ message: string; payload: any }> { + return this.postEmitter.event; + } private documentPromise: Deferred | undefined; private temporaryFile: TemporaryFile | undefined; private postEmitter: EventEmitter<{ message: string; payload: any }> = new EventEmitter<{ @@ -94,10 +99,6 @@ export class IntellisenseProvider implements IInteractiveWindowListener { } } - public get postMessage(): Event<{ message: string; payload: any }> { - return this.postEmitter.event; - } - public onMessage(message: string, payload?: any) { switch (message) { case InteractiveWindowMessages.CancelCompletionItemsRequest: @@ -121,28 +122,8 @@ export class IntellisenseProvider implements IInteractiveWindowListener { this.dispatchMessage(message, payload, this.handleResolveCompletionItemRequest); break; - case InteractiveWindowMessages.EditCell: - this.dispatchMessage(message, payload, this.editCell); - break; - - case InteractiveWindowMessages.AddCell: - this.dispatchMessage(message, payload, this.addCell); - break; - - case InteractiveWindowMessages.InsertCell: - this.dispatchMessage(message, payload, this.insertCell); - break; - - case InteractiveWindowMessages.RemoveCell: - this.dispatchMessage(message, payload, this.removeCell); - break; - - case InteractiveWindowMessages.SwapCells: - this.dispatchMessage(message, payload, this.swapCells); - break; - - case InteractiveWindowMessages.DeleteAllCells: - this.dispatchMessage(message, payload, this.removeAllCells); + case InteractiveWindowMessages.UpdateModel: + this.dispatchMessage(message, payload, this.update); break; case InteractiveWindowMessages.RestartKernel: @@ -162,6 +143,32 @@ export class IntellisenseProvider implements IInteractiveWindowListener { } } + public getDocument(resource?: Uri): Promise { + if (!this.documentPromise) { + this.documentPromise = createDeferred(); + + // Create our dummy document. Compute a file path for it. + if (this.workspaceService.rootPath || resource) { + const dir = resource ? path.dirname(resource.fsPath) : this.workspaceService.rootPath!; + const dummyFilePath = path.join(dir, HiddenFileFormatString.format(uuid().replace(/-/g, ''))); + this.documentPromise.resolve(new IntellisenseDocument(dummyFilePath)); + } else { + this.fileSystem + .createTemporaryFile('.py') + .then(t => { + this.temporaryFile = t; + const dummyFilePath = this.temporaryFile.filePath; + this.documentPromise!.resolve(new IntellisenseDocument(dummyFilePath)); + }) + .catch(e => { + this.documentPromise!.reject(e); + }); + } + } + + return this.documentPromise.promise; + } + protected async getLanguageServer(): Promise { // Resource should be our potential resource if its set. Otherwise workspace root const resource = @@ -213,32 +220,6 @@ export class IntellisenseProvider implements IInteractiveWindowListener { return this.languageServer; } - protected getDocument(resource?: Uri): Promise { - if (!this.documentPromise) { - this.documentPromise = createDeferred(); - - // Create our dummy document. Compute a file path for it. - if (this.workspaceService.rootPath || resource) { - const dir = resource ? path.dirname(resource.fsPath) : this.workspaceService.rootPath!; - const dummyFilePath = path.join(dir, HiddenFileFormatString.format(uuid().replace(/-/g, ''))); - this.documentPromise.resolve(new IntellisenseDocument(dummyFilePath)); - } else { - this.fileSystem - .createTemporaryFile('.py') - .then(t => { - this.temporaryFile = t; - const dummyFilePath = this.temporaryFile.filePath; - this.documentPromise!.resolve(new IntellisenseDocument(dummyFilePath)); - }) - .catch(e => { - this.documentPromise!.reject(e); - }); - } - } - - return this.documentPromise.promise; - } - protected async provideCompletionItems( position: monacoEditor.Position, context: monacoEditor.languages.CompletionContext, @@ -584,65 +565,107 @@ export class IntellisenseProvider implements IInteractiveWindowListener { ); } - private async addCell(request: IAddCell): Promise { - // Save this request file as our potential resource - if (request.cell.file !== Identifiers.EmptyFileName) { - this.potentialResource = Uri.file(request.cell.file); - } - - // Get the document and then pass onto the sub class - const document = await this.getDocument( - request.cell.file === Identifiers.EmptyFileName ? undefined : Uri.file(request.cell.file) - ); - if (document) { - const changes = document.addCell(request.fullText, request.currentText, request.cell.id); - return this.handleChanges(document, changes); + private async update(request: NotebookModelChange): Promise { + // See where this request is coming from + switch (request.source) { + case 'redo': + case 'user': + return this.handleRedo(request); + case 'undo': + return this.handleUndo(request); + default: + break; } } - private async insertCell(request: IInsertCell): Promise { - // Get the document and then pass onto the sub class - const document = await this.getDocument(); - if (document) { - const changes = document.insertCell(request.cell.id, request.code, request.codeCellAboveId); - return this.handleChanges(document, changes); - } + private convertToDocCells(cells: ICell[]): { code: string; id: string }[] { + return cells + .filter(c => c.data.cell_type === 'code') + .map(c => { + return { code: concatMultilineStringInput(c.data.source), id: c.id }; + }); } - private async editCell(request: IEditCell): Promise { - // First get the document + private async handleUndo(request: NotebookModelChange): Promise { const document = await this.getDocument(); - if (document) { - const changes = document.edit(request.changes, request.id); - return this.handleChanges(document, changes); + let changes: TextDocumentContentChangeEvent[] = []; + switch (request.kind) { + case 'clear': + // This one can be ignored, it only clears outputs + break; + case 'edit': + changes = document.editCell(request.reverse, request.id); + break; + case 'add': + case 'insert': + changes = document.remove(request.cell.id); + break; + case 'modify': + // This one can be ignored. it's only used for updating cell finished state. + break; + case 'remove': + changes = document.insertCell( + request.cell.id, + concatMultilineStringInput(request.cell.data.source), + request.index + ); + break; + case 'remove_all': + changes = document.reloadCells(this.convertToDocCells(request.oldCells)); + break; + case 'swap': + changes = document.swap(request.secondCellId, request.firstCellId); + break; + case 'version': + // Also ignored. updates version which we don't keep track of. + break; + default: + break; } - } - private async removeCell(request: IRemoveCell): Promise { - // First get the document - const document = await this.getDocument(); - if (document) { - const changes = document.remove(request.id); - return this.handleChanges(document, changes); - } + return this.handleChanges(document, changes); } - private async swapCells(request: ISwapCells): Promise { - // First get the document + private async handleRedo(request: NotebookModelChange): Promise { const document = await this.getDocument(); - if (document) { - const changes = document.swap(request.firstCellId, request.secondCellId); - return this.handleChanges(document, changes); + let changes: TextDocumentContentChangeEvent[] = []; + switch (request.kind) { + case 'clear': + // This one can be ignored, it only clears outputs + break; + case 'edit': + changes = document.editCell(request.forward, request.id); + break; + case 'add': + changes = document.addCell(request.fullText, request.currentText, request.cell.id); + break; + case 'insert': + changes = document.insertCell( + request.cell.id, + concatMultilineStringInput(request.cell.data.source), + request.codeCellAboveId || request.index + ); + break; + case 'modify': + // This one can be ignored. it's only used for updating cell finished state. + break; + case 'remove': + changes = document.remove(request.cell.id); + break; + case 'remove_all': + changes = document.removeAll(); + break; + case 'swap': + changes = document.swap(request.firstCellId, request.secondCellId); + break; + case 'version': + // Also ignored. updates version which we don't keep track of. + break; + default: + break; } - } - private async removeAllCells(): Promise { - // First get the document - const document = await this.getDocument(); - if (document) { - const changes = document.removeAll(); - return this.handleChanges(document, changes); - } + return this.handleChanges(document, changes); } private async loadAllCells(payload: ILoadAllCells) { diff --git a/src/client/datascience/interactive-common/interactiveBase.ts b/src/client/datascience/interactive-common/interactiveBase.ts index d59512819fb5..5206e923a21f 100644 --- a/src/client/datascience/interactive-common/interactiveBase.ts +++ b/src/client/datascience/interactive-common/interactiveBase.ts @@ -86,7 +86,6 @@ import { IJupyterVariablesResponse, IMessageCell, INotebook, - INotebookEditorProvider, INotebookExporter, INotebookServer, INotebookServerOptions, @@ -143,7 +142,6 @@ export abstract class InteractiveBase extends WebViewHost; protected async clearResult(id: string): Promise { + await this.ensureServerAndNotebook(); if (this._notebook) { this._notebook.clear(id); } @@ -701,47 +692,6 @@ export abstract class InteractiveBase extends WebViewHost { - // Take the list of cells, convert them to a notebook json format and write to disk - if (this._notebook) { - let directoryChange; - const settings = this.configuration.getSettings(await this.getOwningResource()); - if (settings.datascience.changeDirOnImportExport) { - directoryChange = file; - } - - const notebook = await this.jupyterExporter.translateToNotebook(cells, directoryChange); - - try { - const contents = JSON.stringify(notebook); - await this.fileSystem.writeFile(file, contents, { encoding: 'utf8', flag: 'w' }); - const openQuestion1 = localize.DataScience.exportOpenQuestion1(); - const openQuestion2 = (await this.jupyterExecution.isSpawnSupported()) - ? localize.DataScience.exportOpenQuestion() - : undefined; - this.showInformationMessage( - localize.DataScience.exportDialogComplete().format(file), - openQuestion1, - openQuestion2 - ).then(async (str: string | undefined) => { - try { - if (str === openQuestion2 && openQuestion2 && this._notebook) { - // If the user wants to, open the notebook they just generated. - await this.jupyterExecution.spawnNotebook(file); - } else if (str === openQuestion1) { - await this.ipynbProvider.open(Uri.file(file), contents); - } - } catch (e) { - await this.errorHandler.handleError(e); - } - }); - } catch (exc) { - traceError('Error in exporting notebook file'); - this.applicationShell.showInformationMessage(localize.DataScience.exportDialogFailed().format(exc)); - } - } - }; - protected setStatus = (message: string, showInWebView: boolean): Disposable => { const result = this.statusProvider.set(message, showInWebView, undefined, undefined, this); this.potentiallyUnfinishedStatus.push(result); @@ -1272,18 +1222,6 @@ export abstract class InteractiveBase extends WebViewHost { - if (question2) { - return this.applicationShell.showInformationMessage(message, question1, question2); - } else { - return this.applicationShell.showInformationMessage(message, question1); - } - } - private async ensureDarkSet(): Promise { if (!this.setDarkPromise) { this.setDarkPromise = createDeferred(); diff --git a/src/client/datascience/interactive-common/interactiveWindowTypes.ts b/src/client/datascience/interactive-common/interactiveWindowTypes.ts index 028aa09a9791..f18cb25b60d9 100644 --- a/src/client/datascience/interactive-common/interactiveWindowTypes.ts +++ b/src/client/datascience/interactive-common/interactiveWindowTypes.ts @@ -2,15 +2,22 @@ // Licensed under the MIT License. 'use strict'; import * as monacoEditor from 'monaco-editor/esm/vs/editor/editor.api'; +import { Uri } from 'vscode'; import { IServerState } from '../../../datascience-ui/interactive-common/mainState'; -import { CssMessages, IGetCssRequest, IGetCssResponse, IGetMonacoThemeRequest } from '../messages'; +import { CommonActionType, IAddCellAction } from '../../../datascience-ui/interactive-common/redux/reducers/types'; +import { PythonInterpreter } from '../../interpreter/contracts'; +import { LiveKernelModel } from '../jupyter/kernels/types'; +import { CssMessages, IGetCssRequest, IGetCssResponse, IGetMonacoThemeRequest, SharedMessages } from '../messages'; +import { IGetMonacoThemeResponse } from '../monacoMessages'; import { ICell, IInteractiveWindowInfo, + IJupyterKernelSpec, IJupyterVariable, IJupyterVariablesRequest, IJupyterVariablesResponse } from '../types'; +import { BaseReduxActionPayload } from './types'; export enum InteractiveWindowMessages { StartCell = 'start_cell', @@ -23,7 +30,6 @@ export enum InteractiveWindowMessages { Export = 'export_to_ipynb', GetAllCells = 'get_all_cells', ReturnAllCells = 'return_all_cells', - DeleteCell = 'delete_cell', DeleteAllCells = 'delete_all_cells', Undo = 'undo', Redo = 'redo', @@ -58,11 +64,7 @@ export enum InteractiveWindowMessages { ResolveCompletionItemRequest = 'resolve_completion_item_request', CancelResolveCompletionItemRequest = 'cancel_resolve_completion_item_request', ResolveCompletionItemResponse = 'resolve_completion_item_response', - AddCell = 'add_cell', - EditCell = 'edit_cell', - RemoveCell = 'remove_cell', - SwapCells = 'swap_cells', - InsertCell = 'insert_cell', + Sync = 'sync_message_used_to_broadcast_and_sync_editors', LoadOnigasmAssemblyRequest = 'load_onigasm_assembly_request', LoadOnigasmAssemblyResponse = 'load_onigasm_assembly_response', LoadTmLanguageRequest = 'load_tmlanguage_request', @@ -88,11 +90,14 @@ export enum InteractiveWindowMessages { NotebookAddCellBelow = 'notebook_add_cell_below', ExecutionRendered = 'rendered_execution', FocusedCellEditor = 'focused_cell_editor', + UnfocusedCellEditor = 'unfocused_cell_editor', MonacoReady = 'monaco_ready', ClearAllOutputs = 'clear_all_outputs', SelectKernel = 'select_kernel', UpdateKernel = 'update_kernel', SelectJupyterServer = 'select_jupyter_server', + UpdateModel = 'update_model', + ReceivedUpdateModel = 'received_update_model', OpenSettings = 'open_settings' } @@ -303,6 +308,138 @@ export interface IFocusedCellEditor { cellId: string; } +export interface INotebookModelChange { + oldDirty: boolean; + newDirty: boolean; + source: 'undo' | 'user' | 'redo'; +} + +export interface INotebookModelRemoveAllChange extends INotebookModelChange { + kind: 'remove_all'; + oldCells: ICell[]; + newCellId: string; +} +export interface INotebookModelModifyChange extends INotebookModelChange { + kind: 'modify'; + newCells: ICell[]; + oldCells: ICell[]; +} + +export interface INotebookModelClearChange extends INotebookModelChange { + kind: 'clear'; + oldCells: ICell[]; +} + +export interface INotebookModelSwapChange extends INotebookModelChange { + kind: 'swap'; + firstCellId: string; + secondCellId: string; +} + +export interface INotebookModelRemoveChange extends INotebookModelChange { + kind: 'remove'; + cell: ICell; + index: number; +} + +export interface INotebookModelInsertChange extends INotebookModelChange { + kind: 'insert'; + cell: ICell; + index: number; + codeCellAboveId?: string; +} + +export interface INotebookModelAddChange extends INotebookModelChange { + kind: 'add'; + cell: ICell; + fullText: string; + currentText: string; +} + +export interface IEditorPosition { + /** + * line number (starts at 1) + */ + readonly lineNumber: number; + /** + * column (the first character in a line is between column 1 and column 2) + */ + readonly column: number; +} + +export interface IEditorRange { + /** + * Line number on which the range starts (starts at 1). + */ + readonly startLineNumber: number; + /** + * Column on which the range starts in line `startLineNumber` (starts at 1). + */ + readonly startColumn: number; + /** + * Line number on which the range ends. + */ + readonly endLineNumber: number; + /** + * Column on which the range ends in line `endLineNumber`. + */ + readonly endColumn: number; +} + +export interface IEditorContentChange { + /** + * The range that got replaced. + */ + readonly range: IEditorRange; + /** + * The offset of the range that got replaced. + */ + readonly rangeOffset: number; + /** + * The length of the range that got replaced. + */ + readonly rangeLength: number; + /** + * The new text for the range. + */ + readonly text: string; + /** + * The cursor position to be set after the change + */ + readonly position: IEditorPosition; +} + +export interface INotebookModelEditChange extends INotebookModelChange { + kind: 'edit'; + forward: IEditorContentChange[]; + reverse: IEditorContentChange[]; + id: string; +} + +export interface INotebookModelVersionChange extends INotebookModelChange { + kind: 'version'; + interpreter: PythonInterpreter | undefined; + kernelSpec: IJupyterKernelSpec | LiveKernelModel | undefined; +} + +export interface INotebookModelFileChange extends INotebookModelChange { + kind: 'file'; + newFile: Uri; + oldFile: Uri; +} + +export type NotebookModelChange = + | INotebookModelModifyChange + | INotebookModelRemoveAllChange + | INotebookModelClearChange + | INotebookModelSwapChange + | INotebookModelRemoveChange + | INotebookModelInsertChange + | INotebookModelAddChange + | INotebookModelEditChange + | INotebookModelVersionChange + | INotebookModelFileChange; + // Map all messages to specific payloads export class IInteractiveWindowMapping { public [InteractiveWindowMessages.StartCell]: ICell; @@ -316,10 +453,9 @@ export class IInteractiveWindowMapping { public [InteractiveWindowMessages.SelectJupyterServer]: never | undefined; public [InteractiveWindowMessages.OpenSettings]: string | undefined; public [InteractiveWindowMessages.Export]: ICell[]; - public [InteractiveWindowMessages.GetAllCells]: ICell; + public [InteractiveWindowMessages.GetAllCells]: never | undefined; public [InteractiveWindowMessages.ReturnAllCells]: ICell[]; - public [InteractiveWindowMessages.DeleteCell]: never | undefined; - public [InteractiveWindowMessages.DeleteAllCells]: never | undefined; + public [InteractiveWindowMessages.DeleteAllCells]: IAddCellAction; public [InteractiveWindowMessages.Undo]: never | undefined; public [InteractiveWindowMessages.Redo]: never | undefined; public [InteractiveWindowMessages.ExpandAll]: never | undefined; @@ -342,6 +478,7 @@ export class IInteractiveWindowMapping { public [CssMessages.GetCssRequest]: IGetCssRequest; public [CssMessages.GetCssResponse]: IGetCssResponse; public [CssMessages.GetMonacoThemeRequest]: IGetMonacoThemeRequest; + public [CssMessages.GetMonacoThemeResponse]: IGetMonacoThemeResponse; public [InteractiveWindowMessages.ProvideCompletionItemsRequest]: IProvideCompletionItemsRequest; public [InteractiveWindowMessages.CancelCompletionItemsRequest]: ICancelIntellisenseRequest; public [InteractiveWindowMessages.ProvideCompletionItemsResponse]: IProvideCompletionItemsResponse; @@ -354,11 +491,6 @@ export class IInteractiveWindowMapping { public [InteractiveWindowMessages.ResolveCompletionItemRequest]: IResolveCompletionItemRequest; public [InteractiveWindowMessages.CancelResolveCompletionItemRequest]: ICancelIntellisenseRequest; public [InteractiveWindowMessages.ResolveCompletionItemResponse]: IResolveCompletionItemResponse; - public [InteractiveWindowMessages.AddCell]: IAddCell; - public [InteractiveWindowMessages.EditCell]: IEditCell; - public [InteractiveWindowMessages.RemoveCell]: IRemoveCell; - public [InteractiveWindowMessages.SwapCells]: ISwapCells; - public [InteractiveWindowMessages.InsertCell]: IInsertCell; public [InteractiveWindowMessages.LoadOnigasmAssemblyRequest]: never | undefined; public [InteractiveWindowMessages.LoadOnigasmAssemblyResponse]: Buffer; public [InteractiveWindowMessages.LoadTmLanguageRequest]: never | undefined; @@ -377,14 +509,25 @@ export class IInteractiveWindowMapping { public [InteractiveWindowMessages.NotebookDirty]: never | undefined; public [InteractiveWindowMessages.NotebookClean]: never | undefined; public [InteractiveWindowMessages.SaveAll]: ISaveAll; + public [InteractiveWindowMessages.Sync]: { + type: InteractiveWindowMessages | SharedMessages | CommonActionType; + // tslint:disable-next-line: no-any + payload: BaseReduxActionPayload; + }; public [InteractiveWindowMessages.NativeCommand]: INativeCommand; public [InteractiveWindowMessages.VariablesComplete]: never | undefined; public [InteractiveWindowMessages.NotebookRunAllCells]: never | undefined; public [InteractiveWindowMessages.NotebookRunSelectedCell]: never | undefined; - public [InteractiveWindowMessages.NotebookAddCellBelow]: never | undefined; + public [InteractiveWindowMessages.NotebookAddCellBelow]: IAddCellAction; + public [InteractiveWindowMessages.DoSave]: never | undefined; public [InteractiveWindowMessages.ExecutionRendered]: IRenderComplete; public [InteractiveWindowMessages.FocusedCellEditor]: IFocusedCellEditor; + public [InteractiveWindowMessages.UnfocusedCellEditor]: never | undefined; public [InteractiveWindowMessages.MonacoReady]: never | undefined; public [InteractiveWindowMessages.ClearAllOutputs]: never | undefined; public [InteractiveWindowMessages.UpdateKernel]: IServerState | undefined; + public [InteractiveWindowMessages.UpdateModel]: NotebookModelChange; + public [InteractiveWindowMessages.ReceivedUpdateModel]: never | undefined; + public [SharedMessages.UpdateSettings]: string; + public [SharedMessages.LocInit]: string; } diff --git a/src/client/datascience/interactive-common/synchronization.ts b/src/client/datascience/interactive-common/synchronization.ts new file mode 100644 index 000000000000..03c4741f443b --- /dev/null +++ b/src/client/datascience/interactive-common/synchronization.ts @@ -0,0 +1,219 @@ +import { + CommonActionType, + CommonActionTypeMapping +} from '../../../datascience-ui/interactive-common/redux/reducers/types'; +import { CssMessages, SharedMessages } from '../messages'; +import { IInteractiveWindowMapping, InteractiveWindowMessages } from './interactiveWindowTypes'; + +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +export enum MessageType { + /** + * Action dispatched as result of some user action. + */ + userAction = 0, + /** + * Action dispatched to re-broadcast a message across other editors of the same file in the same session. + */ + syncAcrossSameNotebooks = 1 << 0, + /** + * Action dispatched to re-broadcast a message across other sessions (live share). + */ + syncWithLiveShare = 1 << 1, + noIdea = 1 << 2 +} + +type MessageMapping = { + [P in keyof T]: MessageType; +}; + +export type IInteractiveActionMapping = MessageMapping; + +// Do not change to a dictionary or a record. +// The current structure ensures all new enums added will be categorized. +// This way, if a new message is added, we'll make the decision early on whether it needs to be synchronized and how. +// Rather than waiting for users to report issues related to new messages. +const messageWithMessageTypes: MessageMapping & MessageMapping = { + [CommonActionType.ADD_AND_FOCUS_NEW_CELL]: MessageType.userAction, + [CommonActionType.ADD_NEW_CELL]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.ARROW_DOWN]: MessageType.syncWithLiveShare, + [CommonActionType.ARROW_UP]: MessageType.syncWithLiveShare, + [CommonActionType.CHANGE_CELL_TYPE]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.CLICK_CELL]: MessageType.syncWithLiveShare, + [CommonActionType.DELETE_CELL]: MessageType.syncWithLiveShare, + [CommonActionType.CODE_CREATED]: MessageType.noIdea, + [CommonActionType.COPY_CELL_CODE]: MessageType.userAction, + [CommonActionType.EDITOR_LOADED]: MessageType.userAction, + [CommonActionType.EDIT_CELL]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.EXECUTE_CELL_AND_ADVANCE]: MessageType.userAction, + [CommonActionType.EXECUTE_ABOVE]: MessageType.userAction, + [CommonActionType.EXECUTE_ALL_CELLS]: MessageType.userAction, + [CommonActionType.EXECUTE_CELL]: MessageType.userAction, + [CommonActionType.EXECUTE_CELL_AND_BELOW]: MessageType.userAction, + [CommonActionType.EXPORT]: MessageType.userAction, + [CommonActionType.FOCUS_CELL]: MessageType.syncWithLiveShare, + [CommonActionType.GATHER_CELL]: MessageType.userAction, + [CommonActionType.GET_VARIABLE_DATA]: MessageType.userAction, + [CommonActionType.GOTO_CELL]: MessageType.syncWithLiveShare, + [CommonActionType.INSERT_ABOVE_AND_FOCUS_NEW_CELL]: MessageType.userAction, + [CommonActionType.INSERT_ABOVE]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.INSERT_ABOVE_FIRST_AND_FOCUS_NEW_CELL]: MessageType.userAction, + [CommonActionType.INSERT_ABOVE_FIRST]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.INSERT_BELOW]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.INSERT_BELOW_AND_FOCUS_NEW_CELL]: MessageType.userAction, + [CommonActionType.INTERRUPT_KERNEL]: MessageType.userAction, + [CommonActionType.LOADED_ALL_CELLS]: MessageType.userAction, + [CommonActionType.LINK_CLICK]: MessageType.userAction, + [CommonActionType.MOVE_CELL_DOWN]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.MOVE_CELL_UP]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.OPEN_SETTINGS]: MessageType.userAction, + [CommonActionType.RESTART_KERNEL]: MessageType.userAction, + [CommonActionType.SAVE]: MessageType.userAction, + [CommonActionType.SCROLL]: MessageType.syncWithLiveShare, + [CommonActionType.SELECT_CELL]: MessageType.syncWithLiveShare, + [CommonActionType.SELECT_SERVER]: MessageType.userAction, + [CommonActionType.SEND_COMMAND]: MessageType.userAction, + [CommonActionType.SHOW_DATA_VIEWER]: MessageType.userAction, + [CommonActionType.SUBMIT_INPUT]: MessageType.userAction, + [CommonActionType.TOGGLE_INPUT_BLOCK]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [CommonActionType.TOGGLE_LINE_NUMBERS]: MessageType.syncWithLiveShare, + [CommonActionType.TOGGLE_OUTPUT]: MessageType.syncWithLiveShare, + [CommonActionType.TOGGLE_VARIABLE_EXPLORER]: MessageType.syncWithLiveShare, + [CommonActionType.UNFOCUS_CELL]: MessageType.syncWithLiveShare, + [CommonActionType.UNMOUNT]: MessageType.userAction, + [CommonActionType.PostOutgoingMessage]: MessageType.userAction, + [CommonActionType.REFRESH_VARIABLES]: MessageType.userAction, + [CommonActionType.FOCUS_INPUT]: MessageType.userAction, + + // Types from InteractiveWindowMessages + [InteractiveWindowMessages.Activate]: MessageType.userAction, + [InteractiveWindowMessages.AddedSysInfo]: MessageType.userAction, + [InteractiveWindowMessages.CancelCompletionItemsRequest]: MessageType.userAction, + [InteractiveWindowMessages.CancelHoverRequest]: MessageType.userAction, + [InteractiveWindowMessages.CancelResolveCompletionItemRequest]: MessageType.userAction, + [InteractiveWindowMessages.CancelSignatureHelpRequest]: MessageType.userAction, + [InteractiveWindowMessages.ClearAllOutputs]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [InteractiveWindowMessages.CollapseAll]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.CopyCodeCell]: MessageType.userAction, + [InteractiveWindowMessages.DeleteAllCells]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [InteractiveWindowMessages.DoSave]: MessageType.userAction, + [InteractiveWindowMessages.ExecutionRendered]: MessageType.userAction, + [InteractiveWindowMessages.ExpandAll]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.Export]: MessageType.userAction, + [InteractiveWindowMessages.FinishCell]: MessageType.userAction, + [InteractiveWindowMessages.FocusedCellEditor]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.GatherCodeRequest]: MessageType.userAction, + [InteractiveWindowMessages.GetAllCells]: MessageType.userAction, + [InteractiveWindowMessages.GetVariablesRequest]: MessageType.userAction, + [InteractiveWindowMessages.GetVariablesResponse]: MessageType.userAction, + [InteractiveWindowMessages.GotoCodeCell]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.GotoCodeCell]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.Interrupt]: MessageType.userAction, + [InteractiveWindowMessages.LoadAllCells]: MessageType.userAction, + [InteractiveWindowMessages.LoadAllCellsComplete]: MessageType.userAction, + [InteractiveWindowMessages.LoadOnigasmAssemblyRequest]: MessageType.userAction, + [InteractiveWindowMessages.LoadOnigasmAssemblyResponse]: MessageType.userAction, + [InteractiveWindowMessages.LoadTmLanguageRequest]: MessageType.userAction, + [InteractiveWindowMessages.LoadTmLanguageResponse]: MessageType.userAction, + [InteractiveWindowMessages.MonacoReady]: MessageType.userAction, + [InteractiveWindowMessages.NativeCommand]: MessageType.userAction, + [InteractiveWindowMessages.NotebookAddCellBelow]: + MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [InteractiveWindowMessages.NotebookClean]: MessageType.userAction, + [InteractiveWindowMessages.NotebookDirty]: MessageType.userAction, + [InteractiveWindowMessages.NotebookExecutionActivated]: MessageType.userAction, + [InteractiveWindowMessages.NotebookIdentity]: MessageType.userAction, + [InteractiveWindowMessages.NotebookRunAllCells]: MessageType.userAction, + [InteractiveWindowMessages.NotebookRunSelectedCell]: MessageType.userAction, + [InteractiveWindowMessages.OpenLink]: MessageType.userAction, + [InteractiveWindowMessages.OpenSettings]: MessageType.userAction, + [InteractiveWindowMessages.ProvideCompletionItemsRequest]: MessageType.userAction, + [InteractiveWindowMessages.ProvideCompletionItemsResponse]: MessageType.userAction, + [InteractiveWindowMessages.ProvideHoverRequest]: MessageType.userAction, + [InteractiveWindowMessages.ProvideHoverResponse]: MessageType.userAction, + [InteractiveWindowMessages.ProvideSignatureHelpRequest]: MessageType.userAction, + [InteractiveWindowMessages.ProvideSignatureHelpResponse]: MessageType.userAction, + [InteractiveWindowMessages.ReExecuteCells]: MessageType.userAction, + [InteractiveWindowMessages.Redo]: MessageType.userAction, + [InteractiveWindowMessages.RemoteAddCode]: MessageType.userAction, + [InteractiveWindowMessages.ReceivedUpdateModel]: MessageType.userAction, + [InteractiveWindowMessages.RemoteReexecuteCode]: MessageType.userAction, + [InteractiveWindowMessages.ResolveCompletionItemRequest]: MessageType.userAction, + [InteractiveWindowMessages.ResolveCompletionItemResponse]: MessageType.userAction, + [InteractiveWindowMessages.RestartKernel]: MessageType.userAction, + [InteractiveWindowMessages.ReturnAllCells]: MessageType.userAction, + [InteractiveWindowMessages.SaveAll]: MessageType.userAction, + [InteractiveWindowMessages.SavePng]: MessageType.userAction, + [InteractiveWindowMessages.ScrollToCell]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.SelectJupyterServer]: MessageType.userAction, + [InteractiveWindowMessages.SelectKernel]: MessageType.userAction, + [InteractiveWindowMessages.SendInfo]: MessageType.userAction, + [InteractiveWindowMessages.SettingsUpdated]: MessageType.userAction, + [InteractiveWindowMessages.ShowDataViewer]: MessageType.userAction, + [InteractiveWindowMessages.ShowPlot]: MessageType.userAction, + [InteractiveWindowMessages.StartCell]: MessageType.userAction, + [InteractiveWindowMessages.StartDebugging]: MessageType.userAction, + [InteractiveWindowMessages.StartProgress]: MessageType.userAction, + [InteractiveWindowMessages.Started]: MessageType.userAction, + [InteractiveWindowMessages.StopDebugging]: MessageType.userAction, + [InteractiveWindowMessages.StopProgress]: MessageType.userAction, + [InteractiveWindowMessages.SubmitNewCell]: MessageType.userAction, + [InteractiveWindowMessages.Sync]: MessageType.userAction, + [InteractiveWindowMessages.Undo]: MessageType.userAction, + [InteractiveWindowMessages.UnfocusedCellEditor]: MessageType.syncWithLiveShare, + [InteractiveWindowMessages.UpdateCell]: MessageType.userAction, + [InteractiveWindowMessages.UpdateModel]: MessageType.syncAcrossSameNotebooks | MessageType.syncWithLiveShare, + [InteractiveWindowMessages.UpdateKernel]: MessageType.userAction, + [InteractiveWindowMessages.VariableExplorerToggle]: MessageType.userAction, + [InteractiveWindowMessages.VariablesComplete]: MessageType.userAction, + // Types from CssMessages + [CssMessages.GetCssRequest]: MessageType.userAction, + [CssMessages.GetCssResponse]: MessageType.userAction, + [CssMessages.GetMonacoThemeRequest]: MessageType.userAction, + [CssMessages.GetMonacoThemeResponse]: MessageType.userAction, + // Types from Shared Messages + [SharedMessages.LocInit]: MessageType.userAction, + [SharedMessages.Started]: MessageType.userAction, + [SharedMessages.UpdateSettings]: MessageType.userAction +}; + +/** + * If the original message was a sync message, then do not send messages to extension. + * We allow messages to be sent to extension ONLY when the original message was triggered by the user. + * + * @export + * @param {MessageType} [messageType] + * @returns + */ +export function checkToPostBasedOnOriginalMessageType(messageType?: MessageType): boolean { + if (!messageType) { + return true; + } + if ( + (messageType & MessageType.syncAcrossSameNotebooks) === MessageType.syncAcrossSameNotebooks || + (messageType & MessageType.syncWithLiveShare) === MessageType.syncWithLiveShare + ) { + return false; + } + + return true; +} + +export function shouldRebroadcast(message: keyof IInteractiveWindowMapping): [boolean, MessageType] { + // Get the configured type for this message (whether it should be re-broadcasted or not). + const messageType: MessageType | undefined = messageWithMessageTypes[message]; + // Support for liveshare is turned off for now, we can enable that later. + // I.e. we only support synchronizing across editors in the same session. + if ( + messageType === undefined || + (messageType & MessageType.syncAcrossSameNotebooks) !== MessageType.syncAcrossSameNotebooks + ) { + return [false, MessageType.userAction]; + } + + return [ + (messageType & MessageType.syncAcrossSameNotebooks) > 0 || (messageType & MessageType.syncWithLiveShare) > 0, + messageType + ]; +} diff --git a/src/client/datascience/interactive-common/types.ts b/src/client/datascience/interactive-common/types.ts new file mode 100644 index 000000000000..bab7a55d2e09 --- /dev/null +++ b/src/client/datascience/interactive-common/types.ts @@ -0,0 +1,34 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +'use strict'; + +import { MessageType } from './synchronization'; + +// Stuff common to React and Extensions. + +type BaseData = { + messageType?: MessageType; + /** + * Tells us whether this message is incoming for reducer use or + * whether this is a message that needs to be sent out to extension (from reducer). + */ + messageDirection?: 'incoming' | 'outgoing'; +}; + +type BaseDataWithPayload = { + messageType?: MessageType; + /** + * Tells us whether this message is incoming for reducer use or + * whether this is a message that needs to be sent out to extension (from reducer). + */ + messageDirection?: 'incoming' | 'outgoing'; + data: T; +}; + +// This forms the base content of every payload in all dispatchers. +export type BaseReduxActionPayload = T extends never + ? T extends undefined + ? BaseData + : BaseDataWithPayload + : BaseDataWithPayload; diff --git a/src/client/datascience/interactive-ipynb/autoSaveService.ts b/src/client/datascience/interactive-ipynb/autoSaveService.ts deleted file mode 100644 index 166c44a2d6c1..000000000000 --- a/src/client/datascience/interactive-ipynb/autoSaveService.ts +++ /dev/null @@ -1,152 +0,0 @@ -// Copyright (c) Microsoft Corporation. All rights reserved. -// Licensed under the MIT License. - -'use strict'; - -import { inject, injectable } from 'inversify'; -import { ConfigurationChangeEvent, Event, EventEmitter, TextEditor, Uri } from 'vscode'; -import { IApplicationShell, IDocumentManager, IWorkspaceService } from '../../common/application/types'; -import '../../common/extensions'; -import { traceError } from '../../common/logger'; -import { IFileSystem } from '../../common/platform/types'; -import { IDisposable } from '../../common/types'; -import { INotebookIdentity, InteractiveWindowMessages } from '../interactive-common/interactiveWindowTypes'; -import { FileSettings, IInteractiveWindowListener, INotebookEditor, INotebookEditorProvider } from '../types'; - -// tslint:disable: no-any - -/** - * Sends notifications to Notebooks to save the notebook. - * Based on auto save settings, this class will regularly check for changes and send a save requet. - * If window state changes or active editor changes, then notify notebooks (if auto save is configured to do so). - * Monitor save and modified events on editor to determine its current dirty state. - * - * @export - * @class AutoSaveService - * @implements {IInteractiveWindowListener} - */ -@injectable() -export class AutoSaveService implements IInteractiveWindowListener { - private postEmitter: EventEmitter<{ message: string; payload: any }> = new EventEmitter<{ - message: string; - payload: any; - }>(); - private disposables: IDisposable[] = []; - private notebookUri?: Uri; - private timeout?: ReturnType; - constructor( - @inject(IApplicationShell) appShell: IApplicationShell, - @inject(IDocumentManager) documentManager: IDocumentManager, - @inject(INotebookEditorProvider) private readonly notebookProvider: INotebookEditorProvider, - @inject(IFileSystem) private readonly fileSystem: IFileSystem, - @inject(IWorkspaceService) private readonly workspace: IWorkspaceService - ) { - this.workspace.onDidChangeConfiguration(this.onSettingsChanded.bind(this), this, this.disposables); - this.disposables.push(appShell.onDidChangeWindowState(this.onDidChangeWindowState.bind(this))); - this.disposables.push(documentManager.onDidChangeActiveTextEditor(this.onDidChangeActiveTextEditor.bind(this))); - } - - public get postMessage(): Event<{ message: string; payload: any }> { - return this.postEmitter.event; - } - - public onMessage(message: string, payload?: any): void { - if (message === InteractiveWindowMessages.NotebookIdentity) { - this.notebookUri = Uri.parse((payload as INotebookIdentity).resource); - } - if (message === InteractiveWindowMessages.LoadAllCellsComplete) { - const notebook = this.getNotebook(); - if (!notebook) { - traceError( - `Received message ${message}, but there is no notebook for ${ - this.notebookUri ? this.notebookUri.fsPath : undefined - }` - ); - return; - } - this.disposables.push(notebook.modified(this.onNotebookModified, this, this.disposables)); - this.disposables.push(notebook.saved(this.onNotebookSaved, this, this.disposables)); - } - } - public dispose(): void | undefined { - this.disposables.filter(item => !!item).forEach(item => item.dispose()); - this.clearTimeout(); - } - private onNotebookModified(_: INotebookEditor) { - // If we haven't started a timer, then start if necessary. - if (!this.timeout) { - this.setTimer(); - } - } - private onNotebookSaved(_: INotebookEditor) { - // If we haven't started a timer, then start if necessary. - if (!this.timeout) { - this.setTimer(); - } - } - private getNotebook(): INotebookEditor | undefined { - const uri = this.notebookUri; - if (!uri) { - return; - } - return this.notebookProvider.editors.find(item => this.fileSystem.arePathsSame(item.file.fsPath, uri.fsPath)); - } - private getAutoSaveSettings(): FileSettings { - const filesConfig = this.workspace.getConfiguration('files', this.notebookUri); - return { - autoSave: filesConfig.get('autoSave', 'off'), - autoSaveDelay: filesConfig.get('autoSaveDelay', 1000) - }; - } - private onSettingsChanded(e: ConfigurationChangeEvent) { - if ( - e.affectsConfiguration('files.autoSave', this.notebookUri) || - e.affectsConfiguration('files.autoSaveDelay', this.notebookUri) - ) { - // Reset the timer, as we may have increased it, turned it off or other. - this.clearTimeout(); - this.setTimer(); - } - } - private setTimer() { - const settings = this.getAutoSaveSettings(); - if (!settings || settings.autoSave === 'off') { - return; - } - if (settings && settings.autoSave === 'afterDelay') { - // Add a timeout to save after n milli seconds. - // Do not use setInterval, as that will cause all handlers to queue up. - this.timeout = setTimeout(() => { - this.save(); - }, settings.autoSaveDelay); - } - } - private clearTimeout() { - if (this.timeout) { - clearTimeout(this.timeout); - this.timeout = undefined; - } - } - private save() { - this.clearTimeout(); - const notebook = this.getNotebook(); - if (notebook && notebook.isDirty && !notebook.isUntitled) { - // Notify webview to perform a save. - this.postEmitter.fire({ message: InteractiveWindowMessages.DoSave, payload: undefined }); - } else { - this.setTimer(); - } - } - private onDidChangeWindowState() { - const settings = this.getAutoSaveSettings(); - if (settings && settings.autoSave === 'onWindowChange') { - this.save(); - } - } - private onDidChangeActiveTextEditor(_e?: TextEditor) { - const settings = this.getAutoSaveSettings(); - if (settings && settings.autoSave === 'onFocusChange') { - this.save(); - } - } -} diff --git a/src/client/datascience/interactive-ipynb/nativeEditor.ts b/src/client/datascience/interactive-ipynb/nativeEditor.ts index 8e90d416bfdc..82b21d194730 100644 --- a/src/client/datascience/interactive-ipynb/nativeEditor.ts +++ b/src/client/datascience/interactive-ipynb/nativeEditor.ts @@ -3,15 +3,21 @@ 'use strict'; import '../../common/extensions'; -import { nbformat } from '@jupyterlab/coreutils/lib/nbformat'; -import * as detectIndent from 'detect-indent'; import { inject, injectable, multiInject, named } from 'inversify'; import * as path from 'path'; -import * as uuid from 'uuid/v4'; -import { CancellationToken, CancellationTokenSource, Event, EventEmitter, Memento, Uri, ViewColumn } from 'vscode'; +import { + CancellationToken, + CancellationTokenSource, + Event, + EventEmitter, + Memento, + Uri, + ViewColumn, + WebviewPanel +} from 'vscode'; -import { concatMultilineStringInput, splitMultilineString } from '../../../datascience-ui/common'; -import { createCodeCell, createErrorOutput } from '../../../datascience-ui/common/cellFactory'; +import * as uuid from 'uuid/v4'; +import { createErrorOutput } from '../../../datascience-ui/common/cellFactory'; import { IApplicationShell, ICommandManager, @@ -27,17 +33,13 @@ import { GLOBAL_MEMENTO, IAsyncDisposableRegistry, IConfigurationService, - ICryptoUtils, IDisposableRegistry, IExperimentsManager, - IExtensionContext, IMemento, - Resource, - WORKSPACE_MEMENTO + Resource } from '../../common/types'; import { createDeferred, Deferred } from '../../common/utils/async'; import * as localize from '../../common/utils/localize'; -import { noop } from '../../common/utils/misc'; import { StopWatch } from '../../common/utils/stopWatch'; import { EXTENSION_ROOT_DIR } from '../../constants'; import { IInterpreterService } from '../../interpreter/contracts'; @@ -45,25 +47,19 @@ import { captureTelemetry, sendTelemetryEvent } from '../../telemetry'; import { EditorContexts, Identifiers, - KnownNotebookLanguages, NativeKeyboardCommandTelemetryLookup, NativeMouseCommandTelemetryLookup, Telemetry } from '../constants'; import { InteractiveBase } from '../interactive-common/interactiveBase'; import { - IEditCell, - IInsertCell, INativeCommand, InteractiveWindowMessages, IReExecuteCells, - IRemoveCell, - ISaveAll, ISubmitNewCell, - ISwapCells, + NotebookModelChange, SysInfoReason } from '../interactive-common/interactiveWindowTypes'; -import { InvalidNotebookFileError } from '../jupyter/invalidNotebookFileError'; import { ProgressReporter } from '../progress/progressReporter'; import { CellState, @@ -80,25 +76,18 @@ import { INotebookEditorProvider, INotebookExporter, INotebookImporter, + INotebookModel, INotebookServerOptions, IStatusProvider, IThemeFinder, WebViewViewChangeEventArgs } from '../types'; -// tslint:disable-next-line:no-require-imports no-var-requires -const debounce = require('lodash/debounce') as typeof import('lodash/debounce'); +import { nbformat } from '@jupyterlab/coreutils'; +// tslint:disable-next-line: no-require-imports +import cloneDeep = require('lodash/cloneDeep'); const nativeEditorDir = path.join(EXTENSION_ROOT_DIR, 'out', 'datascience-ui', 'notebook'); -enum AskForSaveResult { - Yes, - No, - Cancel -} - -const KeyPrefix = 'notebook-storage-'; -const NotebookTransferKey = 'notebook-transfered'; - @injectable() export class NativeEditor extends InteractiveBase implements INotebookEditor { public get onDidChangeViewState(): Event { @@ -114,16 +103,14 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { } public get file(): Uri { - return this._file; + if (this.model) { + return this.model.file; + } + return Uri.file(''); } public get isUntitled(): boolean { - const baseName = path.basename(this.file.fsPath); - return baseName.includes(localize.DataScience.untitledNotebookFileName()); - } - - public get cells(): ICell[] { - return this.visibleCells; + return this.model ? this.model.isUntitled : false; } public get closed(): Event { @@ -137,38 +124,25 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { public get modified(): Event { return this.modifiedEvent.event; } - public get saved(): Event { return this.savedEvent.event; } - public get metadataUpdated(): Event { - return this.metadataUpdatedEvent.event; - } - public get isDirty(): boolean { - return this._dirty; + return this.model ? this.model.isDirty : false; } + protected savedEvent: EventEmitter = new EventEmitter(); + protected model: INotebookModel | undefined; + protected closedEvent: EventEmitter = new EventEmitter(); + protected modifiedEvent: EventEmitter = new EventEmitter(); + private sentExecuteCellTelemetry: boolean = false; private _onDidChangeViewState = new EventEmitter(); - private closedEvent: EventEmitter = new EventEmitter(); private executedEvent: EventEmitter = new EventEmitter(); - private modifiedEvent: EventEmitter = new EventEmitter(); - private savedEvent: EventEmitter = new EventEmitter(); - private metadataUpdatedEvent: EventEmitter = new EventEmitter(); private loadedPromise: Deferred = createDeferred(); - private contentsLoadedPromise: Deferred = createDeferred(); - private _file: Uri = Uri.file(''); - private _dirty: boolean = false; - private isPromptingToSaveToDisc: boolean = false; - private visibleCells: ICell[] = []; private startupTimer: StopWatch = new StopWatch(); private loadedAllCells: boolean = false; - private indentAmount: string = ' '; - private notebookJson: Partial = {}; - private debouncedWriteToStorage = debounce(this.writeToStorage.bind(this), 250); private executeCancelTokens = new Set(); - private _disposed = false; constructor( @multiInject(IInteractiveWindowListener) listeners: IInteractiveWindowListener[], @@ -187,16 +161,13 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { @inject(ICommandManager) commandManager: ICommandManager, @inject(INotebookExporter) jupyterExporter: INotebookExporter, @inject(IWorkspaceService) workspaceService: IWorkspaceService, - @inject(INotebookEditorProvider) editorProvider: INotebookEditorProvider, + @inject(INotebookEditorProvider) private editorProvider: INotebookEditorProvider, @inject(IDataViewerProvider) dataExplorerProvider: IDataViewerProvider, @inject(IJupyterVariables) jupyterVariables: IJupyterVariables, @inject(IJupyterDebugger) jupyterDebugger: IJupyterDebugger, - @inject(INotebookImporter) private importer: INotebookImporter, + @inject(INotebookImporter) protected readonly importer: INotebookImporter, @inject(IDataScienceErrorHandler) errorHandler: IDataScienceErrorHandler, @inject(IMemento) @named(GLOBAL_MEMENTO) globalStorage: Memento, - @inject(IMemento) @named(WORKSPACE_MEMENTO) private localStorage: Memento, - @inject(ICryptoUtils) private crypto: ICryptoUtils, - @inject(IExtensionContext) private context: IExtensionContext, @inject(ProgressReporter) progressReporter: ProgressReporter, @inject(IExperimentsManager) experimentsManager: IExperimentsManager, @inject(IAsyncDisposableRegistry) asyncRegistry: IAsyncDisposableRegistry @@ -221,7 +192,6 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { dataExplorerProvider, jupyterVariables, jupyterDebugger, - editorProvider, errorHandler, commandManager, globalStorage, @@ -237,48 +207,35 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { ); asyncRegistry.push(this); } + public dispose(): Promise { - this._disposed = true; super.dispose(); return this.close(); } - public getContents(): Promise { - return this.generateNotebookContent(this.visibleCells); - } - - public async load(contents: string, file: Uri): Promise { - // Save our uri - this._file = file; + public async load(model: INotebookModel, webViewPanel: WebviewPanel): Promise { + // Save the model we're using + this.model = model; // Indicate we have our identity this.loadedPromise.resolve(); // Load the web panel using our file path so it can find // relative files next to the notebook. - await super.loadWebPanel(path.dirname(file.fsPath)); - - // Update our title to match - this.setTitle(path.basename(file.fsPath)); + await super.loadWebPanel(path.dirname(this.file.fsPath), webViewPanel); - // Show ourselves - await this.show(); - - // Clear out old global storage the first time somebody opens - // a notebook - if (!this.globalStorage.get(NotebookTransferKey)) { - await this.transferStorage(); - } + // Sign up for dirty events + model.changed(this.modelChanged.bind(this)); - // See if this file was stored in storage prior to shutdown - const dirtyContents = await this.getStoredContents(); - if (dirtyContents) { - // This means we're dirty. Indicate dirty and load from this content - return this.loadContents(dirtyContents, true); - } else { - // Load without setting dirty - return this.loadContents(contents, false); - } + // Load our cells, but don't wait for this to finish, otherwise the window won't load. + this.sendInitialCellsToWebView(model.cells) + .then(() => { + // May alread be dirty, if so send a message + if (model.isDirty) { + this.postMessage(InteractiveWindowMessages.NotebookDirty).ignoreErrors(); + } + }) + .catch(exc => traceError('Error loading cells: ', exc)); } // tslint:disable-next-line: no-any @@ -297,24 +254,8 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { this.handleMessage(message, payload, this.export); break; - case InteractiveWindowMessages.EditCell: - this.handleMessage(message, payload, this.editCell); - break; - - case InteractiveWindowMessages.InsertCell: - this.handleMessage(message, payload, this.insertCell); - break; - - case InteractiveWindowMessages.RemoveCell: - this.handleMessage(message, payload, this.removeCell); - break; - - case InteractiveWindowMessages.SwapCells: - this.handleMessage(message, payload, this.swapCells); - break; - - case InteractiveWindowMessages.DeleteAllCells: - this.handleMessage(message, payload, this.removeAllCells); + case InteractiveWindowMessages.UpdateModel: + this.handleMessage(message, payload, this.updateModel); break; case InteractiveWindowMessages.NativeCommand: @@ -326,10 +267,6 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { this.handleMessage(message, payload, this.loadCellsComplete); break; - case InteractiveWindowMessages.ClearAllOutputs: - this.handleMessage(message, payload, this.clearAllOutputs); - break; - case InteractiveWindowMessages.RestartKernel: this.interruptExecution(); break; @@ -344,13 +281,17 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { } public async getNotebookOptions(): Promise { - const options = await this.ipynbProvider.getNotebookOptions(await this.getOwningResource()); - await this.contentsLoadedPromise.promise; - const metadata = this.notebookJson.metadata; - return { - ...options, - metadata - }; + const options = await this.editorProvider.getNotebookOptions(await this.getOwningResource()); + await this.loadedPromise.promise; + if (this.model) { + const metadata = (await this.model.getJson()).metadata; + return { + ...options, + metadata + }; + } else { + return options; + } } public runAllCells() { @@ -362,14 +303,7 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { } public addCellBelow() { - this.postMessage(InteractiveWindowMessages.NotebookAddCellBelow).ignoreErrors(); - } - - public async removeAllCells(): Promise { - super.removeAllCells(); - // Clear our visible cells - this.visibleCells = []; - return this.setDirty(); + this.postMessage(InteractiveWindowMessages.NotebookAddCellBelow, { newCellId: uuid() }).ignoreErrors(); } public getOwningResource(): Promise { @@ -387,30 +321,6 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { return Promise.resolve(); } - protected async reopen(cells: ICell[]): Promise { - try { - // Reload the web panel too. - await super.loadWebPanel(path.basename(this._file.fsPath)); - await this.show(); - - // Indicate we have our identity - this.loadedPromise.resolve(); - - // Update our title to match - if (this._dirty) { - this._dirty = false; - await this.setDirty(); - } else { - this.setTitle(path.basename(this._file.fsPath)); - } - - // If that works, send the cells to the web view - return this.postMessage(InteractiveWindowMessages.LoadAllCells, { cells }); - } catch (e) { - return this.errorHandler.handleError(e); - } - } - protected submitCode( code: string, file: string, @@ -421,20 +331,9 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { cancelToken?: CancellationToken ): Promise { const stopWatch = new StopWatch(); - const submitCodePromise = super + return super .submitCode(code, file, line, id, data, debug, cancelToken) .finally(() => this.sendPerceivedCellExecute(stopWatch)); - // When code is executed, update the version number in the metadata. - return submitCodePromise.then(value => { - this.updateVersionInfoInNotebook() - .then(() => { - this.metadataUpdatedEvent.fire(this); - }) - .catch(ex => { - traceError('Failed to update version info in notebook file metadata', ex); - }); - return value; - }); } @captureTelemetry(Telemetry.SubmitCellThroughInput, undefined, false) @@ -444,7 +343,7 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { if (info && info.code && info.id) { try { // Activate the other side, and send as if came from a file - this.ipynbProvider + this.editorProvider .show(this.file) .then(_v => { this.shareMessage(InteractiveWindowMessages.RemoteAddCode, { @@ -505,7 +404,7 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { } // File should be set now - return this._file; + return this.file; } protected async setLaunchingFile(_file: string): Promise { @@ -520,14 +419,34 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { // Filter out sysinfo messages. Don't want to show those const filtered = cells.filter(c => c.data.cell_type !== 'messages'); - // Update these cells in our list - cells.forEach(c => { - const index = this.visibleCells.findIndex(v => v.id === c.id); - this.visibleCells[index] = c; - }); + // Update these cells in our storage only when cells are finished + const modified = filtered.filter(c => c.state === CellState.finished || c.state === CellState.error); + const unmodified = this.model?.cells.filter(c => modified.find(m => m.id === c.id)); + if (modified.length > 0 && unmodified && this.model) { + this.model.update({ + source: 'user', + kind: 'modify', + newCells: modified, + oldCells: cloneDeep(unmodified), + oldDirty: this.model.isDirty, + newDirty: true + }); + } - // Indicate dirty - this.setDirty().ignoreErrors(); + // Tell storage about our notebook object + const notebook = this.getNotebook(); + if (notebook && this.model) { + const interpreter = notebook.getMatchingInterpreter(); + const kernelSpec = notebook.getKernelSpec(); + this.model.update({ + source: 'user', + kind: 'version', + oldDirty: this.model.isDirty, + newDirty: this.model.isDirty, + interpreter, + kernelSpec + }); + } // Send onto the webview. super.sendCellsToWebView(filtered); @@ -568,6 +487,49 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { // Actually don't close, just let the error bubble out } + protected async close(): Promise { + // Fire our event + this.closedEvent.fire(this); + + // Restart our kernel so that execution counts are reset + let oldAsk: boolean | undefined = false; + const settings = this.configuration.getSettings(); + if (settings && settings.datascience) { + oldAsk = settings.datascience.askForKernelRestart; + settings.datascience.askForKernelRestart = false; + } + await this.restartKernel(); + if (oldAsk && settings && settings.datascience) { + settings.datascience.askForKernelRestart = true; + } + } + + protected saveAll() { + // Ask user for a save as dialog if no title + if (this.isUntitled) { + this.commandManager.executeCommand('workbench.action.files.saveAs', this.file); + } else { + this.commandManager.executeCommand('workbench.action.files.save', this.file); + } + } + + private async modelChanged(change: NotebookModelChange) { + if (change.source !== 'user') { + // VS code is telling us to broadcast this to our UI. Tell the UI about the new change + await this.postMessage(InteractiveWindowMessages.UpdateModel, change); + } + + // Use the current state of the model to indicate dirty (not the message itself) + if (this.model && change.newDirty !== change.oldDirty) { + this.modifiedEvent.fire(); + if (this.model.isDirty) { + await this.postMessage(InteractiveWindowMessages.NotebookDirty); + } else { + // Then tell the UI + await this.postMessage(InteractiveWindowMessages.NotebookClean); + } + } + } private interruptExecution() { this.executeCancelTokens.forEach(t => t.cancel()); } @@ -602,19 +564,6 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { false, cancelToken ); - - if (!cancelToken.isCancellationRequested) { - // Activate the other side, and send as if came from a file - await this.ipynbProvider.show(this.file); - this.shareMessage(InteractiveWindowMessages.RemoteReexecuteCode, { - code: entry.code, - file: Identifiers.EmptyFileName, - line: 0, - id: entry.cell.id, - originator: this.id, - debug: false - }); - } } } catch (exc) { // Make this error our cell output @@ -649,489 +598,20 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { } } - /** - * Update the Python Version number in the notebook data. - * - * @private - * @memberof NativeEditor - */ - private async updateVersionInfoInNotebook(): Promise { - // Get our kernel_info and language_info from the current notebook - const notebook = this.getNotebook(); - - if (notebook) { - const interpreter = notebook.getMatchingInterpreter(); - const kernelSpec = notebook.getKernelSpec(); - - if ( - interpreter && - interpreter.version && - this.notebookJson.metadata && - this.notebookJson.metadata.language_info - ) { - this.notebookJson.metadata.language_info.version = interpreter.version.raw; - } - - if (kernelSpec && this.notebookJson.metadata && !this.notebookJson.metadata.kernelspec) { - // Add a new spec in this case - this.notebookJson.metadata.kernelspec = { - name: kernelSpec.name || kernelSpec.display_name || '', - display_name: kernelSpec.display_name || kernelSpec.name || '' - }; - } else if (kernelSpec && this.notebookJson.metadata && this.notebookJson.metadata.kernelspec) { - // Spec exists, just update name and display_name - this.notebookJson.metadata.kernelspec.name = kernelSpec.name || kernelSpec.display_name || ''; - this.notebookJson.metadata.kernelspec.display_name = kernelSpec.display_name || kernelSpec.name || ''; - } - } - } - - private async ensureNotebookJson(): Promise { - if (!this.notebookJson || !this.notebookJson.metadata) { - const pythonNumber = await this.extractPythonMainVersion(this.notebookJson); - // Use this to build our metadata object - // Use these as the defaults unless we have been given some in the options. - const metadata: nbformat.INotebookMetadata = { - language_info: { - name: 'python', - codemirror_mode: { - name: 'ipython', - version: pythonNumber - } - }, - orig_nbformat: 2, - file_extension: '.py', - mimetype: 'text/x-python', - name: 'python', - npconvert_exporter: 'python', - pygments_lexer: `ipython${pythonNumber}`, - version: pythonNumber - }; - - // Default notebook data. - this.notebookJson = { - nbformat: 4, - nbformat_minor: 2, - metadata: metadata - }; - } - } - - private async loadContents(contents: string | undefined, forceDirty: boolean): Promise { - // tslint:disable-next-line: no-any - const json = contents ? (JSON.parse(contents) as any) : undefined; - - // Double check json (if we have any) - if (json && !json.cells) { - throw new InvalidNotebookFileError(this.file.fsPath); - } - - // Then compute indent. It's computed from the contents - if (contents) { - this.indentAmount = detectIndent(contents).indent; - } - - // Then save the contents. We'll stick our cells back into this format when we save - if (json) { - this.notebookJson = json; - - // Log language or kernel telemetry - this.sendLanguageTelemetry(this.notebookJson); - } - this.contentsLoadedPromise.resolve(); - - // Extract cells from the json - const cells = contents - ? (json.cells as (nbformat.ICodeCell | nbformat.IRawCell | nbformat.IMarkdownCell)[]) - : []; - - // Then parse the cells - return this.loadCells( - cells.map((c, index) => { - return { - id: `NotebookImport#${index}`, - file: Identifiers.EmptyFileName, - line: 0, - state: CellState.finished, - data: c - }; - }), - forceDirty - ); - } - - private sendLanguageTelemetry(notebookJson: Partial) { - try { - // See if we have a language - let language = ''; - if (notebookJson.metadata?.language_info?.name) { - language = notebookJson.metadata?.language_info?.name; - } else if (notebookJson.metadata?.kernelspec?.language) { - language = notebookJson.metadata?.kernelspec?.language.toString(); - } - if (language && !KnownNotebookLanguages.includes(language.toLowerCase())) { - language = 'unknown'; - } - if (language) { - sendTelemetryEvent(Telemetry.NotebookLanguage, undefined, { language }); - } - } catch { - // If this fails, doesn't really matter - noop(); + private updateModel(change: NotebookModelChange) { + // Send to our model using a command. User has done something that changes the model + if (change.source === 'user' && this.model) { + // Note, originally this was posted with a command but sometimes had problems + // with commands being handled out of order. + this.model.update(change); } } - private async loadCells(cells: ICell[], forceDirty: boolean): Promise { - // Make sure cells have at least 1 - if (cells.length === 0) { - const defaultCell: ICell = { - id: uuid(), - line: 0, - file: Identifiers.EmptyFileName, - state: CellState.finished, - data: createCodeCell() - }; - cells.splice(0, 0, defaultCell); - forceDirty = true; - } - - // Save as our visible list - this.visibleCells = cells; - - // Make dirty if necessary - if (forceDirty) { - await this.setDirty(); - } + private async sendInitialCellsToWebView(cells: ICell[]): Promise { sendTelemetryEvent(Telemetry.CellCount, undefined, { count: cells.length }); return this.postMessage(InteractiveWindowMessages.LoadAllCells, { cells }); } - private getStorageKey(): string { - return `${KeyPrefix}${this._file.toString()}`; - } - /** - * Gets any unsaved changes to the notebook file. - * If the file has been modified since the uncommitted changes were stored, then ignore the uncommitted changes. - * - * @private - * @returns {(Promise)} - * @memberof NativeEditor - */ - private async getStoredContents(): Promise { - const key = this.getStorageKey(); - - // First look in the global storage file location - let result = await this.getStoredContentsFromFile(key); - if (!result) { - result = await this.getStoredContentsFromGlobalStorage(key); - if (!result) { - result = await this.getStoredContentsFromLocalStorage(key); - } - } - - return result; - } - - private async getStoredContentsFromFile(key: string): Promise { - const filePath = this.getHashedFileName(key); - try { - // Use this to read from the extension global location - const contents = await this.fileSystem.readFile(filePath); - const data = JSON.parse(contents); - // Check whether the file has been modified since the last time the contents were saved. - if (data && data.lastModifiedTimeMs && !this.isUntitled && this.file.scheme === 'file') { - const stat = await this.fileSystem.stat(this.file.fsPath); - if (stat.mtime > data.lastModifiedTimeMs) { - return; - } - } - if (data && !this.isUntitled && data.contents) { - return data.contents; - } - } catch { - noop(); - } - } - - private async getStoredContentsFromGlobalStorage(key: string): Promise { - try { - const data = this.globalStorage.get<{ contents?: string; lastModifiedTimeMs?: number }>(key); - - // If we have data here, make sure we eliminate any remnants of storage - if (data) { - await this.transferStorage(); - } - - // Check whether the file has been modified since the last time the contents were saved. - if (data && data.lastModifiedTimeMs && !this.isUntitled && this.file.scheme === 'file') { - const stat = await this.fileSystem.stat(this.file.fsPath); - if (stat.mtime > data.lastModifiedTimeMs) { - return; - } - } - if (data && !this.isUntitled && data.contents) { - return data.contents; - } - } catch { - noop(); - } - } - - private async getStoredContentsFromLocalStorage(key: string): Promise { - const workspaceData = this.localStorage.get(key); - if (workspaceData && !this.isUntitled) { - // Make sure to clear so we don't use this again. - this.localStorage.update(key, undefined); - - // Transfer this to a file so we use that next time instead. - const filePath = this.getHashedFileName(key); - await this.writeToStorage(filePath, workspaceData); - - return workspaceData; - } - } - - // VS code recommended we use the hidden '_values' to iterate over all of the entries in - // the global storage map and delete the ones we own. - private async transferStorage(): Promise { - const promises: Thenable[] = []; - - // Indicate we ran this function - await this.globalStorage.update(NotebookTransferKey, true); - - try { - // tslint:disable-next-line: no-any - if ((this.globalStorage as any)._value) { - // tslint:disable-next-line: no-any - const keys = Object.keys((this.globalStorage as any)._value); - [...keys].forEach((k: string) => { - if (k.startsWith(KeyPrefix)) { - // Write each pair to our alternate storage, but don't bother waiting for each - // to finish. - const filePath = this.getHashedFileName(k); - const contents = this.globalStorage.get(k); - if (contents) { - this.writeToStorage(filePath, JSON.stringify(contents)).ignoreErrors(); - } - - // Remove from the map so that global storage does not have this anymore. - // Use the real API here as we don't know how the map really gets updated. - promises.push(this.globalStorage.update(k, undefined)); - } - }); - } - } catch (e) { - traceError('Exception eliminating global storage parts:', e); - } - - return Promise.all(promises); - } - - /** - * Stores the uncommitted notebook changes into a temporary location. - * Also keep track of the current time. This way we can check whether changes were - * made to the file since the last time uncommitted changes were stored. - * - * @private - * @param {string} [contents] - * @returns {Promise} - * @memberof NativeEditor - */ - private async storeContents(contents?: string): Promise { - // Skip doing this if auto save is enabled. - const filesConfig = this.workspaceService.getConfiguration('files', this.file); - const autoSave = filesConfig.get('autoSave', 'off'); - if (autoSave === 'off') { - const key = this.getStorageKey(); - const filePath = this.getHashedFileName(key); - - // Keep track of the time when this data was saved. - // This way when we retrieve the data we can compare it against last modified date of the file. - const specialContents = contents ? JSON.stringify({ contents, lastModifiedTimeMs: Date.now() }) : undefined; - - // Write but debounced (wait at least 250 ms) - return this.debouncedWriteToStorage(filePath, specialContents); - } - } - - private async writeToStorage(filePath: string, contents?: string): Promise { - try { - if (!this._disposed) { - if (contents) { - await this.fileSystem.createDirectory(path.dirname(filePath)); - return this.fileSystem.writeFile(filePath, contents); - } else { - return this.fileSystem.deleteFile(filePath); - } - } - } catch (exc) { - traceError(`Error writing storage for ${filePath}: `, exc); - } - } - - private getHashedFileName(key: string): string { - const file = `${this.crypto.createHash(key, 'string')}.ipynb`; - return path.join(this.context.globalStoragePath, file); - } - - private async close(): Promise { - const actuallyClose = async () => { - // Tell listeners. - this.closedEvent.fire(this); - - // Restart our kernel so that execution counts are reset - let oldAsk: boolean | undefined = false; - const settings = this.configuration.getSettings(await this.getOwningResource()); - if (settings && settings.datascience) { - oldAsk = settings.datascience.askForKernelRestart; - settings.datascience.askForKernelRestart = false; - } - await this.restartKernel(); - if (oldAsk && settings && settings.datascience) { - settings.datascience.askForKernelRestart = true; - } - }; - - // Ask user if they want to save. It seems hotExit has no bearing on - // whether or not we should ask - if (this._dirty) { - const askResult = await this.askForSave(); - switch (askResult) { - case AskForSaveResult.Yes: - // Save the file - await this.saveToDisk(); - - // Close it - await actuallyClose(); - break; - - case AskForSaveResult.No: - // Mark as not dirty, so we update our storage - await this.setClean(); - - // Close it - await actuallyClose(); - break; - - default: - // Reopen - await this.reopen(this.visibleCells); - break; - } - } else { - // Not dirty, just close normally. - return actuallyClose(); - } - } - - private editCell(request: IEditCell) { - // Apply the changes to the visible cell list. We won't get an update until - // submission otherwise - if (request.changes && request.changes.length) { - const change = request.changes[0]; - const normalized = change.text.replace(/\r/g, ''); - - // Figure out which cell we're editing. - const cell = this.visibleCells.find(c => c.id === request.id); - if (cell) { - // This is an actual edit. - const contents = concatMultilineStringInput(cell.data.source); - const before = contents.substr(0, change.rangeOffset); - const after = contents.substr(change.rangeOffset + change.rangeLength); - const newContents = `${before}${normalized}${after}`; - if (contents !== newContents) { - cell.data.source = newContents; - this.setDirty().ignoreErrors(); - } - } - } - } - - private async insertCell(request: IInsertCell): Promise { - // Insert a cell into our visible list based on the index. They should be in sync - this.visibleCells.splice(request.index, 0, request.cell); - - return this.setDirty(); - } - - private async removeCell(request: IRemoveCell): Promise { - // Filter our list - this.visibleCells = this.visibleCells.filter(v => v.id !== request.id); - return this.setDirty(); - } - - private async swapCells(request: ISwapCells): Promise { - // Swap two cells in our list - const first = this.visibleCells.findIndex(v => v.id === request.firstCellId); - const second = this.visibleCells.findIndex(v => v.id === request.secondCellId); - if (first >= 0 && second >= 0) { - const temp = { ...this.visibleCells[first] }; - this.visibleCells[first] = this.visibleCells[second]; - this.visibleCells[second] = temp; - return this.setDirty(); - } - } - - private async askForSave(): Promise { - const message1 = localize.DataScience.dirtyNotebookMessage1().format(`${path.basename(this.file.fsPath)}`); - const message2 = localize.DataScience.dirtyNotebookMessage2(); - const yes = localize.DataScience.dirtyNotebookYes(); - const no = localize.DataScience.dirtyNotebookNo(); - const result = await this.applicationShell.showInformationMessage( - // tslint:disable-next-line: messages-must-be-localized - `${message1}\n${message2}`, - { modal: true }, - yes, - no - ); - switch (result) { - case yes: - return AskForSaveResult.Yes; - - case no: - return AskForSaveResult.No; - - default: - return AskForSaveResult.Cancel; - } - } - - private async setDirty(): Promise { - // Update storage if not untitled. Don't wait for results. - if (!this.isUntitled) { - this.generateNotebookContent(this.visibleCells) - .then(c => - this.storeContents(c).catch(ex => - traceError('Failed to generate notebook content to store in state', ex) - ) - ) - .ignoreErrors(); - } - - // Then update dirty flag. - if (!this._dirty) { - this._dirty = true; - this.setTitle(`${path.basename(this.file.fsPath)}*`); - - // Tell the webview we're dirty - await this.postMessage(InteractiveWindowMessages.NotebookDirty); - - // Tell listeners we're dirty - this.modifiedEvent.fire(this); - } - } - - private async setClean(): Promise { - // Always update storage - this.storeContents(undefined).catch(ex => traceError('Failed to clear notebook store', ex)); - - if (this._dirty) { - this._dirty = false; - this.setTitle(`${path.basename(this.file.fsPath)}`); - await this.postMessage(InteractiveWindowMessages.NotebookClean); - } - } - @captureTelemetry(Telemetry.ConvertToPythonFile, undefined, false) private async export(cells: ICell[]): Promise { const status = this.setStatus(localize.DataScience.convertingToPythonFile(), false); @@ -1141,9 +621,8 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { tempFile = await this.fileSystem.createTemporaryFile('.ipynb'); // Translate the cells into a notebook - await this.fileSystem.writeFile(tempFile.filePath, await this.generateNotebookContent(cells), { - encoding: 'utf-8' - }); + const content = this.model ? await this.model.getContent(cells) : ''; + await this.fileSystem.writeFile(tempFile.filePath, content, 'utf-8'); // Import this file and show it const contents = await this.importer.importFromFile(tempFile.filePath, this.file.fsPath); @@ -1165,99 +644,6 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { await this.documentManager.showTextDocument(doc, ViewColumn.One); } - private fixupCell(cell: nbformat.ICell): nbformat.ICell { - // Source is usually a single string on input. Convert back to an array - return ({ - ...cell, - source: splitMultilineString(cell.source) - // tslint:disable-next-line: no-any - } as any) as nbformat.ICell; // nyc (code coverage) barfs on this so just trick it. - } - - private async extractPythonMainVersion(notebookData: Partial): Promise { - if ( - notebookData && - notebookData.metadata && - notebookData.metadata.language_info && - notebookData.metadata.language_info.codemirror_mode && - // tslint:disable-next-line: no-any - typeof (notebookData.metadata.language_info.codemirror_mode as any).version === 'number' - ) { - // tslint:disable-next-line: no-any - return (notebookData.metadata.language_info.codemirror_mode as any).version; - } - // Use the active interpreter - const usableInterpreter = await this.jupyterExecution.getUsableJupyterPython(); - return usableInterpreter && usableInterpreter.version ? usableInterpreter.version.major : 3; - } - - private async generateNotebookContent(cells: ICell[]): Promise { - // Make sure we have some - await this.ensureNotebookJson(); - - // Reuse our original json except for the cells. - const json = { - ...(this.notebookJson as nbformat.INotebookContent), - cells: cells.map(c => this.fixupCell(c.data)) - }; - return JSON.stringify(json, null, this.indentAmount); - } - - @captureTelemetry(Telemetry.Save, undefined, true) - private async saveToDisk(): Promise { - // If we're already in the middle of prompting the user to save, then get out of here. - // We could add a debounce decorator, unfortunately that slows saving (by waiting for no more save events to get sent). - if (this.isPromptingToSaveToDisc && this.isUntitled) { - return; - } - try { - let fileToSaveTo: Uri | undefined = this.file; - let isDirty = this._dirty; - - // Ask user for a save as dialog if no title - if (this.isUntitled) { - this.isPromptingToSaveToDisc = true; - const filtersKey = localize.DataScience.dirtyNotebookDialogFilter(); - const filtersObject: { [name: string]: string[] } = {}; - filtersObject[filtersKey] = ['ipynb']; - isDirty = true; - - const defaultUri = - Array.isArray(this.workspaceService.workspaceFolders) && - this.workspaceService.workspaceFolders.length > 0 - ? this.workspaceService.workspaceFolders[0].uri - : undefined; - fileToSaveTo = await this.applicationShell.showSaveDialog({ - saveLabel: localize.DataScience.dirtyNotebookDialogTitle(), - filters: filtersObject, - defaultUri - }); - } - - if (fileToSaveTo && isDirty) { - // Write out our visible cells - await this.fileSystem.writeFile( - fileToSaveTo.fsPath, - await this.generateNotebookContent(this.visibleCells) - ); - - // Update our file name and dirty state - this._file = fileToSaveTo; - await this.setClean(); - this.savedEvent.fire(this); - } - } catch (e) { - traceError(e); - } finally { - this.isPromptingToSaveToDisc = false; - } - } - - private saveAll(args: ISaveAll) { - this.visibleCells = args.cells; - this.saveToDisk().ignoreErrors(); - } - private logNativeCommand(args: INativeCommand) { const telemetryEvent = args.source === 'mouse' @@ -1272,13 +658,4 @@ export class NativeEditor extends InteractiveBase implements INotebookEditor { sendTelemetryEvent(Telemetry.NotebookOpenTime, this.startupTimer.elapsedTime); } } - - private async clearAllOutputs() { - this.visibleCells.forEach(cell => { - cell.data.execution_count = null; - cell.data.outputs = []; - }); - - await this.setDirty(); - } } diff --git a/src/client/datascience/interactive-ipynb/nativeEditorCommandListener.ts b/src/client/datascience/interactive-ipynb/nativeEditorCommandListener.ts index c5ab4baa945c..f0125b346b34 100644 --- a/src/client/datascience/interactive-ipynb/nativeEditorCommandListener.ts +++ b/src/client/datascience/interactive-ipynb/nativeEditorCommandListener.ts @@ -8,7 +8,6 @@ import * as path from 'path'; import { Uri } from 'vscode'; import { ICommandManager } from '../../common/application/types'; -import { IFileSystem } from '../../common/platform/types'; import { IDisposableRegistry } from '../../common/types'; import { captureTelemetry } from '../../telemetry'; import { CommandSource } from '../../testing/common/constants'; @@ -20,8 +19,7 @@ export class NativeEditorCommandListener implements IDataScienceCommandListener constructor( @inject(IDisposableRegistry) private disposableRegistry: IDisposableRegistry, @inject(INotebookEditorProvider) private provider: INotebookEditorProvider, - @inject(IDataScienceErrorHandler) private dataScienceErrorHandler: IDataScienceErrorHandler, - @inject(IFileSystem) private fileSystem: IFileSystem + @inject(IDataScienceErrorHandler) private dataScienceErrorHandler: IDataScienceErrorHandler ) {} public register(commandManager: ICommandManager): void { @@ -117,9 +115,8 @@ export class NativeEditorCommandListener implements IDataScienceCommandListener private async openNotebook(file?: Uri): Promise { if (file && path.extname(file.fsPath).toLocaleLowerCase() === '.ipynb') { try { - const contents = await this.fileSystem.readFile(file.fsPath); // Then take the contents and load it. - await this.provider.open(file, contents); + await this.provider.open(file); } catch (e) { return this.dataScienceErrorHandler.handleError(e); } diff --git a/src/client/datascience/interactive-ipynb/nativeEditorOldWebView.ts b/src/client/datascience/interactive-ipynb/nativeEditorOldWebView.ts new file mode 100644 index 000000000000..b9842632f8b5 --- /dev/null +++ b/src/client/datascience/interactive-ipynb/nativeEditorOldWebView.ts @@ -0,0 +1,286 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. +'use strict'; +import '../../common/extensions'; + +import { inject, injectable, multiInject, named } from 'inversify'; +import * as path from 'path'; +import { Memento, Uri, WebviewPanel } from 'vscode'; + +import { + IApplicationShell, + ICommandManager, + IDocumentManager, + ILiveShareApi, + IWebPanelProvider, + IWorkspaceService +} from '../../common/application/types'; +import { traceError } from '../../common/logger'; +import { IFileSystem } from '../../common/platform/types'; +import { + GLOBAL_MEMENTO, + IAsyncDisposableRegistry, + IConfigurationService, + IDisposableRegistry, + IExperimentsManager, + IMemento +} from '../../common/types'; +import * as localize from '../../common/utils/localize'; +import { IInterpreterService } from '../../interpreter/contracts'; +import { captureTelemetry } from '../../telemetry'; +import { Commands, Telemetry } from '../constants'; +import { InteractiveWindowMessages } from '../interactive-common/interactiveWindowTypes'; +import { ProgressReporter } from '../progress/progressReporter'; +import { + ICodeCssGenerator, + IDataScienceErrorHandler, + IDataViewerProvider, + IInteractiveWindowListener, + IJupyterDebugger, + IJupyterExecution, + IJupyterVariables, + INotebookEditorProvider, + INotebookExporter, + INotebookImporter, + INotebookModel, + IStatusProvider, + IThemeFinder +} from '../types'; +import { NativeEditor } from './nativeEditor'; + +enum AskForSaveResult { + Yes, + No, + Cancel +} + +@injectable() +export class NativeEditorOldWebView extends NativeEditor { + public get visible(): boolean { + return this.viewState.visible; + } + public get active(): boolean { + return this.viewState.active; + } + + private isPromptingToSaveToDisc: boolean = false; + + constructor( + @multiInject(IInteractiveWindowListener) listeners: IInteractiveWindowListener[], + @inject(ILiveShareApi) liveShare: ILiveShareApi, + @inject(IApplicationShell) applicationShell: IApplicationShell, + @inject(IDocumentManager) documentManager: IDocumentManager, + @inject(IInterpreterService) interpreterService: IInterpreterService, + @inject(IWebPanelProvider) provider: IWebPanelProvider, + @inject(IDisposableRegistry) disposables: IDisposableRegistry, + @inject(ICodeCssGenerator) cssGenerator: ICodeCssGenerator, + @inject(IThemeFinder) themeFinder: IThemeFinder, + @inject(IStatusProvider) statusProvider: IStatusProvider, + @inject(IJupyterExecution) jupyterExecution: IJupyterExecution, + @inject(IFileSystem) fileSystem: IFileSystem, + @inject(IConfigurationService) configuration: IConfigurationService, + @inject(ICommandManager) commandManager: ICommandManager, + @inject(INotebookExporter) jupyterExporter: INotebookExporter, + @inject(IWorkspaceService) workspaceService: IWorkspaceService, + @inject(INotebookEditorProvider) editorProvider: INotebookEditorProvider, + @inject(IDataViewerProvider) dataExplorerProvider: IDataViewerProvider, + @inject(IJupyterVariables) jupyterVariables: IJupyterVariables, + @inject(IJupyterDebugger) jupyterDebugger: IJupyterDebugger, + @inject(INotebookImporter) importer: INotebookImporter, + @inject(IDataScienceErrorHandler) errorHandler: IDataScienceErrorHandler, + @inject(IMemento) @named(GLOBAL_MEMENTO) globalStorage: Memento, + @inject(ProgressReporter) progressReporter: ProgressReporter, + @inject(IExperimentsManager) experimentsManager: IExperimentsManager, + @inject(IAsyncDisposableRegistry) asyncRegistry: IAsyncDisposableRegistry + ) { + super( + listeners, + liveShare, + applicationShell, + documentManager, + interpreterService, + provider, + disposables, + cssGenerator, + themeFinder, + statusProvider, + jupyterExecution, + fileSystem, + configuration, + commandManager, + jupyterExporter, + workspaceService, + editorProvider, + dataExplorerProvider, + jupyterVariables, + jupyterDebugger, + importer, + errorHandler, + globalStorage, + progressReporter, + experimentsManager, + asyncRegistry + ); + asyncRegistry.push(this); + } + public async load(model: INotebookModel, webViewPanel: WebviewPanel): Promise { + await super.load(model, webViewPanel); + + // Update our title to match + this.setTitle(path.basename(model.file.fsPath)); + + // Show ourselves + await this.show(); + this.model?.changed(() => { + if (this.model?.isDirty) { + this.setDirty().ignoreErrors(); + } else { + this.setClean().ignoreErrors(); + } + }); + } + protected async close(): Promise { + const actuallyClose = async () => { + // Tell listeners. + this.closedEvent.fire(this); + + // Restart our kernel so that execution counts are reset + let oldAsk: boolean | undefined = false; + const settings = this.configuration.getSettings(await this.getOwningResource()); + if (settings && settings.datascience) { + oldAsk = settings.datascience.askForKernelRestart; + settings.datascience.askForKernelRestart = false; + } + await this.restartKernel(); + if (oldAsk && settings && settings.datascience) { + settings.datascience.askForKernelRestart = true; + } + }; + + // Ask user if they want to save. It seems hotExit has no bearing on + // whether or not we should ask + if (this.isDirty) { + const askResult = await this.askForSave(); + switch (askResult) { + case AskForSaveResult.Yes: + // Save the file + await this.saveToDisk(); + + // Close it + await actuallyClose(); + break; + + case AskForSaveResult.No: + // Close it + await actuallyClose(); + break; + + default: + // Reopen + await this.reopen(); + break; + } + } else { + // Not dirty, just close normally. + return actuallyClose(); + } + } + + protected saveAll() { + this.saveToDisk().ignoreErrors(); + } + + private async reopen(): Promise { + // TODO: Fire command to open an nb. + } + + private async askForSave(): Promise { + const message1 = localize.DataScience.dirtyNotebookMessage1().format(`${path.basename(this.file.fsPath)}`); + const message2 = localize.DataScience.dirtyNotebookMessage2(); + const yes = localize.DataScience.dirtyNotebookYes(); + const no = localize.DataScience.dirtyNotebookNo(); + const result = await this.applicationShell.showInformationMessage( + // tslint:disable-next-line: messages-must-be-localized + `${message1}\n${message2}`, + { modal: true }, + yes, + no + ); + switch (result) { + case yes: + return AskForSaveResult.Yes; + + case no: + return AskForSaveResult.No; + + default: + return AskForSaveResult.Cancel; + } + } + private async setDirty(): Promise { + // Then update dirty flag. + if (this.isDirty) { + this.setTitle(`${path.basename(this.file.fsPath)}*`); + + // Tell the webview we're dirty + await this.postMessage(InteractiveWindowMessages.NotebookDirty); + + // Tell listeners we're dirty + this.modifiedEvent.fire(this); + } + } + + private async setClean(): Promise { + if (!this.isDirty) { + this.setTitle(`${path.basename(this.file.fsPath)}`); + await this.postMessage(InteractiveWindowMessages.NotebookClean); + } + } + + @captureTelemetry(Telemetry.Save, undefined, true) + private async saveToDisk(): Promise { + // If we're already in the middle of prompting the user to save, then get out of here. + // We could add a debounce decorator, unfortunately that slows saving (by waiting for no more save events to get sent). + if ((this.isPromptingToSaveToDisc && this.isUntitled) || !this.model) { + return; + } + try { + if (!this.isUntitled) { + await this.commandManager.executeCommand(Commands.SaveNotebookNonCustomEditor, this.model?.file); + this.savedEvent.fire(this); + return; + } + // Ask user for a save as dialog if no title + let fileToSaveTo: Uri | undefined = this.file; + + this.isPromptingToSaveToDisc = true; + const filtersKey = localize.DataScience.dirtyNotebookDialogFilter(); + const filtersObject: { [name: string]: string[] } = {}; + filtersObject[filtersKey] = ['ipynb']; + + const defaultUri = + Array.isArray(this.workspaceService.workspaceFolders) && + this.workspaceService.workspaceFolders.length > 0 + ? this.workspaceService.workspaceFolders[0].uri + : undefined; + fileToSaveTo = await this.applicationShell.showSaveDialog({ + saveLabel: localize.DataScience.dirtyNotebookDialogTitle(), + filters: filtersObject, + defaultUri + }); + + if (fileToSaveTo) { + await this.commandManager.executeCommand( + Commands.SaveAsNotebookNonCustomEditor, + this.model.file, + fileToSaveTo + ); + this.savedEvent.fire(this); + } + } catch (e) { + traceError('Failed to Save nb', e); + } finally { + this.isPromptingToSaveToDisc = false; + } + } +} diff --git a/src/client/datascience/interactive-ipynb/nativeEditorProvider.ts b/src/client/datascience/interactive-ipynb/nativeEditorProvider.ts index 14430d5d1133..2d5c59ca633f 100644 --- a/src/client/datascience/interactive-ipynb/nativeEditorProvider.ts +++ b/src/client/datascience/interactive-ipynb/nativeEditorProvider.ts @@ -2,12 +2,16 @@ // Licensed under the MIT License. 'use strict'; import { inject, injectable } from 'inversify'; -import * as path from 'path'; -import { Event, EventEmitter, TextDocument, TextEditor, Uri } from 'vscode'; - -import { ICommandManager, IDocumentManager, IWorkspaceService } from '../../common/application/types'; -import { JUPYTER_LANGUAGE } from '../../common/constants'; -import { IFileSystem } from '../../common/platform/types'; +import * as uuid from 'uuid/v4'; +import { Disposable, Event, EventEmitter, Uri, WebviewPanel } from 'vscode'; +import { arePathsSame } from '../../../datascience-ui/react-common/arePathsSame'; +import { + ICustomEditorService, + IWorkspaceService, + WebviewCustomEditorEditingDelegate, + WebviewCustomEditorProvider +} from '../../common/application/types'; +import { traceInfo } from '../../common/logger'; import { IAsyncDisposable, IAsyncDisposableRegistry, @@ -15,70 +19,124 @@ import { IDisposableRegistry, Resource } from '../../common/types'; +import { createDeferred } from '../../common/utils/async'; import * as localize from '../../common/utils/localize'; import { IServiceContainer } from '../../ioc/types'; import { captureTelemetry, sendTelemetryEvent } from '../../telemetry'; import { Identifiers, Settings, Telemetry } from '../constants'; -import { IDataScienceErrorHandler, INotebookEditor, INotebookEditorProvider, INotebookServerOptions } from '../types'; - +import { NotebookModelChange } from '../interactive-common/interactiveWindowTypes'; +import { + INotebookEditor, + INotebookEditorProvider, + INotebookModel, + INotebookServerOptions, + INotebookStorage +} from '../types'; + +// Class that is registered as the custom editor provider for notebooks. VS code will call into this class when +// opening an ipynb file. This class then creates a backing storage, model, and opens a view for the file. @injectable() -export class NativeEditorProvider implements INotebookEditorProvider, IAsyncDisposable { +export class NativeEditorProvider + implements + INotebookEditorProvider, + WebviewCustomEditorProvider, + WebviewCustomEditorEditingDelegate, + IAsyncDisposable { public get onDidChangeActiveNotebookEditor(): Event { return this._onDidChangeActiveNotebookEditor.event; } - private readonly _onDidChangeActiveNotebookEditor = new EventEmitter(); - private activeEditors: Map = new Map(); - private executedEditors: Set = new Set(); - private _onDidOpenNotebookEditor = new EventEmitter(); - private notebookCount: number = 0; - private openedNotebookCount: number = 0; - private nextNumber: number = 1; + public get onDidCloseNotebookEditor(): Event { + return this._onDidCloseNotebookEditor.event; + } + public get onEdit(): Event<{ readonly resource: Uri; readonly edit: NotebookModelChange }> { + return this._editEventEmitter.event; + } + + public get editingDelegate(): WebviewCustomEditorEditingDelegate | undefined { + return this; + } + public get onDidOpenNotebookEditor(): Event { return this._onDidOpenNotebookEditor.event; } + public get activeEditor(): INotebookEditor | undefined { + return this.editors.find(e => e.visible && e.active); + } + + public get editors(): INotebookEditor[] { + return [...this.openedEditors]; + } + // Note, this constant has to match the value used in the package.json to register the webview custom editor. + public static readonly customEditorViewType = 'NativeEditorProvider.ipynb'; + protected readonly _onDidChangeActiveNotebookEditor = new EventEmitter(); + protected readonly _onDidOpenNotebookEditor = new EventEmitter(); + private readonly _onDidCloseNotebookEditor = new EventEmitter(); + private readonly _editEventEmitter = new EventEmitter<{ + readonly resource: Uri; + readonly edit: NotebookModelChange; + }>(); + private openedEditors: Set = new Set(); + private models = new Map>(); + private modelChangedHandlers: Map = new Map(); + private executedEditors: Set = new Set(); + private notebookCount: number = 0; + private openedNotebookCount: number = 0; + private _id = uuid(); constructor( - @inject(IServiceContainer) private serviceContainer: IServiceContainer, - @inject(IAsyncDisposableRegistry) asyncRegistry: IAsyncDisposableRegistry, - @inject(IDisposableRegistry) private disposables: IDisposableRegistry, - @inject(IWorkspaceService) private workspace: IWorkspaceService, - @inject(IConfigurationService) private configuration: IConfigurationService, - @inject(IFileSystem) private fileSystem: IFileSystem, - @inject(IDocumentManager) private documentManager: IDocumentManager, - @inject(ICommandManager) private readonly cmdManager: ICommandManager, - @inject(IDataScienceErrorHandler) private dataScienceErrorHandler: IDataScienceErrorHandler + @inject(IServiceContainer) protected readonly serviceContainer: IServiceContainer, + @inject(IAsyncDisposableRegistry) protected readonly asyncRegistry: IAsyncDisposableRegistry, + @inject(IDisposableRegistry) protected readonly disposables: IDisposableRegistry, + @inject(IWorkspaceService) protected readonly workspace: IWorkspaceService, + @inject(IConfigurationService) protected readonly configuration: IConfigurationService, + @inject(ICustomEditorService) private customEditorService: ICustomEditorService ) { + traceInfo(`id is ${this._id}`); asyncRegistry.push(this); - // No live share sync required as open document from vscode will give us our contents. - // Look through the file system for ipynb files to see how many we have in the workspace. Don't wait // on this though. - const findFilesPromise = this.workspace.findFiles('**/*.ipynb'); + const findFilesPromise = workspace.findFiles('**/*.ipynb'); if (findFilesPromise && findFilesPromise.then) { findFilesPromise.then(r => (this.notebookCount += r.length)); } - this.disposables.push( - this.documentManager.onDidChangeActiveTextEditor(this.onDidChangeActiveTextEditorHandler.bind(this)) - ); + // Register for the custom editor service. + customEditorService.registerWebviewCustomEditorProvider(NativeEditorProvider.customEditorViewType, this, { + enableFindWidget: true, + retainContextWhenHidden: true + }); + } - // Since we may have activated after a document was opened, also run open document for all documents. - // This needs to be async though. Iterating over all of these in the .ctor is crashing the extension - // host, so postpone till after the ctor is finished. - setTimeout(() => { - if (this.documentManager.textDocuments && this.documentManager.textDocuments.forEach) { - this.documentManager.textDocuments.forEach(doc => this.openNotebookAndCloseEditor(doc, false)); + public save(resource: Uri): Promise { + return this.loadStorage(resource).then(async s => { + if (s) { + await s.save(); + } + }); + } + public saveAs(resource: Uri, targetResource: Uri): Promise { + return this.loadStorage(resource).then(async s => { + if (s) { + await s.saveAs(targetResource); + } + }); + } + public applyEdits(resource: Uri, edits: readonly NotebookModelChange[]): Promise { + return this.loadModel(resource).then(s => { + if (s) { + edits.forEach(e => s.update({ ...e, source: 'redo' })); + } + }); + } + public undoEdits(resource: Uri, edits: readonly NotebookModelChange[]): Promise { + return this.loadModel(resource).then(s => { + if (s) { + edits.forEach(e => s.update({ ...e, source: 'undo' })); } - }, 0); - - // // Reopen our list of files that were open during shutdown. Actually not doing this for now. The files - // don't open until the extension loads and all they all steal focus. - // const uriList = this.workspaceStorage.get(NotebookUriListStorageKey); - // if (uriList && uriList.length) { - // uriList.forEach(u => { - // this.fileSystem.readFile(u.fsPath).then(c => this.open(u, c).ignoreErrors()).ignoreErrors(); - // }); - // } + }); + } + public async resolveWebviewEditor(resource: Uri, panel: WebviewPanel) { + await this.createNotebookEditor(resource, panel); } public async dispose(): Promise { @@ -93,48 +151,47 @@ export class NativeEditorProvider implements INotebookEditorProvider, IAsyncDisp sendTelemetryEvent(Telemetry.NotebookWorkspaceCount, undefined, { count: this.notebookCount }); } } - public get activeEditor(): INotebookEditor | undefined { - const active = [...this.activeEditors.entries()].find(e => e[1].active); - if (active) { - return active[1]; - } - } - public get editors(): INotebookEditor[] { - return [...this.activeEditors.values()]; - } + public async open(file: Uri): Promise { + // Create a deferred promise that will fire when the notebook + // actually opens + const deferred = createDeferred(); + + // Sign up for open event once it does open + let disposable: Disposable | undefined; + const handler = (e: INotebookEditor) => { + if (arePathsSame(e.file.fsPath, file.fsPath)) { + if (disposable) { + disposable.dispose(); + } + deferred.resolve(e); + } + }; + disposable = this._onDidOpenNotebookEditor.event(handler); - public async open(file: Uri, contents: string): Promise { - // See if this file is open or not already - let editor = this.activeEditors.get(file.fsPath); - if (!editor) { - editor = await this.create(file, contents); - this.onOpenedEditor(editor); - } else { - await this.showEditor(editor); - } - return editor; + // Send an open command. + this.customEditorService.openEditor(file).ignoreErrors(); + + // Promise should resolve when the file opens. + return deferred.promise; } public async show(file: Uri): Promise { - // See if this file is open or not already - const editor = this.activeEditors.get(file.fsPath); - if (editor) { - await this.showEditor(editor); - } - return editor; + return this.open(file); } @captureTelemetry(Telemetry.CreateNewNotebook, undefined, false) public async createNew(contents?: string): Promise { // Create a new URI for the dummy file using our root workspace path const uri = await this.getNextNewNotebookUri(); + + // Update number of notebooks in the workspace this.notebookCount += 1; - if (contents) { - return this.open(uri, contents); - } else { - return this.open(uri, ''); - } + + // Set these contents into the storage before the file opens + await this.loadStorage(uri, contents); + + return this.open(uri); } public async getNotebookOptions(resource: Resource): Promise { @@ -154,181 +211,115 @@ export class NativeEditorProvider implements INotebookEditorProvider, IAsyncDisp purpose: Identifiers.HistoryPurpose // Share the same one as the interactive window. Just need a new session }; } - - /** - * Open ipynb files when user opens an ipynb file. - * - * @private - * @memberof NativeEditorProvider - */ - private onDidChangeActiveTextEditorHandler(editor?: TextEditor) { - // I we're a source control diff view, then ignore this editor. - if (!editor || this.isEditorPartOfDiffView(editor)) { - return; + protected async createNotebookEditor(resource: Uri, panel?: WebviewPanel) { + try { + // Get the model + const model = await this.loadModel(resource); + + // Create a new editor + const editor = this.serviceContainer.get(INotebookEditor); + + // Load it (should already be visible) + return editor + .load(model, panel) + .then(() => this.openedEditor(editor)) + .then(() => editor); + } catch (exc) { + // Send telemetry indicating a failure + sendTelemetryEvent(Telemetry.OpenNotebookFailure); + throw exc; } - this.openNotebookAndCloseEditor(editor.document, true).ignoreErrors(); } - private async showEditor(editor: INotebookEditor) { - await editor.show(); - this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); + protected openedEditor(editor: INotebookEditor): void { + this.openedNotebookCount += 1; + if (!this.executedEditors.has(editor.file.fsPath)) { + editor.executed(this.onExecuted.bind(this)); + } + this.disposables.push(editor.onDidChangeViewState(this.onChangedViewState, this)); + this.openedEditors.add(editor); + editor.closed(this.closedEditor.bind(this)); + this._onDidOpenNotebookEditor.fire(editor); } - private async create(file: Uri, contents: string): Promise { - const editor = this.serviceContainer.get(INotebookEditor); - await editor.load(contents, file); - this.disposables.push(editor.closed(this.onClosedEditor.bind(this))); - this.disposables.push(editor.executed(this.onExecutedEditor.bind(this))); - await this.showEditor(editor); - return editor; + private closedEditor(editor: INotebookEditor): void { + this.openedEditors.delete(editor); + // If last editor, dispose of the storage + const key = editor.file.toString(); + if (![...this.openedEditors].find(e => e.file.toString() === key)) { + this.modelChangedHandlers.delete(key); + this.models.delete(key); + } + this._onDidCloseNotebookEditor.fire(editor); } - private onClosedEditor(e: INotebookEditor) { - this.activeEditors.delete(e.file.fsPath); + private onChangedViewState(): void { this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); } - private onExecutedEditor(e: INotebookEditor) { - this.executedEditors.add(e.file.fsPath); - } - - private onOpenedEditor(e: INotebookEditor) { - this.activeEditors.set(e.file.fsPath, e); - this.disposables.push(e.saved(this.onSavedEditor.bind(this, e.file.fsPath))); - this.openedNotebookCount += 1; - this._onDidOpenNotebookEditor.fire(e); - this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); - this.disposables.push(e.onDidChangeViewState(this.onDidChangeViewState, this)); - } - private onDidChangeViewState() { - this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); - } - - private onSavedEditor(oldPath: string, e: INotebookEditor) { - // Switch our key for this editor - if (this.activeEditors.has(oldPath)) { - this.activeEditors.delete(oldPath); + private onExecuted(editor: INotebookEditor): void { + if (editor) { + this.executedEditors.add(editor.file.fsPath); } - this.activeEditors.set(e.file.fsPath, e); } - private async getNextNewNotebookUri(): Promise { - // Start in the root and look for files starting with untitled - let number = 1; - const dir = this.workspace.rootPath; - if (dir) { - const existing = await this.fileSystem.search( - path.join(dir, `${localize.DataScience.untitledNotebookFileName()}-*.ipynb`) - ); - - // Sort by number - existing.sort(); - - // Add one onto the end of the last one - if (existing.length > 0) { - const match = /(\w+)-(\d+)\.ipynb/.exec(path.basename(existing[existing.length - 1])); - if (match && match.length > 1) { - number = parseInt(match[2], 10); - } - return Uri.file(path.join(dir, `${localize.DataScience.untitledNotebookFileName()}-${number + 1}`)); + private async modelChanged(file: Uri, change: NotebookModelChange): Promise { + // If the cells change because of a UI event, tell VS code about it + if (change.source === 'user') { + // Skip version and file changes. They can't be undone + switch (change.kind) { + case 'version': + break; + case 'file': + // Update our storage + const promise = this.models.get(change.oldFile.toString()); + this.models.delete(change.oldFile.toString()); + this.models.set(change.newFile.toString(), promise!); + break; + default: + this._editEventEmitter.fire({ resource: file, edit: change }); + break; } } - - const result = Uri.file(`${localize.DataScience.untitledNotebookFileName()}-${this.nextNumber}`); - this.nextNumber += 1; - return result; } - private openNotebookAndCloseEditor = async ( - document: TextDocument, - closeDocumentBeforeOpeningNotebook: boolean - ) => { - // See if this is an ipynb file - if (this.isNotebook(document) && this.configuration.getSettings(document.uri).datascience.useNotebookEditor) { - const closeActiveEditorCommand = 'workbench.action.closeActiveEditor'; - try { - const contents = document.getText(); - const uri = document.uri; - - if (closeDocumentBeforeOpeningNotebook) { - if ( - !this.documentManager.activeTextEditor || - this.documentManager.activeTextEditor.document !== document - ) { - await this.documentManager.showTextDocument(document); - } - await this.cmdManager.executeCommand(closeActiveEditorCommand); - } + private async loadModel(file: Uri, contents?: string): Promise { + const modelAndStorage = await this.loadModelAndStorage(file, contents); + return modelAndStorage.model; + } - // Open our own editor. - await this.open(uri, contents); + private async loadStorage(file: Uri, contents?: string): Promise { + const modelAndStorage = await this.loadModelAndStorage(file, contents); + return modelAndStorage.storage; + } - if (!closeDocumentBeforeOpeningNotebook) { - // Then switch back to the ipynb and close it. - // If we don't do it in this order, the close will switch to the wrong item - await this.documentManager.showTextDocument(document); - await this.cmdManager.executeCommand(closeActiveEditorCommand); + private loadModelAndStorage(file: Uri, contents?: string) { + const key = file.toString(); + let modelPromise = this.models.get(key); + if (!modelPromise) { + const storage = this.serviceContainer.get(INotebookStorage); + modelPromise = storage.load(file, contents).then(m => { + if (!this.modelChangedHandlers.has(key)) { + this.modelChangedHandlers.set(key, m.changed(this.modelChanged.bind(this, file))); } - } catch (e) { - return this.dataScienceErrorHandler.handleError(e); - } - } - }; - /** - * Check if user is attempting to compare two ipynb files. - * If yes, then return `true`, else `false`. - * - * @private - * @param {TextEditor} editor - * @memberof NativeEditorProvider - */ - private isEditorPartOfDiffView(editor?: TextEditor) { - if (!editor) { - return false; - } - // There's no easy way to determine if the user is openeing a diff view. - // One simple way is to check if there are 2 editor opened, and if both editors point to the same file - // One file with the `file` scheme and the other with the `git` scheme. - if (this.documentManager.visibleTextEditors.length <= 1) { - return false; - } + return { model: m, storage }; + }); - // If we have both `git` & `file` schemes for the same file, then we're most likely looking at a diff view. - // Also ensure both editors are in the same view column. - // Possible we have a git diff view (with two editors git and file scheme), and we open the file view - // on the side (different view column). - const gitSchemeEditor = this.documentManager.visibleTextEditors.find( - editorUri => - editorUri.document.uri.scheme === 'git' && - this.fileSystem.arePathsSame(editorUri.document.uri.fsPath, editor.document.uri.fsPath) - ); - - if (!gitSchemeEditor) { - return false; + this.models.set(key, modelPromise); } + return modelPromise; + } - const fileSchemeEditor = this.documentManager.visibleTextEditors.find( - editorUri => - editorUri.document.uri.scheme === 'file' && - this.fileSystem.arePathsSame(editorUri.document.uri.fsPath, editor.document.uri.fsPath) && - editorUri.viewColumn === gitSchemeEditor.viewColumn - ); - if (!fileSchemeEditor) { - return false; - } + private async getNextNewNotebookUri(): Promise { + // See if we have any untitled storage already + const untitledStorage = [...this.models.keys()].filter(k => Uri.parse(k).scheme === 'untitled'); - // Also confirm the document we have passed in, belongs to one of the editors. - // If its not, then its another document (that is not in the diff view). - return gitSchemeEditor === editor || fileSchemeEditor === editor; - } - private isNotebook(document: TextDocument) { - // Only support file uris (we don't want to automatically open any other ipynb file from another resource as a notebook). - // E.g. when opening a document for comparison, the scheme is `git`, in live share the scheme is `vsls`. - const validUriScheme = document.uri.scheme === 'file' || document.uri.scheme === 'vsls'; - return ( - validUriScheme && - (document.languageId === JUPYTER_LANGUAGE || - path.extname(document.fileName).toLocaleLowerCase() === '.ipynb') - ); + // Just use the length (don't bother trying to fill in holes). We never remove storage objects from + // our map, so we'll keep creating new untitled notebooks. + const fileName = `${localize.DataScience.untitledNotebookFileName()}-${untitledStorage.length + 1}.ipynb`; + const fileUri = Uri.file(fileName); + + // Turn this back into an untitled + return fileUri.with({ scheme: 'untitled', path: fileName }); } } diff --git a/src/client/datascience/interactive-ipynb/nativeEditorProviderOld.ts b/src/client/datascience/interactive-ipynb/nativeEditorProviderOld.ts new file mode 100644 index 000000000000..ce2d9da2a795 --- /dev/null +++ b/src/client/datascience/interactive-ipynb/nativeEditorProviderOld.ts @@ -0,0 +1,262 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. +'use strict'; +import { inject, injectable } from 'inversify'; +import * as path from 'path'; +import { TextDocument, TextEditor, Uri } from 'vscode'; + +import { + ICommandManager, + ICustomEditorService, + IDocumentManager, + IWorkspaceService +} from '../../common/application/types'; +import { JUPYTER_LANGUAGE } from '../../common/constants'; +import { IFileSystem } from '../../common/platform/types'; +import { IAsyncDisposableRegistry, IConfigurationService, IDisposableRegistry, Resource } from '../../common/types'; +import { IServiceContainer } from '../../ioc/types'; +import { Commands, Identifiers, Settings } from '../constants'; +import { IDataScienceErrorHandler, INotebookEditor, INotebookServerOptions } from '../types'; +import { NativeEditorProvider } from './nativeEditorProvider'; + +@injectable() +export class NativeEditorProviderOld extends NativeEditorProvider { + public get activeEditor(): INotebookEditor | undefined { + const active = [...this.activeEditors.entries()].find(e => e[1].active); + if (active) { + return active[1]; + } + } + + public get editors(): INotebookEditor[] { + return [...this.activeEditors.values()]; + } + private activeEditors: Map = new Map(); + constructor( + @inject(IServiceContainer) serviceContainer: IServiceContainer, + @inject(IAsyncDisposableRegistry) asyncRegistry: IAsyncDisposableRegistry, + @inject(IDisposableRegistry) disposables: IDisposableRegistry, + @inject(IWorkspaceService) workspace: IWorkspaceService, + @inject(IConfigurationService) configuration: IConfigurationService, + @inject(ICustomEditorService) customEditorService: ICustomEditorService, + @inject(IFileSystem) private fileSystem: IFileSystem, + @inject(IDocumentManager) private documentManager: IDocumentManager, + @inject(ICommandManager) private readonly cmdManager: ICommandManager, + @inject(IDataScienceErrorHandler) private dataScienceErrorHandler: IDataScienceErrorHandler + ) { + super(serviceContainer, asyncRegistry, disposables, workspace, configuration, customEditorService); + + // No live share sync required as open document from vscode will give us our contents. + + this.disposables.push( + this.documentManager.onDidChangeActiveTextEditor(this.onDidChangeActiveTextEditorHandler.bind(this)) + ); + this.disposables.push( + this.cmdManager.registerCommand(Commands.SaveNotebookNonCustomEditor, async (resource: Uri) => { + await this.save(resource); + }) + ); + this.disposables.push( + this.cmdManager.registerCommand( + Commands.SaveAsNotebookNonCustomEditor, + async (resource: Uri, targetResource: Uri) => { + await this.saveAs(resource, targetResource); + } + ) + ); + + // // Since we may have activated after a document was opened, also run open document for all documents. + // // This needs to be async though. Iterating over all of these in the .ctor is crashing the extension + // // host, so postpone till after the ctor is finished. + // setTimeout(() => { + // if (this.documentManager.textDocuments && this.documentManager.textDocuments.forEach) { + // this.documentManager.textDocuments.forEach(doc => this.openNotebookAndCloseEditor(doc, false)); + // } + // }, 0); + + // // Reopen our list of files that were open during shutdown. Actually not doing this for now. The files + // don't open until the extension loads and all they all steal focus. + // const uriList = this.workspaceStorage.get(NotebookUriListStorageKey); + // if (uriList && uriList.length) { + // uriList.forEach(u => { + // this.fileSystem.readFile(u.fsPath).then(c => this.open(u, c).ignoreErrors()).ignoreErrors(); + // }); + // } + } + + public async open(file: Uri): Promise { + // See if this file is open or not already + let editor = this.activeEditors.get(file.fsPath); + if (!editor) { + editor = await this.create(file); + this.onOpenedEditor(editor); + } else { + await this.showEditor(editor); + } + return editor; + } + + public async show(file: Uri): Promise { + // See if this file is open or not already + const editor = this.activeEditors.get(file.fsPath); + if (editor) { + await this.showEditor(editor); + } + return editor; + } + + public async getNotebookOptions(resource: Resource): Promise { + const settings = this.configuration.getSettings(resource); + let serverURI: string | undefined = settings.datascience.jupyterServerURI; + const useDefaultConfig: boolean | undefined = settings.datascience.useDefaultConfigForJupyter; + + // For the local case pass in our URI as undefined, that way connect doesn't have to check the setting + if (serverURI.toLowerCase() === Settings.JupyterServerLocalLaunch) { + serverURI = undefined; + } + + return { + enableDebugging: true, + uri: serverURI, + useDefaultConfig, + purpose: Identifiers.HistoryPurpose // Share the same one as the interactive window. Just need a new session + }; + } + + protected onOpenedEditor(e: INotebookEditor) { + super.openedEditor(e); + this.activeEditors.set(e.file.fsPath, e); + this.disposables.push(e.saved(this.onSavedEditor.bind(this, e.file.fsPath))); + this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); + } + + /** + * Open ipynb files when user opens an ipynb file. + * + * @private + * @memberof NativeEditorProvider + */ + private onDidChangeActiveTextEditorHandler(editor?: TextEditor) { + // I we're a source control diff view, then ignore this editor. + if (!editor || this.isEditorPartOfDiffView(editor)) { + return; + } + this.openNotebookAndCloseEditor(editor.document, true).ignoreErrors(); + } + + private async showEditor(editor: INotebookEditor) { + await editor.show(); + this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); + } + + private async create(file: Uri): Promise { + const editor = await this.createNotebookEditor(file); + this.disposables.push(editor.closed(this.onClosedEditor.bind(this))); + await this.showEditor(editor); + return editor; + } + + private onClosedEditor(e: INotebookEditor) { + this.activeEditors.delete(e.file.fsPath); + this._onDidChangeActiveNotebookEditor.fire(this.activeEditor); + } + private onSavedEditor(oldPath: string, e: INotebookEditor) { + // Switch our key for this editor + if (this.activeEditors.has(oldPath)) { + this.activeEditors.delete(oldPath); + } + this.activeEditors.set(e.file.fsPath, e); + } + + private openNotebookAndCloseEditor = async ( + document: TextDocument, + closeDocumentBeforeOpeningNotebook: boolean + ) => { + // See if this is an ipynb file + if (this.isNotebook(document) && this.configuration.getSettings(document.uri).datascience.useNotebookEditor) { + const closeActiveEditorCommand = 'workbench.action.closeActiveEditor'; + try { + const uri = document.uri; + + if (closeDocumentBeforeOpeningNotebook) { + if ( + !this.documentManager.activeTextEditor || + this.documentManager.activeTextEditor.document !== document + ) { + await this.documentManager.showTextDocument(document); + } + await this.cmdManager.executeCommand(closeActiveEditorCommand); + } + + // Open our own editor. + await this.open(uri); + + if (!closeDocumentBeforeOpeningNotebook) { + // Then switch back to the ipynb and close it. + // If we don't do it in this order, the close will switch to the wrong item + await this.documentManager.showTextDocument(document); + await this.cmdManager.executeCommand(closeActiveEditorCommand); + } + } catch (e) { + return this.dataScienceErrorHandler.handleError(e); + } + } + }; + /** + * Check if user is attempting to compare two ipynb files. + * If yes, then return `true`, else `false`. + * + * @private + * @param {TextEditor} editor + * @memberof NativeEditorProvider + */ + private isEditorPartOfDiffView(editor?: TextEditor) { + if (!editor) { + return false; + } + // There's no easy way to determine if the user is openeing a diff view. + // One simple way is to check if there are 2 editor opened, and if both editors point to the same file + // One file with the `file` scheme and the other with the `git` scheme. + if (this.documentManager.visibleTextEditors.length <= 1) { + return false; + } + + // If we have both `git` & `file` schemes for the same file, then we're most likely looking at a diff view. + // Also ensure both editors are in the same view column. + // Possible we have a git diff view (with two editors git and file scheme), and we open the file view + // on the side (different view column). + const gitSchemeEditor = this.documentManager.visibleTextEditors.find( + editorUri => + editorUri.document.uri.scheme === 'git' && + this.fileSystem.arePathsSame(editorUri.document.uri.fsPath, editor.document.uri.fsPath) + ); + + if (!gitSchemeEditor) { + return false; + } + + const fileSchemeEditor = this.documentManager.visibleTextEditors.find( + editorUri => + editorUri.document.uri.scheme === 'file' && + this.fileSystem.arePathsSame(editorUri.document.uri.fsPath, editor.document.uri.fsPath) && + editorUri.viewColumn === gitSchemeEditor.viewColumn + ); + if (!fileSchemeEditor) { + return false; + } + + // Also confirm the document we have passed in, belongs to one of the editors. + // If its not, then its another document (that is not in the diff view). + return gitSchemeEditor === editor || fileSchemeEditor === editor; + } + private isNotebook(document: TextDocument) { + // Only support file uris (we don't want to automatically open any other ipynb file from another resource as a notebook). + // E.g. when opening a document for comparison, the scheme is `git`, in live share the scheme is `vsls`. + const validUriScheme = document.uri.scheme === 'file' || document.uri.scheme === 'vsls'; + return ( + validUriScheme && + (document.languageId === JUPYTER_LANGUAGE || + path.extname(document.fileName).toLocaleLowerCase() === '.ipynb') + ); + } +} diff --git a/src/client/datascience/interactive-ipynb/nativeEditorStorage.ts b/src/client/datascience/interactive-ipynb/nativeEditorStorage.ts new file mode 100644 index 000000000000..ff6ab8fa6e1e --- /dev/null +++ b/src/client/datascience/interactive-ipynb/nativeEditorStorage.ts @@ -0,0 +1,614 @@ +import { nbformat } from '@jupyterlab/coreutils'; +import * as fastDeepEqual from 'fast-deep-equal'; +import { inject, injectable, named } from 'inversify'; +import * as path from 'path'; +import * as uuid from 'uuid/v4'; +import { Event, EventEmitter, Memento, Uri } from 'vscode'; +import { concatMultilineStringInput, splitMultilineString } from '../../../datascience-ui/common'; +import { createCodeCell } from '../../../datascience-ui/common/cellFactory'; +import { traceError } from '../../common/logger'; +import { IFileSystem } from '../../common/platform/types'; +import { GLOBAL_MEMENTO, ICryptoUtils, IExtensionContext, IMemento, WORKSPACE_MEMENTO } from '../../common/types'; +import { noop } from '../../common/utils/misc'; +import { PythonInterpreter } from '../../interpreter/contracts'; +import { Identifiers, KnownNotebookLanguages, Telemetry } from '../constants'; +import { IEditorContentChange, NotebookModelChange } from '../interactive-common/interactiveWindowTypes'; +import { InvalidNotebookFileError } from '../jupyter/invalidNotebookFileError'; +import { LiveKernelModel } from '../jupyter/kernels/types'; +import { CellState, ICell, IJupyterExecution, IJupyterKernelSpec, INotebookModel, INotebookStorage } from '../types'; + +// tslint:disable-next-line:no-require-imports no-var-requires +import detectIndent = require('detect-indent'); +import { sendTelemetryEvent } from '../../telemetry'; + +const KeyPrefix = 'notebook-storage-'; +const NotebookTransferKey = 'notebook-transfered'; + +interface INativeEditorStorageState { + file: Uri; + cells: ICell[]; + changeCount: number; + saveChangeCount: number; + notebookJson: Partial; +} + +@injectable() +export class NativeEditorStorage implements INotebookModel, INotebookStorage { + public get isDirty(): boolean { + return this._state.changeCount !== this._state.saveChangeCount; + } + public get changed(): Event { + return this._changedEmitter.event; + } + public get file(): Uri { + return this._state.file; + } + + public get isUntitled(): boolean { + return this.file.scheme === 'untitled'; + } + public get cells(): ICell[] { + return this._state.cells; + } + private _changedEmitter = new EventEmitter(); + private _state: INativeEditorStorageState = { + file: Uri.file(''), + changeCount: 0, + saveChangeCount: 0, + cells: [], + notebookJson: {} + }; + private indentAmount: string = ' '; + + constructor( + @inject(IJupyterExecution) private jupyterExecution: IJupyterExecution, + @inject(IFileSystem) private fileSystem: IFileSystem, + @inject(ICryptoUtils) private crypto: ICryptoUtils, + @inject(IExtensionContext) private context: IExtensionContext, + @inject(IMemento) @named(GLOBAL_MEMENTO) private globalStorage: Memento, + @inject(IMemento) @named(WORKSPACE_MEMENTO) private localStorage: Memento + ) {} + + public async load(file: Uri, possibleContents?: string): Promise { + // Reload our cells + await this.loadFromFile(file, possibleContents); + return this; + } + + public update(change: NotebookModelChange): void { + this.handleModelChange(change); + } + + public save(): Promise { + return this.saveAs(this.file); + } + + public async saveAs(file: Uri): Promise { + const contents = await this.getContent(); + await this.fileSystem.writeFile(file.fsPath, contents, 'utf-8'); + if (this.isDirty || file.fsPath !== this.file.fsPath) { + this.handleModelChange({ + source: 'user', + kind: 'file', + newFile: file, + oldFile: this.file, + newDirty: false, + oldDirty: this.isDirty + }); + } + return this; + } + + public async getJson(): Promise> { + await this.ensureNotebookJson(); + return this._state.notebookJson; + } + + public getContent(cells?: ICell[]): Promise { + return this.generateNotebookContent(cells ? cells : this.cells); + } + + private sendLanguageTelemetry(notebookJson: Partial) { + try { + // See if we have a language + let language = ''; + if (notebookJson.metadata?.language_info?.name) { + language = notebookJson.metadata?.language_info?.name; + } else if (notebookJson.metadata?.kernelspec?.language) { + language = notebookJson.metadata?.kernelspec?.language.toString(); + } + if (language && !KnownNotebookLanguages.includes(language.toLowerCase())) { + language = 'unknown'; + } + if (language) { + sendTelemetryEvent(Telemetry.NotebookLanguage, undefined, { language }); + } + } catch { + // If this fails, doesn't really matter + noop(); + } + } + + private handleModelChange(change: NotebookModelChange) { + const oldDirty = this.isDirty; + let changed = false; + + switch (change.source) { + case 'redo': + case 'user': + changed = this.handleRedo(change); + break; + case 'undo': + changed = this.handleUndo(change); + break; + default: + break; + } + + // Forward onto our listeners if necessary + if (changed || this.isDirty !== oldDirty) { + this._changedEmitter.fire({ ...change, newDirty: this.isDirty, oldDirty }); + } + } + + private handleRedo(change: NotebookModelChange): boolean { + let changed = false; + switch (change.kind) { + case 'clear': + changed = this.clearOutputs(); + break; + case 'edit': + changed = this.editCell(change.forward, change.id); + break; + case 'insert': + changed = this.insertCell(change.cell, change.index); + break; + case 'modify': + changed = this.modifyCells(change.newCells); + break; + case 'remove': + changed = this.removeCell(change.cell); + break; + case 'remove_all': + changed = this.removeAllCells(change.newCellId); + break; + case 'swap': + changed = this.swapCells(change.firstCellId, change.secondCellId); + break; + case 'version': + this.updateVersionInfo(change.interpreter, change.kernelSpec); + break; + case 'file': + changed = !this.fileSystem.arePathsSame(this._state.file.fsPath, change.newFile.fsPath); + this._state.file = change.newFile; + this._state.saveChangeCount = this._state.changeCount; + break; + default: + break; + } + + // Dirty state comes from undo. At least VS code will track it that way. However + // skip version and file changes as we don't forward those to VS code + if (change.kind !== 'file' && change.kind !== 'version') { + this._state.changeCount += 1; + } + + return changed; + } + + private handleUndo(change: NotebookModelChange): boolean { + let changed = false; + switch (change.kind) { + case 'clear': + changed = !fastDeepEqual(this._state.cells, change.oldCells); + this._state.cells = change.oldCells; + break; + case 'edit': + this.editCell(change.reverse, change.id); + changed = true; + break; + case 'insert': + changed = this.removeCell(change.cell); + break; + case 'modify': + changed = this.modifyCells(change.oldCells); + break; + case 'remove': + changed = this.insertCell(change.cell, change.index); + break; + case 'remove_all': + this._state.cells = change.oldCells; + changed = true; + break; + case 'swap': + changed = this.swapCells(change.firstCellId, change.secondCellId); + break; + default: + break; + } + + // Dirty state comes from undo. At least VS code will track it that way. + // Note unlike redo, 'file' and 'version' are not possible on undo as + // we don't send them to VS code. + this._state.changeCount -= 1; + + return changed; + } + + private removeAllCells(newCellId: string) { + this._state.cells = []; + this._state.cells.push(this.createEmptyCell(newCellId)); + return true; + } + + private applyCellContentChange(change: IEditorContentChange, id: string): boolean { + const normalized = change.text.replace(/\r/g, ''); + + // Figure out which cell we're editing. + const index = this.cells.findIndex(c => c.id === id); + if (index >= 0) { + // This is an actual edit. + const contents = concatMultilineStringInput(this.cells[index].data.source); + const before = contents.substr(0, change.rangeOffset); + const after = contents.substr(change.rangeOffset + change.rangeLength); + const newContents = `${before}${normalized}${after}`; + if (contents !== newContents) { + const newCell = { ...this.cells[index], data: { ...this.cells[index].data, source: newContents } }; + this._state.cells[index] = this.asCell(newCell); + return true; + } + } + return false; + } + + private editCell(changes: IEditorContentChange[], id: string): boolean { + // Apply the changes to the visible cell list + if (changes && changes.length) { + return changes.map(c => this.applyCellContentChange(c, id)).reduce((p, c) => p || c, false); + } + + return false; + } + + private swapCells(firstCellId: string, secondCellId: string) { + const first = this.cells.findIndex(v => v.id === firstCellId); + const second = this.cells.findIndex(v => v.id === secondCellId); + if (first >= 0 && second >= 0 && first !== second) { + const temp = { ...this.cells[first] }; + this._state.cells[first] = this.asCell(this.cells[second]); + this._state.cells[second] = this.asCell(temp); + return true; + } + return false; + } + + private modifyCells(cells: ICell[]): boolean { + // Update these cells in our list + cells.forEach(c => { + const index = this.cells.findIndex(v => v.id === c.id); + this._state.cells[index] = this.asCell(c); + }); + return true; + } + + private removeCell(cell: ICell): boolean { + const index = this.cells.findIndex(c => c.id === cell.id); + if (index >= 0) { + this._state.cells.splice(index, 1); + return true; + } + return false; + } + + private clearOutputs(): boolean { + const newCells = this.cells.map(c => + this.asCell({ ...c, data: { ...c.data, execution_count: null, outputs: [] } }) + ); + const result = !fastDeepEqual(newCells, this.cells); + this._state.cells = newCells; + return result; + } + + private insertCell(cell: ICell, index: number): boolean { + // Insert a cell into our visible list based on the index. They should be in sync + this._state.cells.splice(index, 0, cell); + return true; + } + + private updateVersionInfo( + interpreter: PythonInterpreter | undefined, + kernelSpec: IJupyterKernelSpec | LiveKernelModel | undefined + ) { + // Get our kernel_info and language_info from the current notebook + if ( + interpreter && + interpreter.version && + this._state.notebookJson.metadata && + this._state.notebookJson.metadata.language_info + ) { + this._state.notebookJson.metadata.language_info.version = interpreter.version.raw; + } + + if (kernelSpec && this._state.notebookJson.metadata && !this._state.notebookJson.metadata.kernelspec) { + // Add a new spec in this case + this._state.notebookJson.metadata.kernelspec = { + name: kernelSpec.name || kernelSpec.display_name || '', + display_name: kernelSpec.display_name || kernelSpec.name || '' + }; + } else if (kernelSpec && this._state.notebookJson.metadata && this._state.notebookJson.metadata.kernelspec) { + // Spec exists, just update name and display_name + this._state.notebookJson.metadata.kernelspec.name = kernelSpec.name || kernelSpec.display_name || ''; + this._state.notebookJson.metadata.kernelspec.display_name = + kernelSpec.display_name || kernelSpec.name || ''; + } + } + + // tslint:disable-next-line: no-any + private asCell(cell: any): ICell { + // Works around problems with setting a cell to another one in the nyc compiler. + return cell as ICell; + } + + private async loadFromFile(file: Uri, possibleContents?: string) { + // Save file + this._state.file = file; + + try { + // Attempt to read the contents if a viable file + const contents = + file.scheme === 'untitled' ? possibleContents : await this.fileSystem.readFile(this.file.fsPath); + + // See if this file was stored in storage prior to shutdown + const dirtyContents = await this.getStoredContents(); + if (dirtyContents) { + // This means we're dirty. Indicate dirty and load from this content + this.loadContents(dirtyContents); + } else { + // Load without setting dirty + this.loadContents(contents); + } + } catch { + // May not exist at this time. Should always have a single cell though + return [this.createEmptyCell(uuid())]; + } + } + + private createEmptyCell(id: string) { + return { + id, + line: 0, + file: Identifiers.EmptyFileName, + state: CellState.finished, + data: createCodeCell() + }; + } + + private loadContents(contents: string | undefined) { + // tslint:disable-next-line: no-any + const json = contents ? (JSON.parse(contents) as Partial) : undefined; + + // Double check json (if we have any) + if (json && !json.cells) { + throw new InvalidNotebookFileError(this.file.fsPath); + } + + // Then compute indent. It's computed from the contents + if (contents) { + this.indentAmount = detectIndent(contents).indent; + } + + // Then save the contents. We'll stick our cells back into this format when we save + if (json) { + this._state.notebookJson = json; + + // Log language or kernel telemetry + this.sendLanguageTelemetry(this._state.notebookJson); + } + + // Extract cells from the json + const cells = json ? (json.cells as (nbformat.ICodeCell | nbformat.IRawCell | nbformat.IMarkdownCell)[]) : []; + + // Remap the ids + const remapped = cells.map((c, index) => { + return { + id: `NotebookImport#${index}`, + file: Identifiers.EmptyFileName, + line: 0, + state: CellState.finished, + data: c + }; + }); + + // Make sure at least one + if (remapped.length === 0) { + remapped.splice(0, 0, this.createEmptyCell(uuid())); + } + + // Save as our visible list + this._state.cells = remapped; + } + + private async extractPythonMainVersion(notebookData: Partial): Promise { + if ( + notebookData && + notebookData.metadata && + notebookData.metadata.language_info && + notebookData.metadata.language_info.codemirror_mode && + // tslint:disable-next-line: no-any + typeof (notebookData.metadata.language_info.codemirror_mode as any).version === 'number' + ) { + // tslint:disable-next-line: no-any + return (notebookData.metadata.language_info.codemirror_mode as any).version; + } + // Use the active interpreter + const usableInterpreter = await this.jupyterExecution.getUsableJupyterPython(); + return usableInterpreter && usableInterpreter.version ? usableInterpreter.version.major : 3; + } + + private async ensureNotebookJson(): Promise { + if (!this._state.notebookJson || !this._state.notebookJson.metadata) { + const pythonNumber = await this.extractPythonMainVersion(this._state.notebookJson); + // Use this to build our metadata object + // Use these as the defaults unless we have been given some in the options. + const metadata: nbformat.INotebookMetadata = { + language_info: { + name: 'python', + codemirror_mode: { + name: 'ipython', + version: pythonNumber + } + }, + orig_nbformat: 2, + file_extension: '.py', + mimetype: 'text/x-python', + name: 'python', + npconvert_exporter: 'python', + pygments_lexer: `ipython${pythonNumber}`, + version: pythonNumber + }; + + // Default notebook data. + this._state.notebookJson = { + nbformat: 4, + nbformat_minor: 2, + metadata: metadata + }; + } + } + + private async generateNotebookContent(cells: ICell[]): Promise { + // Make sure we have some + await this.ensureNotebookJson(); + + // Reuse our original json except for the cells. + const json = { + ...(this._state.notebookJson as nbformat.INotebookContent), + cells: cells.map(c => this.fixupCell(c.data)) + }; + return JSON.stringify(json, null, this.indentAmount); + } + + private fixupCell(cell: nbformat.ICell): nbformat.ICell { + // Source is usually a single string on input. Convert back to an array + return ({ + ...cell, + source: splitMultilineString(cell.source) + // tslint:disable-next-line: no-any + } as any) as nbformat.ICell; // nyc (code coverage) barfs on this so just trick it. + } + + private getStorageKey(): string { + return `${KeyPrefix}${this.file.toString()}`; + } + + /** + * Gets any unsaved changes to the notebook file from the old locations. + * If the file has been modified since the uncommitted changes were stored, then ignore the uncommitted changes. + * + * @private + * @returns {(Promise)} + * @memberof NativeEditor + */ + private async getStoredContents(): Promise { + const key = this.getStorageKey(); + + // First look in the global storage file location + let result = await this.getStoredContentsFromFile(key); + if (!result) { + result = await this.getStoredContentsFromGlobalStorage(key); + if (!result) { + result = await this.getStoredContentsFromLocalStorage(key); + } + } + + return result; + } + + private async getStoredContentsFromFile(key: string): Promise { + const filePath = this.getHashedFileName(key); + try { + // Use this to read from the extension global location + const contents = await this.fileSystem.readFile(filePath); + const data = JSON.parse(contents); + // Check whether the file has been modified since the last time the contents were saved. + if (data && data.lastModifiedTimeMs && !this.isUntitled && this.file.scheme === 'file') { + const stat = await this.fileSystem.stat(this.file.fsPath); + if (stat.mtime > data.lastModifiedTimeMs) { + return; + } + } + if (data && !this.isUntitled && data.contents) { + return data.contents; + } + } catch { + noop(); + } + } + + private async getStoredContentsFromGlobalStorage(key: string): Promise { + try { + const data = this.globalStorage.get<{ contents?: string; lastModifiedTimeMs?: number }>(key); + + // If we have data here, make sure we eliminate any remnants of storage + if (data) { + await this.transferStorage(); + } + + // Check whether the file has been modified since the last time the contents were saved. + if (data && data.lastModifiedTimeMs && !this.isUntitled && this.file.scheme === 'file') { + const stat = await this.fileSystem.stat(this.file.fsPath); + if (stat.mtime > data.lastModifiedTimeMs) { + return; + } + } + if (data && !this.isUntitled && data.contents) { + return data.contents; + } + } catch { + noop(); + } + } + + private async getStoredContentsFromLocalStorage(key: string): Promise { + const workspaceData = this.localStorage.get(key); + if (workspaceData && !this.isUntitled) { + // Make sure to clear so we don't use this again. + this.localStorage.update(key, undefined); + + return workspaceData; + } + } + + // VS code recommended we use the hidden '_values' to iterate over all of the entries in + // the global storage map and delete the ones we own. + private async transferStorage(): Promise { + const promises: Thenable[] = []; + + // Indicate we ran this function + await this.globalStorage.update(NotebookTransferKey, true); + + try { + // tslint:disable-next-line: no-any + if ((this.globalStorage as any)._value) { + // tslint:disable-next-line: no-any + const keys = Object.keys((this.globalStorage as any)._value); + [...keys].forEach((k: string) => { + if (k.startsWith(KeyPrefix)) { + // Remove from the map so that global storage does not have this anymore. + // Use the real API here as we don't know how the map really gets updated. + promises.push(this.globalStorage.update(k, undefined)); + } + }); + } + } catch (e) { + traceError('Exception eliminating global storage parts:', e); + } + + return Promise.all(promises); + } + + private getHashedFileName(key: string): string { + const file = `${this.crypto.createHash(key, 'string')}.ipynb`; + return path.join(this.context.globalStoragePath, file); + } +} diff --git a/src/client/datascience/interactive-window/interactiveWindow.ts b/src/client/datascience/interactive-window/interactiveWindow.ts index 7212626118d2..537736c40691 100644 --- a/src/client/datascience/interactive-window/interactiveWindow.ts +++ b/src/client/datascience/interactive-window/interactiveWindow.ts @@ -31,7 +31,12 @@ import { IInterpreterService } from '../../interpreter/contracts'; import { captureTelemetry, sendTelemetryEvent } from '../../telemetry'; import { EditorContexts, Identifiers, Telemetry } from '../constants'; import { InteractiveBase } from '../interactive-common/interactiveBase'; -import { InteractiveWindowMessages, ISubmitNewCell, SysInfoReason } from '../interactive-common/interactiveWindowTypes'; +import { + InteractiveWindowMessages, + ISubmitNewCell, + NotebookModelChange, + SysInfoReason +} from '../interactive-common/interactiveWindowTypes'; import { ProgressReporter } from '../progress/progressReporter'; import { ICell, @@ -45,7 +50,6 @@ import { IJupyterDebugger, IJupyterExecution, IJupyterVariables, - INotebookEditorProvider, INotebookExporter, INotebookServerOptions, IStatusProvider, @@ -97,7 +101,6 @@ export class InteractiveWindow extends InteractiveBase implements IInteractiveWi @inject(IDataViewerProvider) dataExplorerProvider: IDataViewerProvider, @inject(IJupyterVariables) jupyterVariables: IJupyterVariables, @inject(IJupyterDebugger) jupyterDebugger: IJupyterDebugger, - @inject(INotebookEditorProvider) editorProvider: INotebookEditorProvider, @inject(IDataScienceErrorHandler) errorHandler: IDataScienceErrorHandler, @inject(IPersistentStateFactory) private readonly stateFactory: IPersistentStateFactory, @inject(IMemento) @named(GLOBAL_MEMENTO) globalStorage: Memento, @@ -124,7 +127,6 @@ export class InteractiveWindow extends InteractiveBase implements IInteractiveWi dataExplorerProvider, jupyterVariables, jupyterDebugger, - editorProvider, errorHandler, commandManager, globalStorage, @@ -211,6 +213,10 @@ export class InteractiveWindow extends InteractiveBase implements IInteractiveWi this.handleMessage(message, payload, this.handleReturnAllCells); break; + case InteractiveWindowMessages.UpdateModel: + this.handleMessage(message, payload, this.handleModelChange); + break; + default: break; } @@ -365,7 +371,7 @@ export class InteractiveWindow extends InteractiveBase implements IInteractiveWi filters: filtersObject }); if (uri) { - await this.exportToFile(cells, uri.fsPath); + await this.jupyterExporter.exportToFile(cells, uri.fsPath); } } finally { this.stopProgress(); @@ -373,6 +379,17 @@ export class InteractiveWindow extends InteractiveBase implements IInteractiveWi } } + private handleModelChange(update: NotebookModelChange) { + // Send telemetry for delete and delete all. We don't send telemetry for the other updates yet + if (update.source === 'user') { + if (update.kind === 'remove_all') { + sendTelemetryEvent(Telemetry.DeleteAllCells); + } else if (update.kind === 'remove') { + sendTelemetryEvent(Telemetry.DeleteCell); + } + } + } + // tslint:disable-next-line:no-any private handleReturnAllCells(cells: ICell[]) { // See what we're waiting for. diff --git a/src/client/datascience/interactive-window/interactiveWindowCommandListener.ts b/src/client/datascience/interactive-window/interactiveWindowCommandListener.ts index a056fdb17f9d..74c0d16d0193 100644 --- a/src/client/datascience/interactive-window/interactiveWindowCommandListener.ts +++ b/src/client/datascience/interactive-window/interactiveWindowCommandListener.ts @@ -217,7 +217,7 @@ export class InteractiveWindowCommandListener implements IDataScienceCommandList ...questions ); if (selection === openQuestion1) { - await this.ipynbProvider.open(uri, await this.fileSystem.readFile(uri.fsPath)); + await this.ipynbProvider.open(uri); } if (selection === openQuestion2) { // If the user wants to, open the notebook they just generated. @@ -287,7 +287,7 @@ export class InteractiveWindowCommandListener implements IDataScienceCommandList ...questions ); if (selection === openQuestion1) { - await this.ipynbProvider.open(Uri.file(output), await this.fileSystem.readFile(output)); + await this.ipynbProvider.open(Uri.file(output)); } if (selection === openQuestion2) { // If the user wants to, open the notebook they just generated. diff --git a/src/client/datascience/jupyter/jupyterExporter.ts b/src/client/datascience/jupyter/jupyterExporter.ts index 36e70a5767d1..4675d08fc59d 100644 --- a/src/client/datascience/jupyter/jupyterExporter.ts +++ b/src/client/datascience/jupyter/jupyterExporter.ts @@ -7,16 +7,25 @@ import * as os from 'os'; import * as path from 'path'; import * as uuid from 'uuid/v4'; +import { Uri } from 'vscode'; import { concatMultilineStringInput } from '../../../datascience-ui/common'; import { createCodeCell } from '../../../datascience-ui/common/cellFactory'; -import { IWorkspaceService } from '../../common/application/types'; +import { IApplicationShell, IWorkspaceService } from '../../common/application/types'; +import { traceError } from '../../common/logger'; import { IFileSystem, IPlatformService } from '../../common/platform/types'; import { IConfigurationService } from '../../common/types'; import * as localize from '../../common/utils/localize'; import { noop } from '../../common/utils/misc'; import { CellMatcher } from '../cellMatcher'; import { CodeSnippits, Identifiers } from '../constants'; -import { CellState, ICell, IJupyterExecution, INotebookExporter } from '../types'; +import { + CellState, + ICell, + IDataScienceErrorHandler, + IJupyterExecution, + INotebookEditorProvider, + INotebookExporter +} from '../types'; @injectable() export class JupyterExporter implements INotebookExporter { @@ -25,13 +34,54 @@ export class JupyterExporter implements INotebookExporter { @inject(IWorkspaceService) private workspaceService: IWorkspaceService, @inject(IConfigurationService) private configService: IConfigurationService, @inject(IFileSystem) private fileSystem: IFileSystem, - @inject(IPlatformService) private readonly platform: IPlatformService + @inject(IPlatformService) private readonly platform: IPlatformService, + @inject(IApplicationShell) private readonly applicationShell: IApplicationShell, + @inject(INotebookEditorProvider) protected ipynbProvider: INotebookEditorProvider, + @inject(IDataScienceErrorHandler) protected errorHandler: IDataScienceErrorHandler ) {} public dispose() { noop(); } + public async exportToFile(cells: ICell[], file: string): Promise { + let directoryChange; + const settings = this.configService.getSettings(); + if (settings.datascience.changeDirOnImportExport) { + directoryChange = file; + } + + const notebook = await this.translateToNotebook(cells, directoryChange); + + try { + // tslint:disable-next-line: no-any + const contents = JSON.stringify(notebook); + await this.fileSystem.writeFile(file, contents, { encoding: 'utf8', flag: 'w' }); + const openQuestion1 = localize.DataScience.exportOpenQuestion1(); + const openQuestion2 = (await this.jupyterExecution.isSpawnSupported()) + ? localize.DataScience.exportOpenQuestion() + : undefined; + this.showInformationMessage( + localize.DataScience.exportDialogComplete().format(file), + openQuestion1, + openQuestion2 + ).then(async (str: string | undefined) => { + try { + if (str === openQuestion2 && openQuestion2) { + // If the user wants to, open the notebook they just generated. + await this.jupyterExecution.spawnNotebook(file); + } else if (str === openQuestion1) { + await this.ipynbProvider.open(Uri.file(file)); + } + } catch (e) { + await this.errorHandler.handleError(e); + } + }); + } catch (exc) { + traceError('Error in exporting notebook file'); + this.applicationShell.showInformationMessage(localize.DataScience.exportDialogFailed().format(exc)); + } + } public async translateToNotebook( cells: ICell[], changeDirectory?: string @@ -73,6 +123,18 @@ export class JupyterExporter implements INotebookExporter { }; } + private showInformationMessage( + message: string, + question1: string, + question2?: string + ): Thenable { + if (question2) { + return this.applicationShell.showInformationMessage(message, question1, question2); + } else { + return this.applicationShell.showInformationMessage(message, question1); + } + } + // For exporting, put in a cell that will change the working directory back to the workspace directory so relative data paths will load correctly private addDirectoryChangeCell = async (cells: ICell[], file: string): Promise => { const changeDirectory = await this.calculateDirectoryChange(file, cells); diff --git a/src/client/datascience/messages.ts b/src/client/datascience/messages.ts index b2d17ad18404..c7e7ca14dcf8 100644 --- a/src/client/datascience/messages.ts +++ b/src/client/datascience/messages.ts @@ -2,18 +2,18 @@ // Licensed under the MIT License. 'use strict'; -export namespace CssMessages { - export const GetCssRequest = 'get_css_request'; - export const GetCssResponse = 'get_css_response'; - export const GetMonacoThemeRequest = 'get_monaco_theme_request'; - export const GetMonacoThemeResponse = 'get_monaco_theme_response'; +export enum CssMessages { + GetCssRequest = 'get_css_request', + GetCssResponse = 'get_css_response', + GetMonacoThemeRequest = 'get_monaco_theme_request', + GetMonacoThemeResponse = 'get_monaco_theme_response' } -export namespace SharedMessages { - export const UpdateSettings = 'update_settings'; - export const Started = 'started'; - export const LocInit = 'loc_init'; - export const StyleUpdate = 'style_update'; +export enum SharedMessages { + UpdateSettings = 'update_settings', + Started = 'started', + LocInit = 'loc_init', + StyleUpdate = 'style_update' } export interface IGetCssRequest { diff --git a/src/client/datascience/serviceRegistry.ts b/src/client/datascience/serviceRegistry.ts index 920e4faeede0..e991aefff727 100644 --- a/src/client/datascience/serviceRegistry.ts +++ b/src/client/datascience/serviceRegistry.ts @@ -2,7 +2,7 @@ // Licensed under the MIT License. 'use strict'; import { IExtensionSingleActivationService } from '../activation/types'; -import { IWorkspaceService } from '../common/application/types'; +import { IApplicationEnvironment, IWorkspaceService } from '../common/application/types'; import { IServiceManager } from '../ioc/types'; import { Activation } from './activation'; import { CodeCssGenerator } from './codeCssGenerator'; @@ -27,10 +27,12 @@ import { DebugListener } from './interactive-common/debugListener'; import { IntellisenseProvider } from './interactive-common/intellisense/intellisenseProvider'; import { LinkProvider } from './interactive-common/linkProvider'; import { ShowPlotListener } from './interactive-common/showPlotListener'; -import { AutoSaveService } from './interactive-ipynb/autoSaveService'; import { NativeEditor } from './interactive-ipynb/nativeEditor'; import { NativeEditorCommandListener } from './interactive-ipynb/nativeEditorCommandListener'; +import { NativeEditorOldWebView } from './interactive-ipynb/nativeEditorOldWebView'; import { NativeEditorProvider } from './interactive-ipynb/nativeEditorProvider'; +import { NativeEditorProviderOld } from './interactive-ipynb/nativeEditorProviderOld'; +import { NativeEditorStorage } from './interactive-ipynb/nativeEditorStorage'; import { InteractiveWindow } from './interactive-window/interactiveWindow'; import { InteractiveWindowCommandListener } from './interactive-window/interactiveWindowCommandListener'; import { InteractiveWindowProvider } from './interactive-window/interactiveWindowProvider'; @@ -98,6 +100,7 @@ import { INotebookExporter, INotebookImporter, INotebookServer, + INotebookStorage, IPlotViewer, IPlotViewerProvider, IStatusProvider, @@ -140,7 +143,6 @@ export function registerTypes(serviceManager: IServiceManager) { serviceManager.add(IInteractiveWindowListener, ShowPlotListener); serviceManager.add(IInteractiveWindowListener, DebugListener); serviceManager.add(IInteractiveWindowListener, GatherListener); - serviceManager.add(IInteractiveWindowListener, AutoSaveService); serviceManager.addSingleton(IPlotViewerProvider, PlotViewerProvider); serviceManager.add(IPlotViewer, PlotViewer); serviceManager.addSingleton(IJupyterDebugger, JupyterDebugger); @@ -151,8 +153,16 @@ export function registerTypes(serviceManager: IServiceManager) { serviceManager.addBinding(ICellHashProvider, IInteractiveWindowListener); serviceManager.addBinding(ICellHashProvider, INotebookExecutionLogger); serviceManager.addBinding(IJupyterDebugger, ICellHashListener); - serviceManager.addSingleton(INotebookEditorProvider, NativeEditorProvider); - serviceManager.add(INotebookEditor, NativeEditor); + const app = serviceManager.get(IApplicationEnvironment); + serviceManager.addSingleton( + INotebookEditorProvider, + app.packageJson.enableProposedApi ? NativeEditorProvider : NativeEditorProviderOld + ); + serviceManager.add(INotebookStorage, NativeEditorStorage); + serviceManager.add( + INotebookEditor, + app.packageJson.enableProposedApi ? NativeEditor : NativeEditorOldWebView + ); serviceManager.addSingleton(IDataScienceCommandListener, NativeEditorCommandListener); serviceManager.addBinding(ICodeLensFactory, IInteractiveWindowListener); serviceManager.addSingleton(IDebugLocationTracker, DebugLocationTrackerFactory); diff --git a/src/client/datascience/types.ts b/src/client/datascience/types.ts index 4b6c239aafed..272611f2b5c1 100644 --- a/src/client/datascience/types.ts +++ b/src/client/datascience/types.ts @@ -16,7 +16,8 @@ import { Range, TextDocument, TextEditor, - Uri + Uri, + WebviewPanel } from 'vscode'; import { ServerStatus } from '../../datascience-ui/interactive-common/mainState'; import { ICommandManager } from '../common/application/types'; @@ -25,6 +26,7 @@ import { IAsyncDisposable, IDataScienceSettings, IDisposable, Resource } from '. import { StopWatch } from '../common/utils/stopWatch'; import { PythonInterpreter } from '../interpreter/contracts'; import { JupyterCommands } from './constants'; +import { NotebookModelChange } from './interactive-common/interactiveWindowTypes'; import { JupyterServerInfo } from './jupyter/jupyterConnection'; import { JupyterInstallError } from './jupyter/jupyterInstallError'; import { JupyterKernelSpec } from './jupyter/kernels/jupyterKernelSpec'; @@ -295,6 +297,7 @@ export interface INotebookImporter extends Disposable { export const INotebookExporter = Symbol('INotebookExporter'); export interface INotebookExporter extends Disposable { translateToNotebook(cells: ICell[], directoryChange?: string): Promise; + exportToFile(cells: ICell[], file: string): Promise; } export const IInteractiveWindowProvider = Symbol('IInteractiveWindowProvider'); @@ -358,7 +361,8 @@ export interface INotebookEditorProvider { readonly editors: INotebookEditor[]; readonly onDidOpenNotebookEditor: Event; readonly onDidChangeActiveNotebookEditor: Event; - open(file: Uri, contents: string): Promise; + readonly onDidCloseNotebookEditor: Event; + open(file: Uri): Promise; show(file: Uri): Promise; createNew(contents?: string): Promise; getNotebookOptions(resource: Resource): Promise; @@ -372,7 +376,6 @@ export interface INotebookEditor extends IInteractiveBase { readonly executed: Event; readonly modified: Event; readonly saved: Event; - readonly metadataUpdated: Event; /** * Is this notebook representing an untitled file which has never been saved yet. */ @@ -384,7 +387,7 @@ export interface INotebookEditor extends IInteractiveBase { readonly file: Uri; readonly visible: boolean; readonly active: boolean; - load(contents: string, file: Uri): Promise; + load(storage: INotebookModel, webViewPanel?: WebviewPanel): Promise; runAllCells(): void; runSelectedCell(): void; addCellBelow(): void; @@ -805,6 +808,24 @@ export interface IJupyterInterpreterDependencyManager { installMissingDependencies(err?: JupyterInstallError): Promise; } +export interface INotebookModel { + readonly file: Uri; + readonly isDirty: boolean; + readonly isUntitled: boolean; + readonly changed: Event; + readonly cells: ICell[]; + getJson(): Promise>; + getContent(cells?: ICell[]): Promise; + update(change: NotebookModelChange): void; +} + +export const INotebookStorage = Symbol('INotebookStorage'); + +export interface INotebookStorage { + load(file: Uri, contents?: string): Promise; + save(): Promise; + saveAs(file: Uri): Promise; +} type WebViewViewState = { readonly visible: boolean; readonly active: boolean; diff --git a/src/client/datascience/webViewHost.ts b/src/client/datascience/webViewHost.ts index 303145e0c037..25dc1f99073f 100644 --- a/src/client/datascience/webViewHost.ts +++ b/src/client/datascience/webViewHost.ts @@ -4,7 +4,7 @@ import '../common/extensions'; import { injectable, unmanaged } from 'inversify'; -import { ConfigurationChangeEvent, ViewColumn, WorkspaceConfiguration } from 'vscode'; +import { ConfigurationChangeEvent, ViewColumn, WebviewPanel, WorkspaceConfiguration } from 'vscode'; import { IWebPanel, IWebPanelMessageListener, IWebPanelProvider, IWorkspaceService } from '../common/application/types'; import { traceInfo, traceWarning } from '../common/logger'; @@ -217,7 +217,7 @@ export abstract class WebViewHost implements IDisposable { return this.themeIsDarkPromise ? this.themeIsDarkPromise.promise : Promise.resolve(false); } - protected async loadWebPanel(cwd: string) { + protected async loadWebPanel(cwd: string, webViewPanel?: WebviewPanel) { // Make not disposed anymore this.disposed = false; @@ -266,7 +266,8 @@ export abstract class WebViewHost implements IDisposable { scripts: this.scripts, settings, startHttpServer: false, - cwd + cwd, + webViewPanel }); traceInfo('Web view created.'); @@ -325,6 +326,8 @@ export abstract class WebViewHost implements IDisposable { // Resolve our started promise. This means the webpanel is ready to go. this.webPanelInit.resolve(); + + traceInfo('Web view react rendered'); } } diff --git a/src/client/telemetry/index.ts b/src/client/telemetry/index.ts index 9c94e2db9fff..bba8eb594ce5 100644 --- a/src/client/telemetry/index.ts +++ b/src/client/telemetry/index.ts @@ -1753,6 +1753,10 @@ export interface IEventNamePropertyMapping { * @memberof IEventNamePropertyMapping */ [Telemetry.StartSessionFailedJupyter]: undefined | never; + /** + * Telemetry event fired if a failure occurs loading a notebook + */ + [Telemetry.OpenNotebookFailure]: undefined | never; /** * Telemetry event sent to capture total time taken for completions list to be provided by LS. * This is used to compare against time taken by Jupyter. diff --git a/src/client/testing/nosetest/services/parserService.ts b/src/client/testing/nosetest/services/parserService.ts index 0e77b505c5f8..e67844cf9591 100644 --- a/src/client/testing/nosetest/services/parserService.ts +++ b/src/client/testing/nosetest/services/parserService.ts @@ -76,7 +76,6 @@ export class TestsParser implements ITestsParser { } } - // tslint:disable-next-line: max-func-body-length private parseNoseTestModuleCollectionResult(rootDirectory: string, lines: string[], testFiles: TestFile[]) { let currentPackage: string = ''; let fileName = ''; diff --git a/src/datascience-ui/data-explorer/mainPanel.tsx b/src/datascience-ui/data-explorer/mainPanel.tsx index f5ef8279fcf8..dfdaa7382032 100644 --- a/src/datascience-ui/data-explorer/mainPanel.tsx +++ b/src/datascience-ui/data-explorer/mainPanel.tsx @@ -94,7 +94,7 @@ export class MainPanel extends React.Component this.postOffice.addHandler(this); // Tell the dataviewer code we have started. - this.postOffice.sendMessage(DataViewerMessages.Started); + this.postOffice.sendMessage(DataViewerMessages.Started); } public componentWillUnmount() { diff --git a/src/datascience-ui/history-react/interactiveCell.tsx b/src/datascience-ui/history-react/interactiveCell.tsx index 434a6409345b..94825e2721be 100644 --- a/src/datascience-ui/history-react/interactiveCell.tsx +++ b/src/datascience-ui/history-react/interactiveCell.tsx @@ -11,7 +11,6 @@ import { connect } from 'react-redux'; import { Identifiers } from '../../client/datascience/constants'; import { CellState, IDataScienceExtraSettings } from '../../client/datascience/types'; -import { concatMultilineStringInput } from '../common'; import { CellInput } from '../interactive-common/cellInput'; import { CellOutput } from '../interactive-common/cellOutput'; import { CollapseButton } from '../interactive-common/collapseButton'; @@ -23,6 +22,7 @@ import { IKeyboardEvent } from '../react-common/event'; import { Image, ImageName } from '../react-common/image'; import { ImageButton } from '../react-common/imageButton'; import { getLocString } from '../react-common/locReactSide'; +import { IMonacoModelContentChangeEvent } from '../react-common/monacoHelpers'; import { actionCreators } from './redux/actions'; interface IInteractiveCellBaseProps { @@ -141,7 +141,6 @@ export class InteractiveCell extends React.Component { tabIndex={0} onKeyDown={this.onKeyDown} onClick={this.onMouseClick} - onDoubleClick={this.onMouseDoubleClick} >
{this.renderControls()} @@ -227,14 +226,6 @@ export class InteractiveCell extends React.Component { } }; - private onMouseDoubleClick = (ev: React.MouseEvent) => { - // When we receive double click, propagate upwards. Might change our state - if (this.props.doubleClickCell) { - ev.stopPropagation(); - this.props.doubleClickCell(this.props.cellVM.cell.id); - } - }; - private renderControls = () => { const busy = this.props.cellVM.cell.state === CellState.init || this.props.cellVM.cell.state === CellState.executing; @@ -294,6 +285,8 @@ export class InteractiveCell extends React.Component { keyDown={this.isEditCell() ? this.onEditCellKeyDown : undefined} showLineNumbers={this.props.cellVM.showLineNumbers} font={this.props.font} + disableUndoStack={this.props.cellVM.cell.id !== Identifiers.EditCellId} + codeVersion={this.props.cellVM.codeVersion ? this.props.cellVM.codeVersion : 0} focusPending={this.props.focusPending} /> ); @@ -309,14 +302,8 @@ export class InteractiveCell extends React.Component { this.props.unfocus(this.getCell().id); }; - private getCurrentCode(): string { - // Get current monaco code, if not available fallback to cell data source - const contents = this.codeRef.current ? this.codeRef.current.getContents() : undefined; - return contents || concatMultilineStringInput(this.props.cellVM.cell.data.source); - } - - private onCodeChange = (changes: monacoEditor.editor.IModelContentChange[], cellId: string, modelId: string) => { - this.props.editCell(cellId, changes, modelId, this.getCurrentCode()); + private onCodeChange = (e: IMonacoModelContentChangeEvent) => { + this.props.editCell(this.getCell().id, e); }; private onCodeCreated = (_code: string, _file: string, cellId: string, modelId: string) => { @@ -367,6 +354,8 @@ export class InteractiveCell extends React.Component { this.editCellEscape(e); } else if (e.code === 'Enter' && e.shiftKey) { this.editCellSubmit(e); + } else if (e.code === 'NumpadEnter' && e.shiftKey) { + this.editCellSubmit(e); } }; diff --git a/src/datascience-ui/history-react/redux/actions.ts b/src/datascience-ui/history-react/redux/actions.ts index d8d174078302..51c145ecc9f0 100644 --- a/src/datascience-ui/history-react/redux/actions.ts +++ b/src/datascience-ui/history-react/redux/actions.ts @@ -1,120 +1,103 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT License. 'use strict'; -import * as monacoEditor from 'monaco-editor/esm/vs/editor/editor.api'; -import { IRefreshVariablesRequest } from '../../../client/datascience/interactive-common/interactiveWindowTypes'; +import { + IInteractiveWindowMapping, + InteractiveWindowMessages +} from '../../../client/datascience/interactive-common/interactiveWindowTypes'; import { IJupyterVariable, IJupyterVariablesRequest } from '../../../client/datascience/types'; import { CommonAction, CommonActionType, + CommonActionTypeMapping, ICellAction, ICodeAction, ICodeCreatedAction, IEditCellAction, ILinkClickAction, + IOpenSettingsAction, IScrollAction, - IShowDataViewerAction, - IShowPlotAction + IShowDataViewerAction } from '../../interactive-common/redux/reducers/types'; +import { IMonacoModelContentChangeEvent } from '../../react-common/monacoHelpers'; + +// This function isn't made common and not exported, to ensure it isn't used elsewhere. +function createIncomingActionWithPayload< + M extends IInteractiveWindowMapping & CommonActionTypeMapping, + K extends keyof M +>(type: K, data: M[K]): CommonAction { + // tslint:disable-next-line: no-any + return { type, payload: { data, messageDirection: 'incoming' } as any } as any; +} +// This function isn't made common and not exported, to ensure it isn't used elsewhere. +function createIncomingAction(type: CommonActionType | InteractiveWindowMessages): CommonAction { + return { type, payload: { messageDirection: 'incoming', data: undefined } }; +} // See https://react-redux.js.org/using-react-redux/connect-mapdispatch#defining-mapdispatchtoprops-as-an-object export const actionCreators = { - focusInput: (): CommonAction => ({ type: CommonActionType.FOCUS_INPUT }), - refreshVariables: (newExecutionCount?: number): CommonAction => ({ - type: CommonActionType.REFRESH_VARIABLES, - payload: { newExecutionCount } - }), - restartKernel: (): CommonAction => ({ type: CommonActionType.RESTART_KERNEL }), - interruptKernel: (): CommonAction => ({ type: CommonActionType.INTERRUPT_KERNEL }), - deleteAllCells: (): CommonAction => ({ type: CommonActionType.DELETE_ALL_CELLS }), - deleteCell: (cellId: string): CommonAction => ({ - type: CommonActionType.DELETE_CELL, - payload: { cellId } - }), - undo: (): CommonAction => ({ type: CommonActionType.UNDO }), - redo: (): CommonAction => ({ type: CommonActionType.REDO }), - linkClick: (href: string): CommonAction => ({ - type: CommonActionType.LINK_CLICK, - payload: { href } - }), - showPlot: (imageHtml: string): CommonAction => ({ - type: CommonActionType.SHOW_PLOT, - payload: { imageHtml } - }), - toggleInputBlock: (cellId: string): CommonAction => ({ - type: CommonActionType.TOGGLE_INPUT_BLOCK, - payload: { cellId } - }), - gotoCell: (cellId: string): CommonAction => ({ - type: CommonActionType.GOTO_CELL, - payload: { cellId } - }), - copyCellCode: (cellId: string): CommonAction => ({ - type: CommonActionType.COPY_CELL_CODE, - payload: { cellId } - }), - gatherCell: (cellId: string): CommonAction => ({ - type: CommonActionType.GATHER_CELL, - payload: { cellId } - }), - clickCell: (cellId: string): CommonAction => ({ - type: CommonActionType.CLICK_CELL, - payload: { cellId } - }), - doubleClickCell: (cellId: string): CommonAction => ({ - type: CommonActionType.DOUBLE_CLICK_CELL, - payload: { cellId } - }), - editCell: ( - cellId: string, - changes: monacoEditor.editor.IModelContentChange[], - modelId: string, - code: string - ): CommonAction => ({ - type: CommonActionType.EDIT_CELL, - payload: { cellId, changes, modelId, code } - }), - submitInput: (code: string, cellId: string): CommonAction => ({ - type: CommonActionType.SUBMIT_INPUT, - payload: { code, cellId } - }), - toggleVariableExplorer: (): CommonAction => ({ - type: CommonActionType.TOGGLE_VARIABLE_EXPLORER - }), - expandAll: (): CommonAction => ({ type: CommonActionType.EXPAND_ALL }), - collapseAll: (): CommonAction => ({ type: CommonActionType.COLLAPSE_ALL }), - export: (): CommonAction => ({ type: CommonActionType.EXPORT }), - showDataViewer: (variable: IJupyterVariable, columnSize: number): CommonAction => ({ - type: CommonActionType.SHOW_DATA_VIEWER, - payload: { variable, columnSize } - }), - editorLoaded: (): CommonAction => ({ type: CommonActionType.EDITOR_LOADED }), - scroll: (isAtBottom: boolean): CommonAction => ({ - type: CommonActionType.SCROLL, - payload: { isAtBottom } - }), - unfocus: (cellId: string | undefined): CommonAction => ({ - type: CommonActionType.UNFOCUS_CELL, - payload: { cellId } - }), - codeCreated: (cellId: string | undefined, modelId: string): CommonAction => ({ - type: CommonActionType.CODE_CREATED, - payload: { cellId, modelId } - }), - editorUnmounted: (): CommonAction => ({ type: CommonActionType.UNMOUNT }), - selectKernel: (): CommonAction => ({ type: CommonActionType.SELECT_KERNEL }), - selectServer: (): CommonAction => ({ type: CommonActionType.SELECT_SERVER }), - openSettings: (setting?: string): CommonAction => ({ - type: CommonActionType.OPEN_SETTINGS, - payload: setting - }), + focusInput: (): CommonAction => createIncomingAction(CommonActionType.FOCUS_INPUT), + restartKernel: (): CommonAction => createIncomingAction(CommonActionType.RESTART_KERNEL), + interruptKernel: (): CommonAction => createIncomingAction(CommonActionType.INTERRUPT_KERNEL), + deleteAllCells: (): CommonAction => createIncomingAction(InteractiveWindowMessages.DeleteAllCells), + deleteCell: (cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.DELETE_CELL, { cellId }), + undo: (): CommonAction => createIncomingAction(InteractiveWindowMessages.Undo), + redo: (): CommonAction => createIncomingAction(InteractiveWindowMessages.Redo), + linkClick: (href: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.LINK_CLICK, { href }), + showPlot: (imageHtml: string) => createIncomingActionWithPayload(InteractiveWindowMessages.ShowPlot, imageHtml), + toggleInputBlock: (cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.TOGGLE_INPUT_BLOCK, { cellId }), + gotoCell: (cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.GOTO_CELL, { cellId }), + copyCellCode: (cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.COPY_CELL_CODE, { cellId }), + gatherCell: (cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.GATHER_CELL, { cellId }), + clickCell: (cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.CLICK_CELL, { cellId }), + editCell: (cellId: string, e: IMonacoModelContentChangeEvent): CommonAction => + createIncomingActionWithPayload(CommonActionType.EDIT_CELL, { + cellId, + version: e.versionId, + modelId: e.model.id, + forward: e.forward, + reverse: e.reverse, + id: cellId, + code: e.model.getValue() + }), + submitInput: (code: string, cellId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.SUBMIT_INPUT, { code, cellId }), + toggleVariableExplorer: (): CommonAction => createIncomingAction(CommonActionType.TOGGLE_VARIABLE_EXPLORER), + expandAll: (): CommonAction => createIncomingAction(InteractiveWindowMessages.ExpandAll), + collapseAll: (): CommonAction => createIncomingAction(InteractiveWindowMessages.CollapseAll), + export: (): CommonAction => createIncomingAction(CommonActionType.EXPORT), + showDataViewer: (variable: IJupyterVariable, columnSize: number): CommonAction => + createIncomingActionWithPayload(CommonActionType.SHOW_DATA_VIEWER, { variable, columnSize }), + editorLoaded: (): CommonAction => createIncomingAction(CommonActionType.EDITOR_LOADED), + scroll: (isAtBottom: boolean): CommonAction => + createIncomingActionWithPayload(CommonActionType.SCROLL, { isAtBottom }), + unfocus: (cellId: string | undefined): CommonAction => + createIncomingActionWithPayload(CommonActionType.UNFOCUS_CELL, { cellId }), + codeCreated: (cellId: string | undefined, modelId: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.CODE_CREATED, { cellId, modelId }), + editorUnmounted: (): CommonAction => createIncomingAction(CommonActionType.UNMOUNT), + selectKernel: (): CommonAction => createIncomingAction(InteractiveWindowMessages.SelectKernel), + selectServer: (): CommonAction => createIncomingAction(CommonActionType.SELECT_SERVER), + openSettings: (setting?: string): CommonAction => + createIncomingActionWithPayload(CommonActionType.OPEN_SETTINGS, { setting }), getVariableData: ( newExecutionCount: number, startIndex: number = 0, pageSize: number = 100 - ): CommonAction => ({ - type: CommonActionType.GET_VARIABLE_DATA, - payload: { executionCount: newExecutionCount, sortColumn: 'name', sortAscending: true, startIndex, pageSize } - }) + ): CommonAction => + createIncomingActionWithPayload(CommonActionType.GET_VARIABLE_DATA, { + executionCount: newExecutionCount, + sortColumn: 'name', + sortAscending: true, + startIndex, + pageSize + }) }; diff --git a/src/datascience-ui/history-react/redux/mapping.ts b/src/datascience-ui/history-react/redux/mapping.ts index b456141f6317..92002dfe409b 100644 --- a/src/datascience-ui/history-react/redux/mapping.ts +++ b/src/datascience-ui/history-react/redux/mapping.ts @@ -1,77 +1,30 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT License. 'use strict'; -import { IScrollToCell } from '../../../client/datascience/interactive-common/interactiveWindowTypes'; -import { IGetCssResponse } from '../../../client/datascience/messages'; -import { IGetMonacoThemeResponse } from '../../../client/datascience/monacoMessages'; -import { ICell } from '../../../client/datascience/types'; -import { IMainState, IServerState } from '../../interactive-common/mainState'; -import { IncomingMessageActions } from '../../interactive-common/redux/postOffice'; import { - CommonActionType, - ICellAction, - ICodeAction, - IEditCellAction, - ILinkClickAction, - IScrollAction, - IShowDataViewerAction, - IShowPlotAction -} from '../../interactive-common/redux/reducers/types'; + IInteractiveWindowMapping, + InteractiveWindowMessages +} from '../../../client/datascience/interactive-common/interactiveWindowTypes'; +import { BaseReduxActionPayload } from '../../../client/datascience/interactive-common/types'; +import { IMainState } from '../../interactive-common/mainState'; +import { CommonActionType, CommonActionTypeMapping } from '../../interactive-common/redux/reducers/types'; import { ReducerArg, ReducerFunc } from '../../react-common/reduxUtils'; -type InteractiveReducerFunc = ReducerFunc; +export type InteractiveReducerFunc = ReducerFunc< + IMainState, + CommonActionType | InteractiveWindowMessages, + BaseReduxActionPayload +>; -export type InteractiveReducerArg = ReducerArg; +export type InteractiveReducerArg = ReducerArg< + IMainState, + CommonActionType | InteractiveWindowMessages, + BaseReduxActionPayload +>; -export class IInteractiveActionMapping { - public [CommonActionType.FOCUS_INPUT]: InteractiveReducerFunc; - public [CommonActionType.RESTART_KERNEL]: InteractiveReducerFunc; - public [CommonActionType.SELECT_KERNEL]: InteractiveReducerFunc; - public [CommonActionType.SELECT_SERVER]: InteractiveReducerFunc; - public [CommonActionType.OPEN_SETTINGS]: InteractiveReducerFunc; - public [CommonActionType.INTERRUPT_KERNEL]: InteractiveReducerFunc; - public [CommonActionType.EXPORT]: InteractiveReducerFunc; - public [CommonActionType.SAVE]: InteractiveReducerFunc; - public [CommonActionType.UNDO]: InteractiveReducerFunc; - public [CommonActionType.REDO]: InteractiveReducerFunc; - public [CommonActionType.SHOW_DATA_VIEWER]: InteractiveReducerFunc; - public [CommonActionType.DELETE_CELL]: InteractiveReducerFunc; - public [CommonActionType.LINK_CLICK]: InteractiveReducerFunc; - public [CommonActionType.SHOW_PLOT]: InteractiveReducerFunc; - public [CommonActionType.TOGGLE_INPUT_BLOCK]: InteractiveReducerFunc; - public [CommonActionType.GOTO_CELL]: InteractiveReducerFunc; - public [CommonActionType.COPY_CELL_CODE]: InteractiveReducerFunc; - public [CommonActionType.GATHER_CELL]: InteractiveReducerFunc; - public [CommonActionType.EDIT_CELL]: InteractiveReducerFunc; - public [CommonActionType.SUBMIT_INPUT]: InteractiveReducerFunc; - public [CommonActionType.DELETE_ALL_CELLS]: InteractiveReducerFunc; - public [CommonActionType.EXPAND_ALL]: InteractiveReducerFunc; - public [CommonActionType.COLLAPSE_ALL]: InteractiveReducerFunc; - public [CommonActionType.EDITOR_LOADED]: InteractiveReducerFunc; - public [CommonActionType.SCROLL]: InteractiveReducerFunc; - public [CommonActionType.CLICK_CELL]: InteractiveReducerFunc; - public [CommonActionType.UNFOCUS_CELL]: InteractiveReducerFunc; - public [CommonActionType.UNMOUNT]: InteractiveReducerFunc; +type InteractiveWindowReducerFunctions = { + [P in keyof T]: T[P] extends never | undefined ? InteractiveReducerFunc : InteractiveReducerFunc; +}; - // Messages from the extension - public [IncomingMessageActions.STARTCELL]: InteractiveReducerFunc; - public [IncomingMessageActions.FINISHCELL]: InteractiveReducerFunc; - public [IncomingMessageActions.UPDATECELL]: InteractiveReducerFunc; - public [IncomingMessageActions.ACTIVATE]: InteractiveReducerFunc; - public [IncomingMessageActions.RESTARTKERNEL]: InteractiveReducerFunc; - public [IncomingMessageActions.GETCSSRESPONSE]: InteractiveReducerFunc; - public [IncomingMessageActions.MONACOREADY]: InteractiveReducerFunc; - public [IncomingMessageActions.GETMONACOTHEMERESPONSE]: InteractiveReducerFunc; - public [IncomingMessageActions.GETALLCELLS]: InteractiveReducerFunc; - public [IncomingMessageActions.EXPANDALL]: InteractiveReducerFunc; - public [IncomingMessageActions.COLLAPSEALL]: InteractiveReducerFunc; - public [IncomingMessageActions.DELETEALLCELLS]: InteractiveReducerFunc; - public [IncomingMessageActions.STARTPROGRESS]: InteractiveReducerFunc; - public [IncomingMessageActions.STOPPROGRESS]: InteractiveReducerFunc; - public [IncomingMessageActions.UPDATESETTINGS]: InteractiveReducerFunc; - public [IncomingMessageActions.STARTDEBUGGING]: InteractiveReducerFunc; - public [IncomingMessageActions.STOPDEBUGGING]: InteractiveReducerFunc; - public [IncomingMessageActions.SCROLLTOCELL]: InteractiveReducerFunc; - public [IncomingMessageActions.UPDATEKERNEL]: InteractiveReducerFunc; - public [IncomingMessageActions.LOCINIT]: InteractiveReducerFunc; -} +export type IInteractiveActionMapping = InteractiveWindowReducerFunctions & + InteractiveWindowReducerFunctions; diff --git a/src/datascience-ui/history-react/redux/reducers/creation.ts b/src/datascience-ui/history-react/redux/reducers/creation.ts index 029f204b5b96..4586967ea4a2 100644 --- a/src/datascience-ui/history-react/redux/reducers/creation.ts +++ b/src/datascience-ui/history-react/redux/reducers/creation.ts @@ -6,9 +6,9 @@ import { InteractiveWindowMessages } from '../../../../client/datascience/intera import { ICell, IDataScienceExtraSettings } from '../../../../client/datascience/types'; import { removeLinesFromFrontAndBack } from '../../../common'; import { createCellVM, extractInputText, ICellViewModel, IMainState } from '../../../interactive-common/mainState'; -import { createPostableAction } from '../../../interactive-common/redux/postOffice'; +import { postActionToExtension } from '../../../interactive-common/redux/helpers'; import { Helpers } from '../../../interactive-common/redux/reducers/helpers'; -import { ICellAction } from '../../../interactive-common/redux/reducers/types'; +import { IAddCellAction, ICellAction } from '../../../interactive-common/redux/reducers/types'; import { InteractiveReducerArg } from '../mapping'; export namespace Creation { @@ -100,19 +100,24 @@ export namespace Creation { } export function startCell(arg: InteractiveReducerArg): IMainState { - if (isCellSupported(arg.prevState, arg.payload)) { + if (isCellSupported(arg.prevState, arg.payload.data)) { const result = Helpers.updateOrAdd(arg, prepareCellVM); - if (result.cellVMs.length > arg.prevState.cellVMs.length && arg.payload.id !== Identifiers.EditCellId) { + if ( + result.cellVMs.length > arg.prevState.cellVMs.length && + arg.payload.data.id !== Identifiers.EditCellId + ) { const cellVM = result.cellVMs[result.cellVMs.length - 1]; // We're adding a new cell here. Tell the intellisense engine we have a new cell - arg.queueAction( - createPostableAction(InteractiveWindowMessages.AddCell, { - fullText: extractInputText(cellVM, result.settings), - currentText: cellVM.inputBlockText, - cell: cellVM.cell - }) - ); + postActionToExtension(arg, InteractiveWindowMessages.UpdateModel, { + source: 'user', + kind: 'add', + oldDirty: arg.prevState.dirty, + newDirty: true, + cell: cellVM.cell, + fullText: extractInputText(cellVM, result.settings), + currentText: cellVM.inputBlockText + }); } return result; @@ -121,38 +126,42 @@ export namespace Creation { } export function updateCell(arg: InteractiveReducerArg): IMainState { - if (isCellSupported(arg.prevState, arg.payload)) { + if (isCellSupported(arg.prevState, arg.payload.data)) { return Helpers.updateOrAdd(arg, prepareCellVM); } return arg.prevState; } export function finishCell(arg: InteractiveReducerArg): IMainState { - if (isCellSupported(arg.prevState, arg.payload)) { + if (isCellSupported(arg.prevState, arg.payload.data)) { return Helpers.updateOrAdd(arg, prepareCellVM); } return arg.prevState; } - export function deleteAllCells(arg: InteractiveReducerArg): IMainState { + export function deleteAllCells(arg: InteractiveReducerArg): IMainState { // Send messages to other side to indicate the deletes - arg.queueAction(createPostableAction(InteractiveWindowMessages.DeleteAllCells)); + postActionToExtension(arg, InteractiveWindowMessages.DeleteAllCells); return { ...arg.prevState, cellVMs: [], - undoStack: Helpers.pushStack(arg.prevState.undoStack, arg.prevState.cellVMs), - selectedCellId: undefined, - focusedCellId: undefined + undoStack: Helpers.pushStack(arg.prevState.undoStack, arg.prevState.cellVMs) }; } export function deleteCell(arg: InteractiveReducerArg): IMainState { - const index = arg.prevState.cellVMs.findIndex(c => c.cell.id === arg.payload.cellId); - if (index >= 0 && arg.payload.cellId) { + const index = arg.prevState.cellVMs.findIndex(c => c.cell.id === arg.payload.data.cellId); + if (index >= 0 && arg.payload.data.cellId) { // Send messages to other side to indicate the delete - arg.queueAction(createPostableAction(InteractiveWindowMessages.DeleteCell)); - arg.queueAction(createPostableAction(InteractiveWindowMessages.RemoveCell, { id: arg.payload.cellId })); + postActionToExtension(arg, InteractiveWindowMessages.UpdateModel, { + source: 'user', + kind: 'remove', + index, + oldDirty: arg.prevState.dirty, + newDirty: true, + cell: arg.prevState.cellVMs[index].cell + }); const newVMs = arg.prevState.cellVMs.filter((_c, i) => i !== index); return { diff --git a/src/datascience-ui/history-react/redux/reducers/effects.ts b/src/datascience-ui/history-react/redux/reducers/effects.ts index a1e667911f8b..408e9547de9f 100644 --- a/src/datascience-ui/history-react/redux/reducers/effects.ts +++ b/src/datascience-ui/history-react/redux/reducers/effects.ts @@ -6,7 +6,7 @@ import { IScrollToCell } from '../../../../client/datascience/interactive-common import { CssMessages } from '../../../../client/datascience/messages'; import { IDataScienceExtraSettings } from '../../../../client/datascience/types'; import { IMainState } from '../../../interactive-common/mainState'; -import { createPostableAction } from '../../../interactive-common/redux/postOffice'; +import { postActionToExtension } from '../../../interactive-common/redux/helpers'; import { Helpers } from '../../../interactive-common/redux/reducers/helpers'; import { ICellAction, IScrollAction } from '../../../interactive-common/redux/reducers/types'; import { computeEditorOptions } from '../../../react-common/settingsReactSide'; @@ -41,9 +41,9 @@ export namespace Effects { } export function toggleInputBlock(arg: InteractiveReducerArg): IMainState { - if (arg.payload.cellId) { + if (arg.payload.data.cellId) { const newVMs = [...arg.prevState.cellVMs]; - const index = arg.prevState.cellVMs.findIndex(c => c.cell.id === arg.payload.cellId); + const index = arg.prevState.cellVMs.findIndex(c => c.cell.id === arg.payload.data.cellId); const oldVM = arg.prevState.cellVMs[index]; newVMs[index] = Creation.alterCellVM({ ...oldVM }, arg.prevState.settings, true, !oldVM.inputBlockOpen); return { @@ -56,7 +56,7 @@ export namespace Effects { export function updateSettings(arg: InteractiveReducerArg): IMainState { // String arg should be the IDataScienceExtraSettings - const newSettingsJSON = JSON.parse(arg.payload); + const newSettingsJSON = JSON.parse(arg.payload.data); const newSettings = newSettingsJSON; const newEditorOptions = computeEditorOptions(newSettings); const newFontFamily = newSettings.extraSettings @@ -74,8 +74,8 @@ export namespace Effects { ) { const knownDark = Helpers.computeKnownDark(newSettings); // User changed the current theme. Rerender - arg.queueAction(createPostableAction(CssMessages.GetCssRequest, { isDark: knownDark })); - arg.queueAction(createPostableAction(CssMessages.GetMonacoThemeRequest, { isDark: knownDark })); + postActionToExtension(arg, CssMessages.GetCssRequest, { isDark: knownDark }); + postActionToExtension(arg, CssMessages.GetMonacoThemeRequest, { isDark: knownDark }); } // Update our input cell state if the user changed this setting @@ -105,7 +105,7 @@ export namespace Effects { export function scrollToCell(arg: InteractiveReducerArg): IMainState { // Up the scroll count on the necessary cell - const index = arg.prevState.cellVMs.findIndex(c => c.cell.id === arg.payload.id); + const index = arg.prevState.cellVMs.findIndex(c => c.cell.id === arg.payload.data.id); if (index >= 0) { const newVMs = [...arg.prevState.cellVMs]; @@ -124,13 +124,13 @@ export namespace Effects { export function scrolled(arg: InteractiveReducerArg): IMainState { return { ...arg.prevState, - isAtBottom: arg.payload.isAtBottom + isAtBottom: arg.payload.data.isAtBottom }; } export function clickCell(arg: InteractiveReducerArg): IMainState { if ( - arg.payload.cellId === Identifiers.EditCellId && + arg.payload.data.cellId === Identifiers.EditCellId && arg.prevState.editCellVM && !arg.prevState.editCellVM.focused ) { @@ -156,7 +156,7 @@ export namespace Effects { export function unfocusCell(arg: InteractiveReducerArg): IMainState { if ( - arg.payload.cellId === Identifiers.EditCellId && + arg.payload.data.cellId === Identifiers.EditCellId && arg.prevState.editCellVM && arg.prevState.editCellVM.focused ) { diff --git a/src/datascience-ui/history-react/redux/reducers/execution.ts b/src/datascience-ui/history-react/redux/reducers/execution.ts index 4293ca944fec..1605cc79f8c5 100644 --- a/src/datascience-ui/history-react/redux/reducers/execution.ts +++ b/src/datascience-ui/history-react/redux/reducers/execution.ts @@ -11,7 +11,7 @@ import { CellState } from '../../../../client/datascience/types'; import { generateMarkdownFromCodeLines } from '../../../common'; import { createCellFrom } from '../../../common/cellFactory'; import { createCellVM, IMainState } from '../../../interactive-common/mainState'; -import { createPostableAction } from '../../../interactive-common/redux/postOffice'; +import { postActionToExtension } from '../../../interactive-common/redux/helpers'; import { Helpers } from '../../../interactive-common/redux/reducers/helpers'; import { ICodeAction } from '../../../interactive-common/redux/reducers/types'; import { InteractiveReducerArg } from '../mapping'; @@ -24,16 +24,13 @@ export namespace Execution { const cells = arg.prevState.undoStack[arg.prevState.undoStack.length - 1]; const undoStack = arg.prevState.undoStack.slice(0, arg.prevState.undoStack.length - 1); const redoStack = Helpers.pushStack(arg.prevState.redoStack, arg.prevState.cellVMs); - const selected = cells.findIndex(c => c.selected); - arg.queueAction(createPostableAction(InteractiveWindowMessages.Undo)); + postActionToExtension(arg, InteractiveWindowMessages.Undo); return { ...arg.prevState, cellVMs: cells, undoStack: undoStack, redoStack: redoStack, - skipNextScroll: true, - selectedCellId: selected >= 0 ? cells[selected].cell.id : undefined, - focusedCellId: selected >= 0 && cells[selected].focused ? cells[selected].cell.id : undefined + skipNextScroll: true }; } @@ -46,16 +43,13 @@ export namespace Execution { const cells = arg.prevState.redoStack[arg.prevState.redoStack.length - 1]; const redoStack = arg.prevState.redoStack.slice(0, arg.prevState.redoStack.length - 1); const undoStack = Helpers.pushStack(arg.prevState.undoStack, arg.prevState.cellVMs); - const selected = cells.findIndex(c => c.selected); - arg.queueAction(createPostableAction(InteractiveWindowMessages.Redo)); + postActionToExtension(arg, InteractiveWindowMessages.Redo); return { ...arg.prevState, cellVMs: cells, undoStack: undoStack, redoStack: redoStack, - skipNextScroll: true, - selectedCellId: selected >= 0 ? cells[selected].cell.id : undefined, - focusedCellId: selected >= 0 && cells[selected].focused ? cells[selected].cell.id : undefined + skipNextScroll: true }; } @@ -79,16 +73,16 @@ export namespace Execution { export function submitInput(arg: InteractiveReducerArg): IMainState { // noop if the submitted code is just a cell marker const matcher = new CellMatcher(arg.prevState.settings); - if (matcher.stripFirstMarker(arg.payload.code).length > 0 && arg.prevState.editCellVM) { + if (matcher.stripFirstMarker(arg.payload.data.code).length > 0 && arg.prevState.editCellVM) { // This should be from the edit cell VM. Copy it and change the cell id let newCell = cloneDeep(arg.prevState.editCellVM); // Change this editable cell to not editable. newCell.cell.state = CellState.executing; - newCell.cell.data.source = arg.payload.code; + newCell.cell.data.source = arg.payload.data.code; // Change type to markdown if necessary - const split = arg.payload.code.splitLines({ trim: false }); + const split = arg.payload.data.code.splitLines({ trim: false }); const firstLine = split[0]; if (matcher.isMarkdown(firstLine)) { newCell.cell.data = createCellFrom(newCell.cell.data, 'markdown'); @@ -115,12 +109,10 @@ export namespace Execution { // Send a message to execute this code if necessary. if (newCell.cell.state !== CellState.finished) { - arg.queueAction( - createPostableAction(InteractiveWindowMessages.SubmitNewCell, { - code: arg.payload.code, - id: newCell.cell.id - }) - ); + postActionToExtension(arg, InteractiveWindowMessages.SubmitNewCell, { + code: arg.payload.data.code, + id: newCell.cell.id + }); } // Stick in a new cell at the bottom that's editable and update our state diff --git a/src/datascience-ui/history-react/redux/reducers/index.ts b/src/datascience-ui/history-react/redux/reducers/index.ts index b122e5264cbf..a4e626310263 100644 --- a/src/datascience-ui/history-react/redux/reducers/index.ts +++ b/src/datascience-ui/history-react/redux/reducers/index.ts @@ -1,7 +1,8 @@ // Copyright (c) Microsoft Corporation. All rights reserved. // Licensed under the MIT License. 'use strict'; -import { IncomingMessageActions } from '../../../interactive-common/redux/postOffice'; +import { InteractiveWindowMessages } from '../../../../client/datascience/interactive-common/interactiveWindowTypes'; +import { CssMessages, SharedMessages } from '../../../../client/datascience/messages'; import { CommonEffects } from '../../../interactive-common/redux/reducers/commonEffects'; import { Kernel } from '../../../interactive-common/redux/reducers/kernel'; import { Transfer } from '../../../interactive-common/redux/reducers/transfer'; @@ -12,20 +13,18 @@ import { Effects } from './effects'; import { Execution } from './execution'; // The list of reducers. 1 per message/action. -export const reducerMap: IInteractiveActionMapping = { +export const reducerMap: Partial = { // State updates [CommonActionType.RESTART_KERNEL]: Kernel.restartKernel, [CommonActionType.INTERRUPT_KERNEL]: Kernel.interruptKernel, - [CommonActionType.SELECT_KERNEL]: Kernel.selectKernel, + [InteractiveWindowMessages.SelectKernel]: Kernel.selectKernel, [CommonActionType.SELECT_SERVER]: Kernel.selectJupyterURI, [CommonActionType.OPEN_SETTINGS]: CommonEffects.openSettings, [CommonActionType.EXPORT]: Transfer.exportCells, [CommonActionType.SAVE]: Transfer.save, [CommonActionType.SHOW_DATA_VIEWER]: Transfer.showDataViewer, [CommonActionType.DELETE_CELL]: Creation.deleteCell, - [CommonActionType.UNDO]: Execution.undo, - [CommonActionType.REDO]: Execution.redo, - [CommonActionType.SHOW_PLOT]: Transfer.showPlot, + [InteractiveWindowMessages.ShowPlot]: Transfer.showPlot, [CommonActionType.LINK_CLICK]: Transfer.linkClick, [CommonActionType.GOTO_CELL]: Transfer.gotoCell, [CommonActionType.TOGGLE_INPUT_BLOCK]: Effects.toggleInputBlock, @@ -33,9 +32,7 @@ export const reducerMap: IInteractiveActionMapping = { [CommonActionType.GATHER_CELL]: Transfer.gather, [CommonActionType.EDIT_CELL]: Transfer.editCell, [CommonActionType.SUBMIT_INPUT]: Execution.submitInput, - [CommonActionType.DELETE_ALL_CELLS]: Creation.deleteAllCells, - [CommonActionType.EXPAND_ALL]: Effects.expandAll, - [CommonActionType.COLLAPSE_ALL]: Effects.collapseAll, + [InteractiveWindowMessages.ExpandAll]: Effects.expandAll, [CommonActionType.EDITOR_LOADED]: Transfer.started, [CommonActionType.SCROLL]: Effects.scrolled, [CommonActionType.CLICK_CELL]: Effects.clickCell, @@ -44,24 +41,26 @@ export const reducerMap: IInteractiveActionMapping = { [CommonActionType.FOCUS_INPUT]: CommonEffects.focusInput, // Messages from the webview (some are ignored) - [IncomingMessageActions.STARTCELL]: Creation.startCell, - [IncomingMessageActions.FINISHCELL]: Creation.finishCell, - [IncomingMessageActions.UPDATECELL]: Creation.updateCell, - [IncomingMessageActions.ACTIVATE]: CommonEffects.activate, - [IncomingMessageActions.RESTARTKERNEL]: Kernel.handleRestarted, - [IncomingMessageActions.GETCSSRESPONSE]: CommonEffects.handleCss, - [IncomingMessageActions.MONACOREADY]: CommonEffects.monacoReady, - [IncomingMessageActions.GETMONACOTHEMERESPONSE]: CommonEffects.monacoThemeChange, - [IncomingMessageActions.GETALLCELLS]: Transfer.getAllCells, - [IncomingMessageActions.EXPANDALL]: Effects.expandAll, - [IncomingMessageActions.COLLAPSEALL]: Effects.collapseAll, - [IncomingMessageActions.DELETEALLCELLS]: Creation.deleteAllCells, - [IncomingMessageActions.STARTPROGRESS]: CommonEffects.startProgress, - [IncomingMessageActions.STOPPROGRESS]: CommonEffects.stopProgress, - [IncomingMessageActions.UPDATESETTINGS]: Effects.updateSettings, - [IncomingMessageActions.STARTDEBUGGING]: Execution.startDebugging, - [IncomingMessageActions.STOPDEBUGGING]: Execution.stopDebugging, - [IncomingMessageActions.SCROLLTOCELL]: Effects.scrollToCell, - [IncomingMessageActions.UPDATEKERNEL]: Kernel.updateStatus, - [IncomingMessageActions.LOCINIT]: CommonEffects.handleLocInit + [InteractiveWindowMessages.Undo]: Execution.undo, + [InteractiveWindowMessages.Redo]: Execution.redo, + [InteractiveWindowMessages.StartCell]: Creation.startCell, + [InteractiveWindowMessages.FinishCell]: Creation.finishCell, + [InteractiveWindowMessages.UpdateCell]: Creation.updateCell, + [InteractiveWindowMessages.Activate]: CommonEffects.activate, + [InteractiveWindowMessages.RestartKernel]: Kernel.handleRestarted, + [CssMessages.GetCssResponse]: CommonEffects.handleCss, + [InteractiveWindowMessages.MonacoReady]: CommonEffects.monacoReady, + [CssMessages.GetMonacoThemeResponse]: CommonEffects.monacoThemeChange, + [InteractiveWindowMessages.GetAllCells]: Transfer.getAllCells, + [InteractiveWindowMessages.ExpandAll]: Effects.expandAll, + [InteractiveWindowMessages.CollapseAll]: Effects.collapseAll, + [InteractiveWindowMessages.DeleteAllCells]: Creation.deleteAllCells, + [InteractiveWindowMessages.StartProgress]: CommonEffects.startProgress, + [InteractiveWindowMessages.StopProgress]: CommonEffects.stopProgress, + [SharedMessages.UpdateSettings]: Effects.updateSettings, + [InteractiveWindowMessages.StartDebugging]: Execution.startDebugging, + [InteractiveWindowMessages.StopDebugging]: Execution.stopDebugging, + [InteractiveWindowMessages.ScrollToCell]: Effects.scrollToCell, + [InteractiveWindowMessages.UpdateKernel]: Kernel.updateStatus, + [SharedMessages.LocInit]: CommonEffects.handleLocInit }; diff --git a/src/datascience-ui/interactive-common/cellInput.tsx b/src/datascience-ui/interactive-common/cellInput.tsx index 744fba7dcde6..770e2e1ce6b8 100644 --- a/src/datascience-ui/interactive-common/cellInput.tsx +++ b/src/datascience-ui/interactive-common/cellInput.tsx @@ -9,6 +9,7 @@ import * as React from 'react'; import { concatMultilineStringInput } from '../common'; import { IKeyboardEvent } from '../react-common/event'; +import { IMonacoModelContentChangeEvent } from '../react-common/monacoHelpers'; import { Code } from './code'; import { InputHistory } from './inputHistory'; import { ICellViewModel, IFont } from './mainState'; @@ -17,6 +18,7 @@ import { Markdown } from './markdown'; // tslint:disable-next-line: no-require-importss interface ICellInputProps { cellVM: ICellViewModel; + codeVersion: number; codeTheme: string; testMode?: boolean; history: InputHistory | undefined; @@ -26,8 +28,9 @@ interface ICellInputProps { editorMeasureClassName?: string; showLineNumbers?: boolean; font: IFont; + disableUndoStack: boolean; focusPending: number; - onCodeChange(changes: monacoEditor.editor.IModelContentChange[], cellId: string, modelId: string): void; + onCodeChange(e: IMonacoModelContentChangeEvent): void; onCodeCreated(code: string, file: string, cellId: string, modelId: string): void; openLink(uri: monacoEditor.Uri): void; keyDown?(cellId: string, e: IKeyboardEvent): void; @@ -97,7 +100,7 @@ export class CellInput extends React.Component { readOnly={!this.props.cellVM.editable} showWatermark={this.props.showWatermark} ref={this.codeRef} - onChange={this.onCodeChange} + onChange={this.props.onCodeChange} onCreated={this.onCodeCreated} outermostParentClass="cell-wrapper" monacoTheme={this.props.monacoTheme} @@ -111,6 +114,9 @@ export class CellInput extends React.Component { showLineNumbers={this.props.showLineNumbers} useQuickEdit={this.props.cellVM.useQuickEdit} font={this.props.font} + disableUndoStack={this.props.disableUndoStack} + version={this.props.codeVersion} + previousCode={this.props.cellVM.uncommittedText} focusPending={this.props.focusPending} />
@@ -130,7 +136,7 @@ export class CellInput extends React.Component { markdown={source} codeTheme={this.props.codeTheme} testMode={this.props.testMode ? true : false} - onChange={this.onCodeChange} + onChange={this.props.onCodeChange} onCreated={this.onCodeCreated} outermostParentClass="cell-wrapper" hasFocus={this.props.cellVM.focused} @@ -144,6 +150,9 @@ export class CellInput extends React.Component { ref={this.markdownRef} useQuickEdit={false} font={this.props.font} + disableUndoStack={this.props.disableUndoStack} + version={this.props.codeVersion} + previousMarkdown={this.props.cellVM.uncommittedText} /> ); @@ -182,10 +191,6 @@ export class CellInput extends React.Component { } }; - private onCodeChange = (changes: monacoEditor.editor.IModelContentChange[], modelId: string) => { - this.props.onCodeChange(changes, this.props.cellVM.cell.id, modelId); - }; - private onCodeCreated = (code: string, modelId: string) => { this.props.onCodeCreated(code, this.props.cellVM.cell.file, this.props.cellVM.cell.id, modelId); }; diff --git a/src/datascience-ui/interactive-common/code.tsx b/src/datascience-ui/interactive-common/code.tsx index 152cfb28a7b1..eebad69c56ef 100644 --- a/src/datascience-ui/interactive-common/code.tsx +++ b/src/datascience-ui/interactive-common/code.tsx @@ -7,11 +7,14 @@ import * as React from 'react'; import { InputHistory } from '../interactive-common/inputHistory'; import { IKeyboardEvent } from '../react-common/event'; import { getLocString } from '../react-common/locReactSide'; +import { IMonacoModelContentChangeEvent } from '../react-common/monacoHelpers'; import { Editor } from './editor'; import { CursorPos, IFont } from './mainState'; export interface ICodeProps { code: string; + previousCode: string | undefined; + version: number; codeTheme: string; testMode: boolean; readOnly: boolean; @@ -25,10 +28,11 @@ export interface ICodeProps { useQuickEdit?: boolean; font: IFont; hasFocus: boolean; - cursorPos: CursorPos; + cursorPos: CursorPos | monacoEditor.IPosition; + disableUndoStack: boolean; focusPending: number; onCreated(code: string, modelId: string): void; - onChange(changes: monacoEditor.editor.IModelContentChange[], modelId: string): void; + onChange(e: IMonacoModelContentChangeEvent): void; openLink(uri: monacoEditor.Uri): void; keyDown?(e: IKeyboardEvent): void; focused?(): void; @@ -84,6 +88,9 @@ export class Code extends React.Component { showLineNumbers={this.props.showLineNumbers} useQuickEdit={this.props.useQuickEdit} font={this.props.font} + disableUndoStack={this.props.disableUndoStack} + version={this.props.version} + previousContent={this.props.previousCode} />
{this.getWatermarkString()} @@ -114,13 +121,10 @@ export class Code extends React.Component { return getLocString('DataScience.inputWatermark', 'Type code here and press shift-enter to run'); }; - private onModelChanged = ( - changes: monacoEditor.editor.IModelContentChange[], - model: monacoEditor.editor.ITextModel - ) => { - if (!this.props.readOnly && model) { - this.setState({ allowWatermark: model.getValueLength() === 0 }); + private onModelChanged = (e: IMonacoModelContentChangeEvent) => { + if (!this.props.readOnly && e.model) { + this.setState({ allowWatermark: e.model.getValueLength() === 0 }); } - this.props.onChange(changes, model.id); + this.props.onChange(e); }; } diff --git a/src/datascience-ui/interactive-common/editor.tsx b/src/datascience-ui/interactive-common/editor.tsx index cb3426017697..bf13ed3a488e 100644 --- a/src/datascience-ui/interactive-common/editor.tsx +++ b/src/datascience-ui/interactive-common/editor.tsx @@ -7,12 +7,15 @@ import * as React from 'react'; import { noop } from '../../client/common/utils/misc'; import { IKeyboardEvent } from '../react-common/event'; import { MonacoEditor } from '../react-common/monacoEditor'; +import { IMonacoModelContentChangeEvent } from '../react-common/monacoHelpers'; import { InputHistory } from './inputHistory'; import { CursorPos, IFont } from './mainState'; // tslint:disable-next-line: import-name export interface IEditorProps { content: string; + previousContent: string | undefined; + version: number; codeTheme: string; readOnly: boolean; testMode: boolean; @@ -26,84 +29,54 @@ export interface IEditorProps { useQuickEdit?: boolean; font: IFont; hasFocus: boolean; - cursorPos: CursorPos; + cursorPos: CursorPos | monacoEditor.IPosition; + disableUndoStack: boolean; onCreated(code: string, modelId: string): void; - onChange(changes: monacoEditor.editor.IModelContentChange[], model: monacoEditor.editor.ITextModel): void; + onChange(e: IMonacoModelContentChangeEvent): void; openLink(uri: monacoEditor.Uri): void; keyDown?(e: IKeyboardEvent): void; focused?(): void; unfocused?(): void; } -interface IEditorState { - editor: monacoEditor.editor.IStandaloneCodeEditor | undefined; - model: monacoEditor.editor.ITextModel | null; - forceMonaco: boolean; -} - -export class Editor extends React.Component { +export class Editor extends React.Component { private subscriptions: monacoEditor.IDisposable[] = []; private lastCleanVersionId: number = 0; private monacoRef: React.RefObject = React.createRef(); constructor(prop: IEditorProps) { super(prop); - this.state = { editor: undefined, model: null, forceMonaco: false }; } public componentWillUnmount = () => { this.subscriptions.forEach(d => d.dispose()); }; - public componentDidUpdate(prevProps: IEditorProps, prevState: IEditorState) { - if (this.props.hasFocus && (!prevProps.hasFocus || !prevState.editor)) { + public componentDidUpdate(prevProps: IEditorProps) { + if (this.props.hasFocus && !prevProps.hasFocus) { this.giveFocus(this.props.cursorPos); } } public render() { const classes = this.props.readOnly ? 'editor-area' : 'editor-area editor-area-editable'; - const renderEditor = - this.state.forceMonaco || this.props.useQuickEdit === undefined || this.props.useQuickEdit === false - ? this.renderMonacoEditor - : this.renderQuickEditor; + const renderEditor = this.renderMonacoEditor; return
{renderEditor()}
; } - public giveFocus(cursorPos: CursorPos) { - const readOnly = this.props.readOnly; - if (this.state.editor && !readOnly) { - this.state.editor.focus(); - } - if (this.state.editor && cursorPos !== CursorPos.Current) { - const current = this.state.editor.getPosition(); - const lineNumber = cursorPos === CursorPos.Top ? 1 : this.state.editor.getModel()!.getLineCount(); - const column = current && current.lineNumber === lineNumber ? current.column : 1; - this.state.editor.setPosition({ lineNumber, column }); + public giveFocus(cursorPos: CursorPos | monacoEditor.IPosition) { + if (this.monacoRef.current) { + this.monacoRef.current.giveFocus(cursorPos); } } public getContents(): string { - if (this.state.model) { - return this.state.model.getValue().replace(/\r/g, ''); + if (this.monacoRef.current) { + return this.monacoRef.current.getContents(); } return ''; } - private renderQuickEditor = (): JSX.Element => { - const readOnly = this.props.readOnly; - return ( -