A different way of fixing escaping #14186
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rchiodo
requested review from
DavidKutu,
DonJayamanne,
greazer,
IanMatthewHuff and
joyceerhl
rchiodo
commented
Sep 30, 2020
| const n = parseInt(num, 10); | ||
| const newLine = offset + n - 1; | ||
| return `${prefix}<a href='file://${match[0]}?line=${newLine}'>${newLine + 1}</a>`; | ||
| return `${_escape(prefix)}<a href='file://${match[0]}?line=${newLine}'>${newLine + 1}</a>${_escape( |
There was a problem hiding this comment.
This is also changing the output that would normally be there so I made it transient. It will only show up in while the interactive window is open. Exporting to a notebook won't show it anymore (which I think is probably what we want)
There was a problem hiding this comment.
Yeah, that sounds better as it's specific to us.
rchiodo
commented
Sep 30, 2020
| @@ -1237,7 +1228,10 @@ export class JupyterNotebookBase implements INotebook { | |||
| ) { | |||
| // Check our length on text output | |||
| if (msg.content.data && msg.content.data.hasOwnProperty('text/plain')) { | |||
| msg.content.data['text/plain'] = escape(trimFunc(msg.content.data['text/plain'] as string)); | |||
| msg.content.data['text/plain'] = splitMultilineString( | |||
There was a problem hiding this comment.
The other spot where output was different is saving as an array instead of a single string. Jupyter makes stuff into arrays.
There was a problem hiding this comment.
This seems ok, but I thought that Jupyter did either a string array or a single string. Does it have to be the string array?
There was a problem hiding this comment.
Should be fine I guess. If it's single string the split won't do anything.
rchiodo
commented
Sep 30, 2020
| @@ -328,7 +330,7 @@ export class CellOutput extends React.Component<ICellOutputProps> { | |||
| // tslint:disable-next-line: no-any | |||
| const text = (input as any)['text/plain']; | |||
| input = { | |||
| 'text/html': text // XML tags should have already been escaped. | |||
| 'text/html': lodashEscape(text) | |||
There was a problem hiding this comment.
Now the escaping happens when we format the output. Should have done it this way from the beginning.
Codecov Report
@@ Coverage Diff @@
## main #14186 +/- ##
==========================================
- Coverage 59.94% 59.93% -0.01%
==========================================
Files 705 706 +1
Lines 39101 39182 +81
Branches 5667 5683 +16
==========================================
+ Hits 23440 23485 +45
- Misses 14429 14457 +28
- Partials 1232 1240 +8
Continue to review full report at Codecov.
|
DavidKutu
approved these changes
Oct 1, 2020
| @@ -880,6 +881,151 @@ df.head()`; | |||
| verifyHtmlOnCell(ne.mount.wrapper, 'NativeCell', `3`, 2); | |||
| }); | |||
|
|
|||
| runMountedTest('Roundtrip with jupyter', async () => { | |||
IanMatthewHuff
approved these changes
Oct 1, 2020
|
Kudos, SonarCloud Quality Gate passed!
|
rchiodo
added a commit
that referenced
this pull request
Oct 1, 2020
* Move escaping to just output * Add some tests to verify round tripping * Fixup test for rountrip and make roundtripping actually work * Add news entry * Add to manual test file * Fix streaming problem and add more to the test * Fix traceback unit test * Fix problem caught by functional tests :) * Another functional test catch
rchiodo
added a commit
that referenced
this pull request
Oct 1, 2020
* A different way of fixing escaping (#14186) * Move escaping to just output * Add some tests to verify round tripping * Fixup test for rountrip and make roundtripping actually work * Add news entry * Add to manual test file * Fix streaming problem and add more to the test * Fix traceback unit test * Fix problem caught by functional tests :) * Another functional test catch * Update changelog
rchiodo
added a commit
that referenced
this pull request
Oct 1, 2020
* Add an optional onUpdated event to the iterator returned by ILocator.iterEnvs(). (#13950) In order to ensure that ILocator.iterEnvs() can finish as fast as possible, we add a side-channel event for each iteration. This event fires any time an already-yielded env info object is updated (e.g. reduced/merged or resolved/completed). The update only relates to operations triggered by that particular iteration. ILocator.onChanged remains separate and only relates to when the locators finds a new env, notices one was removed, or that one was otherwise fundamentally changed. There are two similar approaches we could take for this update event. Either we added the event as a property of IPythonEnvsIterator or we change the return value of ILocator.iterEnvs() to be a 2-tuple ([PythonEnvsIterator, Event<PythonEnvUpdatedEvent>]). We took the property approach since most of the time callers won't need to worry about the update events. * fully bump raw kernel (#13941) Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Virtualenvwrapper locator (#13895) * virtualenvwrapper locator * skip os-specific tests * Test getDefaultVirtualenvwrapperDir * Update src/client/pythonEnvironments/discovery/locators/services/virtualenvwrapperLocator.ts Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * add pathExists check to WORKON_HOME dir * Change how the locator works + move util func out * Fix functional tests * utils unit tests * Update locator tests * Stub getUserHomeDir instead of getDefaultetc Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * Clean up eslint errors (#13951) * Clean up eslint errors * Fix linting and address comments. * Add a legacy DI adapter for the Python envs component. (#13858) This allows us to start using the new discovery code in the extension. * Extension api for DataScience (#13923) * Extension API * Api for DS * Fix installing ipykernel into interpreters for raw kernel (#13959) * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Extension API to return path to debugpy for DS (#13973) * Use the component adapter in the extension. (#13869) This allows us to start using the new discovery code in the extension. The key thing is to be careful not to regress in the available functionality. So for now we have disabled use of the component. When we feel comfortable with it we can enable it by setting the default for `ComponentAdapter.enabled` to `true`. This PR involves small fixes to a large number of files due to API changes. The adapter is actually used (injected) in the following files: src/client/interpreter/interpreterService.ts src/client/interpreter/helpers.ts src/client/pythonEnvironments/discovery/locators/index.ts src/client/pythonEnvironments/discovery/locators/services/condaService.ts src/client/pythonEnvironments/discovery/locators/services/windowsStoreInterpreter.ts * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * update test plan for ipykernel (python repo) (#13993) * Find interpreter based on hash in kernelspec of nb metadata (#13856) * Pyenv locator (#13996) * Pyenv locator * Skip tests per platform * Wrong pyenv path order in ternary * Add description * Autoformat venv locator * Revert "Autoformat venv locator" This reverts commit 5c8c4ab. * Add links * Windows-specific fixes * Typo * merge * Revert "merge" This reverts commit ee0cad9. * Remove cwd while running using isolate script (#14014) * Remove cwd while running using isolate script * Run inplace. * Fix formatting * Upgrade isort to 5.5.3 (#14035) * Modify environment info worker to support new type and to work with resolver (#13997) * Modify environment info worker to support new environment type * Do not replace existing fields and return a new object * Modify cache to carry deferred instead * Change worker to return interpreter information instead * Handle error and rename * Fix bug with interpreterInfo.py * Code reviews * Rename old PythonEnvInfo type to InterpreterInfoJson * Remove cross process lock (#14036) * Add API to get language server from external extensions (#14021) * Ideas for lang server API with python extension * Working with new API * Minimize API surface * Fix up tests for intellisense * Fix unit tests * Put back custom editor service * Remove unnecessary changes for service registry * Code review feedback * Move connection type to types file * Fix failing virtualenvwrapper Windows unit tests (#14012) * Fix tests * Typo Co-authored-by: Karthik Nadig <kanadig@microsoft.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * make sure that we run our show code when a custom editor is shown (#14022) * make sure that we run our show code when a custom editor is shown * Trigger notebook load in the constructor, not show Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Rely on AppInsights API for exceptions (#13878) * Fix path * Actually fix settings * Add news * Add test * Format * Suppress 'jediEnabled' removal * Drop survey first launch threshold * Telemetry for exceptions * Remove custom exception telemetry * Remove unused * Fix HTML escaping to match what Jupyter does (#14038) * Basic idea * add some functional tests * Add news entry * Fix functional tests * Have dependabot ignore pytest and py (#14039) Otherwise it doesn't understand the Python 2.7 restrictions for those dependencies. * Added environments reducer (#13953) * Add environments reducer * Added tests * Use path.join to construct paths * Code reviews * Correct dummy implementations and adjust tests * Modify resolveEnv() * Rename to a general parentLocator * Merge release back to master (#14059) * Port ipykernel install fix to release (#13975) * Fix installing ipykernel into interpreters for raw kernel (#13959) * update news Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Merge in changes to release (#13976) * Up release version for new release (#13928) * Up release version * Update changelog * Update changelog * Workaround test issue (#13930) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * More tests failing with gpu error (#13935) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * Two more version changes * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Delete news file * Port two fixes to the release branch (#13995) * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * Fixup changelog Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> * add jedi-language-server to 3rd party notices (#13977) * add jedi-language-server to 3rd party notices * move license from distribution to repository file * disable test_discover_complex_default and (#14024) test_discover_complex_doctest * Upgrade isort to 5.5.3 (#14035) (#14037) * prepare release (#14042) * fixed annoying warnings (#14049) * update version Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Rich Chiodo <rchiodo@users.noreply.github.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> * Fix object sort order in tools tests (#14050) * Fix object sort order in tools tests * rebase with main * Fix formatting * Update ubuntu images for GPU issue fix (#14064) * Update ubuntu images for GPU issue fix * Try with ubuntu-20.04 * Added environments resolver (#14019) * Modify environment info worker to support new environment type * Change worker to return interpreter information instead * Modify resolveEnv() * Code reviews * Code reviews * Move stuff * Add Windows unit tests to the PR pipeline (#14106) * Do not opt users out of the insiders program if they have a stable version installed (#14091) * Remove code that opts users out of the insiders program * News entry * Code reviews * Do not quote isolated in exec module (#14108) * Do not quote isolated in exec module * Revert "Do not quote isolated in exec module" This reverts commit b9fa04c. * Revert "IPyKernel install issue with windows paths (#13667)" This reverts commit 23725ab. * Update cell output and metadata using Edit API (#13737) * Fix unit test broken by recent revert (#14122) * revert VSC_PYTHON_CI_TEST_VSC_CHANNEL changes (#14109) * revert VSC_PYTHON_CI_TEST_VSC_CHANNEL changes * Revert "revert VSC_PYTHON_CI_TEST_VSC_CHANNEL changes" This reverts commit 6ed8f61. * Remove force 1.48 on Test VSC channel * Merge environment and compare environments (#14026) * Add some heuristic functions * Support merging. * Clean up identifier code * Tweaks and fixes * More clean up. * versions tweak * Address comments * more comments * Fix merge issues. * Fix more merge issues. * Add PythonEnvInfo-related helpers. (#14051) This PR adds some basic helpers that we use in a subsequent PR. The following small drive-by changes are also included: * drop PythonEnvInfo.id property * make some internal helpers public * Escaping fix broke a number of things (#14145) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task * Auto assign datascience issues (#14147) * Change name of event used to activate extension (#14153) * Fixes to streamed output in native notebooks (#14158) For #13611 * Fixes to streamed output * added tests * Fixes to for escaping of output in native notebooks (#14159) * Look for `data science` labels when auto assigning issues (#14161) * Environment info cache class (#14065) * Add persistent storage external deps * PythonEnvInfoCache class + tests * Instantiate & initialize cache class in createAPI * Add extra test for flush() and initialize() * Env cache fixes: storage key + find() result check * Update src/client/pythonEnvironments/common/externalDependencies.ts Co-authored-by: Kartik Raj <karraj@microsoft.com> * Use areSameEnvironment in getEnv * Don't ping persistent storage for every initialize * No need to export CompleteEnvInfoFunction * PythonEnvInfoCache doc comment * Rename createGlobalPersistentStoreStub to get... * Preemptively drop id key (#14051) * Return deep copies * IPersistentStore wrapper around IPersistentState * Use correct areSameEnvironment + fix stub * Remove obsolete comment * getEnv -> filterEnvs * Remove stubbing of areSameEnvironment * Update areSameEnv * Move IPersistentStateFactory registration to registerForIOC * Revert "Move IPersistentStateFactory registration to registerForIOC" This reverts commit edc6ce5. * Don't instantiate nor initialize cache for now Co-authored-by: Kartik Raj <karraj@microsoft.com> * Update Component Governance notes in the release plan * Merge point release back into main (#14171) * Port ipykernel install fix to release (#13975) * Fix installing ipykernel into interpreters for raw kernel (#13959) * update news Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Merge in changes to release (#13976) * Up release version for new release (#13928) * Up release version * Update changelog * Update changelog * Workaround test issue (#13930) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * More tests failing with gpu error (#13935) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * Two more version changes * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Delete news file * Port two fixes to the release branch (#13995) * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * Fixup changelog Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> * add jedi-language-server to 3rd party notices (#13977) * add jedi-language-server to 3rd party notices * move license from distribution to repository file * disable test_discover_complex_default and (#14024) test_discover_complex_doctest * Upgrade isort to 5.5.3 (#14035) (#14037) * prepare release (#14042) * fixed annoying warnings (#14049) * Cherry pick to address path issues. (#14125) * Do not quote isolated in exec module (#14108) * Do not quote isolated in exec module * Revert "Do not quote isolated in exec module" This reverts commit b9fa04c. * Revert "IPyKernel install issue with windows paths (#13667)" This reverts commit 23725ab. * Fix unit test broken by recent revert (#14122) * Port escape fix to release branch (#14133) * Fix HTML escaping to match what Jupyter does (#14038) * Basic idea * add some functional tests * Add news entry * Fix functional tests * Update changelog * update version and changelog (#14139) * Escaping fix broke a number of things (#14145) (#14154) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task * remove leftovers * import pytest * stop skipping python tests Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Rich Chiodo <rchiodo@users.noreply.github.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * NB Convert 6.0 support for export (#14177) * Windows store locator (#14162) * Initial commit for windows store locator * More tests * Simplify locator * Tweaks * Test fixes * Fix tests * A different way of fixing escaping (#14186) * Move escaping to just output * Add some tests to verify round tripping * Fixup test for rountrip and make roundtripping actually work * Add news entry * Add to manual test file * Fix streaming problem and add more to the test * Fix traceback unit test * Fix problem caught by functional tests :) * Another functional test catch * Fixes to escaping of output in native notebooks (#14199) * Display description in kernel picker (#14196) * Bump node-fetch to 2.6.1 (#14181) * Bump pytest from 6.0.1 to 6.1.0 in /news (#14190) Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.0.1 to 6.1.0. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst) - [Commits](pytest-dev/pytest@6.0.1...6.1.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Messed up merge commits Co-authored-by: Eric Snow <eric.snow@microsoft.com> Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Kim-Adeline Miguel <51720070+kimadeline@users.noreply.github.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Graham Wheeler <gram@geekraver.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Joyce Er <joyceerhuiling@gmail.com> Co-authored-by: Mikhail Arkhipov <mikhaila@microsoft.com> Co-authored-by: Brett Cannon <brett@python.org> Co-authored-by: David Kutugata <dakutuga@microsoft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
rchiodo
added a commit
that referenced
this pull request
Oct 7, 2020
* Port ipykernel install fix to release (#13975) * Fix installing ipykernel into interpreters for raw kernel (#13959) * update news Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Merge in changes to release (#13976) * Up release version for new release (#13928) * Up release version * Update changelog * Update changelog * Workaround test issue (#13930) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * More tests failing with gpu error (#13935) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * Two more version changes * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Delete news file * Port two fixes to the release branch (#13995) * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * Fixup changelog Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> * add jedi-language-server to 3rd party notices (#13977) * add jedi-language-server to 3rd party notices * move license from distribution to repository file * disable test_discover_complex_default and (#14024) test_discover_complex_doctest * Upgrade isort to 5.5.3 (#14035) (#14037) * prepare release (#14042) * fixed annoying warnings (#14049) * Cherry pick to address path issues. (#14125) * Do not quote isolated in exec module (#14108) * Do not quote isolated in exec module * Revert "Do not quote isolated in exec module" This reverts commit b9fa04c. * Revert "IPyKernel install issue with windows paths (#13667)" This reverts commit 23725ab. * Fix unit test broken by recent revert (#14122) * Port escape fix to release branch (#14133) * Fix HTML escaping to match what Jupyter does (#14038) * Basic idea * add some functional tests * Add news entry * Fix functional tests * Update changelog * update version and changelog (#14139) * Escaping fix broke a number of things (#14145) (#14154) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task * Port NB Convert Fix to point release branch (#14200) * Port escape fix to release branch (#14202) * A different way of fixing escaping (#14186) * Move escaping to just output * Add some tests to verify round tripping * Fixup test for rountrip and make roundtripping actually work * Add news entry * Add to manual test file * Fix streaming problem and add more to the test * Fix traceback unit test * Fix problem caught by functional tests :) * Another functional test catch * Update changelog * Port interactive window export fix (#14232) * Select kernel based on metadata in notebook (#14217) (#14234) * Fix picking kernel based on metadata * Port more escape fixes to point release (#14242) * Fix two problems with escaping (#14228) * Fixup changelog * Port prune fix from main to release (#14243) * Remove unneeded cell keys when exporting (#14241) * Remove transient output when exporting from the interactive window * Add news entry * Update changelog * Merge fixes related to remembering interpreter (#14270) * update version, changelog and thrid party notices (#14280) * remove VSC_PYTHON_CI_TEST_VSC_CHANNEL and test skips * delete ipynb file * delete solved news files * delete more news Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Rich Chiodo <rchiodo@users.noreply.github.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com>
DonJayamanne
pushed a commit
that referenced
this pull request
Oct 30, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For #14182
The original fix I did for escaping broke the data stored in the ipynb. It should not be escaped there. The escaping should be happening in the UI.
I also added a test for round tripping to make sure this doesn't happen again (or for other reasons).
That test found some other spots where we were changing stuff unnecessarily.
package-lock.jsonhas been regenerated by runningnpm install(if dependencies have changed).