HTML escaped dictionary cannot open in the data viewer

[rchiodo]

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

1. Debug a file with this code

import numpy as np
import pandas as pd
from pandas import Series
dict2 = {'a> <a href="bobs"/>':[1, 2, 3], '<b':[4, 5, 6]} 
df6 = pd.DataFrame.from_dict(dict2) 
print(df6) # Set breakpoint here

2. Attempt to view the df6 or dict2 in the data viewer

Actual behavior

Error parsing json

Expected behavior

Viewer opens

Your Jupyter and/or Python environment

Please provide as much info as you readily know

• Jupyter server running: Local | Remote | N/A
• Extension version: 20YY.MM.#####-xxx
• VS Code version: #.##
• Setting python.jediEnabled: true | false
• Setting python.languageServer: Jedi | Microsoft | None
• Python and/or Anaconda version: #.#.#
• OS: Windows | Mac | Linux (distro):
• Virtual environment: conda | venv | virtualenv | N/A | ...

Python Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

[joyceerhl]

Debugged this some. For the code you provided @rchiodo, this is what we're getting back from the debugger prior to calling JSON.parse:

"{\"columns\": [{\"key\": \"index\", \"name\": \"index\", \"type\": \"int64\"}, {\"key\": \"a> <a href=\\\\\"bobs\\\\\"/>\", \"name\": \"a> <a href=\\\\\"bobs\\\\\"/>\", \"type\": \"int64\"}, {\"key\": \"<b\", \"name\": \"<b\", \"type\": \"int64\"}], \"indexColumn\": \"index\", \"rowCount\": 3}"

The lodash unescape doesn't work here--it returns a string that is identical to what the debugger eval returns.

[rchiodo]

That looks like it should parse. I'd recommend just creating a test.js file that calls JSON.parse on that static string and removing items until it works.

Edit..
Looking on github (instead of through my e-mail) it might be the escaping around the quotes then.

[rchiodo]

Yeah this string works:

var t = "{\"columns\": [{\"key\": \"index\", \"name\": \"index\", \"type\": \"int64\"}, {\"key\": \"a> <a href=\\\"bobs\\\"/>\", \"name\": \"a> <a href=\\\"bobs\\\"/>\", \"type\": \"int64\"}, {\"key\": \"<b\", \"name\": \"<b\", \"type\": \"int64\"}], \"indexColumn\": \"index\", \"rowCount\": 3}"

So the quotes are escaped too much.

[joyceerhl]

To summarize it seems like the issue is that the debugger evaluate is returning a string where quotes are escaped too many times. I could do a string replace on double backslashes perhaps?

[rchiodo]

Yeah that sounds like a reasonable first step. Then see if you can break it by adding extra backslashes to something

[rchiodo]

It might only be around quotes?

[joyceerhl]

We should be able to close this. IIRC we used the rawString property on custom evaluate requests to ensure the debugger doesn't send us escaped strings.