Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect URL for Vega data files #7017

Closed
malramsay64 opened this issue Aug 15, 2019 · 9 comments · Fixed by #7022
Closed

Incorrect URL for Vega data files #7017

malramsay64 opened this issue Aug 15, 2019 · 9 comments · Fixed by #7022

Comments

@malramsay64
Copy link

@malramsay64 malramsay64 commented Aug 15, 2019

Description

When creating a figure with altair, using the data_transformer functionality, which stores the data in a local file and gets Vega load the file, results in a URL which is improperly formed, so the data never loads.

The URL has the _xsrf information in front of the name of the file which is being loaded.

403 GET /files/?_xsrf=2|572196ba|42fbfea4f325901e3948cbc75bd747c8|1565583318/altair-data-6572fe4ca88650e09d67c52dd6a94b8a.json (172.17.0.1): is not a file

Issue first raised altair-viz/altair#1651 and directed here.

Reproduce

Install jupyterlab

conda create -n jupyter-test jupyterlab==1.0.5 altair==3.2.0 pandas==0.25.0

In a new notebook run

import altair as alt
import pandas as pd
alt.data_transformers.enable("json")
df = pd.DataFrame({"x": [1,2,3], "y": [1,2,3]})
alt.Chart(df).mark_line().encode(x="x", y="y")

In the browser tools there is the error

WARN Loading failed altair-data-6572fe4ca88650e09d67c52dd6a94b8a.json Error: "403Forbidden"

while the console has the error message

403 GET /files/?_xsrf=2|572196ba|42fbfea4f325901e3948cbc75bd747c8|1565583318/altair-data-6572fe4ca88650e09d67c52dd6a94b8a.json (172.17.0.1): is not a file

The problem seems to be in the GET request, which has the _xsrf information before the file information. Using the firefox dev tools to edit and resend the request, both the removal of the _xsrf parameter from the url, or moving it after the filename result in a successful request.

Expected behavior

It is expected that the chart is filled with a straight line,
instead there is no data in the chart.

Context

  • Operating System and version: Fedora 30
  • Browser and version: Firefox 68
  • JupyterLab version: 1.0.5
Troubleshoot Output
$PATH:
	/opt/conda/bin
	/usr/local/sbin
	/usr/local/bin
	/usr/sbin
	/usr/bin
	/sbin
	/bin

sys.path:
/opt/conda/envs/jupyter/bin
/opt/conda/envs/jupyter/lib/python37.zip
/opt/conda/envs/jupyter/lib/python3.7
/opt/conda/envs/jupyter/lib/python3.7/lib-dynload
/opt/conda/envs/jupyter/lib/python3.7/site-packages

sys.executable:
/opt/conda/envs/jupyter/bin/python3.7

sys.version:
3.7.4 (default, Aug 9 2019, 18:51:30)
[GCC 7.3.0]

platform.platform():
Linux-5.2.7-200.fc30.x86_64-x86_64-with-debian-9.8

pip list:
Package Version
----------------- ---------
altair 3.2.0
attrs 19.1.0
backcall 0.1.0
bleach 3.1.0
certifi 2019.6.16
decorator 4.4.0
defusedxml 0.6.0
entrypoints 0.3
ipykernel 5.1.1
ipython 7.7.0
ipython-genutils 0.2.0
jedi 0.13.3
Jinja2 2.10.1
json5 0.8.5
jsonschema 3.0.1
jupyter-client 5.3.1
jupyter-core 4.5.0
jupyterlab 1.0.5
jupyterlab-server 1.0.0
MarkupSafe 1.1.1
mistune 0.8.4
mkl-fft 1.0.12
mkl-random 1.0.2
nbconvert 5.5.0
nbformat 4.4.0
notebook 6.0.0
numpy 1.16.4
pandas 0.25.0
pandocfilters 1.4.2
parso 0.5.0
pexpect 4.7.0
pickleshare 0.7.5
pip 19.1.1
prometheus-client 0.7.1
prompt-toolkit 2.0.9
ptyprocess 0.6.0
Pygments 2.4.2
pyrsistent 0.14.11
python-dateutil 2.8.0
pytz 2019.1
pyzmq 18.0.0
Send2Trash 1.5.0
setuptools 41.0.1
six 1.12.0
terminado 0.8.2
testpath 0.4.2
toolz 0.10.0
tornado 6.0.3
traitlets 4.3.2
wcwidth 0.1.7
webencodings 0.5.1
wheel 0.33.4

conda list:
# packages in environment at /opt/conda:
#
# Name Version Build Channel
asn1crypto 0.24.0 py37_0
ca-certificates 2019.1.23 0
certifi 2019.3.9 py37_0
cffi 1.12.2 py37h2e261b9_1
chardet 3.0.4 py37_1
conda 4.6.14 py37_0
cryptography 2.6.1 py37h1ba5d50_0
idna 2.8 py37_0
libedit 3.1.20181209 hc058e9b_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libstdcxx-ng 8.2.0 hdf63c60_1
ncurses 6.1 he6710b0_1
openssl 1.1.1b h7b6447c_1
pip 19.0.3 py37_0
pycosat 0.6.3 py37h14c3975_0
pycparser 2.19 py37_0
pyopenssl 19.0.0 py37_0
pysocks 1.6.8 py37_0
python 3.7.3 h0371630_0
readline 7.0 h7b6447c_5
requests 2.21.0 py37_0
ruamel_yaml 0.15.46 py37h14c3975_0
setuptools 41.0.0 py37_0
six 1.12.0 py37_0
sqlite 3.27.2 h7b6447c_0
tk 8.6.8 hbc83047_0
urllib3 1.24.1 py37_0
wheel 0.33.1 py37_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zlib 1.2.11 h7b6447c_3

Command Line Output
[W 00:20:36.600 LabApp] 403 GET /files/?_xsrf=2|572196ba|42fbfea4f325901e3948cbc75bd747c8|1565583318/altair-data-6572fe4ca88650e09d67c52dd6a94b8a.json (172.17.0.1):  is not a file
[W 00:20:36.601 LabApp] 403 GET /files/?_xsrf=2|572196ba|42fbfea4f325901e3948cbc75bd747c8|1565583318/altair-data-6572fe4ca88650e09d67c52dd6a94b8a.json (172.17.0.1) 1.67ms referer=http://127.0.0.1:8889/lab
Browser Output
WARN Loading failed altair-data-6572fe4ca88650e09d67c52dd6a94b8a.json Error: "403Forbidden"
@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

Looks like Vega extension should not assume that getdownloadurl returns just a path it can append to:

const baseURL = await this._resolver.getDownloadUrl(path);

@jasongrout jasongrout added this to the 1.0.x milestone Aug 15, 2019
@jasongrout jasongrout removed this from the 1.0.x milestone Aug 15, 2019
@jasongrout jasongrout added this to the 1.1 milestone Aug 15, 2019
@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

@domoritz - is there an easy way to override https://github.com/vega/vega/blob/dd006488d508e0f1e5154cb20eeea3ac7a3f4cd2/packages/vega-loader/src/loader.js#L85-L91 in the vega 4/5 jlab plugin to call a function in JLab that, given a relative path, will return a URL to use to download it?

We could try to work under the assumption that we can get a base url (which is where we have an error in the vega4/5 extensions now), but really it would be more correct to let the system give us a download URL given a path.

@domoritz
Copy link
Member

@domoritz domoritz commented Aug 15, 2019

We can provide a custom loader to Vega and replace all the logic we want. We are currently only customizing the base URL in

and HTTP header. The easiest way would probably be to just override the sanitize function. A cleaner way would probably be to request a file:// url and write a custom file handler that we set to https://github.com/vega/vega/blob/dd006488d508e0f1e5154cb20eeea3ac7a3f4cd2/packages/vega-loader/src/loader.js#L33.

Does this answer your question?

@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

It gives hope :).

  1. What do you mean by "request a file:// url" - ask the python side in altair to explicitly give a file:// url instead of just a relative path to load?
  2. To override the default loader, we just pass in a {loader: {load, sanitize, http, file}} object at
    loader: {
    baseURL,
    http: { credentials: 'same-origin' }
    }
    instead of the loader options we are currently passing in?

@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

  1. What do you mean by "request a file:// url" - ask the python side in altair to explicitly give a file:// url instead of just a relative path to load?

I read https://tools.ietf.org/html/rfc8089 for a bit, and it seems that there is no correct way to give a file: uri that is relative to some external root - it seems that file: uris are always absolute. That won't work for our case, where the python code does not know the right absolute path to give.

@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

@malramsay64 - did this ever work in JupyterLab? If so, what version?

Copying reproducing instructions from altair-viz/altair#1651:

conda create -n jupyter-test jupyterlab==1.0.5 altair==3.2.0 pandas==0.25.0
import altair as alt
import pandas as pd
from pprint import pprint
alt.data_transformers.enable("json")
df = pd.DataFrame({"x": [1,2,3], "y": [1,2,3]})
alt.Chart(df).mark_line().encode(x="x", y="y")

@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

@malramsay64 - did this ever work in JupyterLab? If so, what version?

It's working for me in jlab 1.0.2:

conda create -n jupyter-test jupyterlab==1.0.2 altair==3.2.0 pandas==0.25.0

jasongrout added a commit to jasongrout/jupyterlab that referenced this issue Aug 15, 2019
…rver.

Perhaps we should also use the services package to actually send the request to the server. That would complicate these plugins beyond simple rendermime plugins, though.

Fixes jupyterlab#7017
jasongrout added a commit to jasongrout/jupyterlab that referenced this issue Aug 15, 2019
…rver.

Perhaps we should also use the services package to actually send the request to the server. That would complicate these plugins beyond simple rendermime plugins, though.

Fixes jupyterlab#7017
@jasongrout
Copy link
Contributor

@jasongrout jasongrout commented Aug 15, 2019

This should be fixed in #7022

@malramsay64
Copy link
Author

@malramsay64 malramsay64 commented Aug 16, 2019

The fix works for me.

Thank you @jasongrout for such a quick response and fix for this issue!!

@lock lock bot locked as resolved and limited conversation to collaborators Sep 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants