-
Notifications
You must be signed in to change notification settings - Fork 782
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: update to Vega-Lite 5 #2517
Conversation
# python 3
from urllib.request import urlopen
import json
import jsonschema # 3.2.0
def validate(vl_spec, vl_schema="v5.1.0"):
schema = json.load(urlopen(f'https://vega.github.io/schema/vega-lite/{vl_schema}.json'))
spec = json.loads(vl_spec)
jsonschema.validate(spec, schema)
vl_spec = """
{
"layer": [{"mark": "circle", "width": 100}, {"mark": "circle"}],
"data": {"values": []}
}
"""
validate(vl_spec)
The above validates still OK in v4.17.0 ( When placing the vl_spec = """
{
"width": 100,
"layer": [{"mark": "circle"}, {"mark": "circle"}],
"data": {"values": []}
}
"""
validate(vl_spec) |
If we wanted, we could add logic to It's a minor thing, but it would be nice to remain backward-compatible on this. |
Thank you! Do you have an example of a property where we do something similar to this? (It doesn't have to be in LayerChart.) If I have a template to use, I should be able to do the same thing for width and height. So far at least with |
What I had in mind was the base = alt.Chart(dataframe).encode(...)
chart = base.mark_point() + base.mark_line() and only embed a single copy of the serialized dataframe in the output. We could do something similar for |
I looked around a bit and I think this was initially introduced in VL4.1.1, but then reverted and included with a deprecation warning because Jake reported that it broke Altair. That deprecated code was removed in 5.0 in this PR, which is probably why it is breaking now. It seems like in addition to |
Thank you @joelostblom for tracking that down. Would you be able to give me an example where we use this |
One relevant piece of info here is that the Vega-lite renderer maintains backward compatibility for some things (such as passing We have a couple options for dealing with that:
I would lean toward (3) here, as (1) would lead to poorer usability, and my sense is that (2) would add an undue maintenance burden, and (4) would be a large departure from Altair's current implementation |
@ChristopherDavisUCI I am not sure how much it is actually used, but alt.Chart(alt.Data(values=[{}]), height=200, view=alt.ViewConfig()).mark_bar().to_dict()
and using
|
@joelostblom Thank you! @jakevdp Thanks for the pointer to Edit: My current guess for the below example is we should just accept it needing to switch "selection" to "param". It seems reasonable to me that using an invalid property name in the dictionary is not going to have a nice fix. Aside from rendering examples (literally zero of which are working), there's just one example that doesn't work that I know of,
|
Here's a first example of using
The chart could also be defined this way:
|
That's nice! I think it would be cool if we could use the variables directly in the expression, i.e. something like We do a similar thing with |
I played a bit in ChristopherDavisUCI@0b34ec6 and now this works: cars = data.cars.url
year_slider = alt.binding_range(min=0, max=1, step=0.05)
size_var = alt.variable(bind=year_slider, value=0.2, name="sample")
c = alt.Chart(cars).mark_circle(
size=size_var*1000,
opacity=size_var + 0.2
).encode(
x='Miles_per_Gallon:Q',
y='Horsepower:Q',
).add_variable(size_var)
c but this won't, Because the def __add__(self, other):
return ExprRef(expr.core.BinaryExpressionReference("+", self.name, other)) It get stuck in the second round of addition, since |
I can completely imagine this should be done very different, but now this also works: import altair as alt
year_slider = alt.binding_range(min=0, max=1000, step=5)
size_var = alt.variable(bind=year_slider, value=200, name="sample")
expr = size_var ** 2 + 0.1
expr
Where Changed code in this commit ChristopherDavisUCI@530a11a |
Thanks @mattijn! I glanced and didn't immediately follow, but I will look more slowly later. Is it obvious to you why the charts don't display for me in Jupyter notebook? Do you think it's in the same category as the changes you made related to |
If you're using the jupyterlab/mimebundle renderer, that will not work, because the vega-lite extension does not yet support vl 5 Switch to the default renderer. Even with the default renderer, you can still run into issues if charts with previous vega-lite versions are displayed in the notebook, because once one version of vega-lite is loaded, new ones will not load even if the version does not match. Try clearing all outputs, saving the notebook, then reloading the page. If you're using jupyterlab, the same is true of any notebook tab you have opened in the history of the session, because different notebooks' JS & CSS environments are not sandboxed from each other (don't even get me started...). So close all notebooks, clear outputs from ones you plan to open, click reload the browser, and then open the notebooks you want to use. |
Basic idea is to add a number and comparison protocol to the import altair as alt
year_slider = alt.binding_range(min=0, max=1000, step=5)
size_var = alt.variable(bind=year_slider, value=200, name="sample")
size_var
And: size_var.name
Then er = size_var + 1
er
And then the same works if these mathematical operations are also defined as methods for er + 2 becomes
The actual implementation is a different story though. I tried to integrate with the existing code in |
In the following example (which doesn't work at the moment),
I'd like to get something like this working and then add on the arithmetic functionality that @mattijn described above. |
Hmm, per https://vega.github.io/vega-lite/docs/parameter.html, I don't think an expression reference is supported as input for an encoding channel. Within an encoding channel it can can only be included as predicate and for data extents (but then as |
@mattijn Ah, thank you, very interesting! Playing around on the Vega editor, I think something like this is allowed, where the
I guess on the Altair side, we should allow something like |
Ah yes I see, yes that make sense. So then you can eventually do the same for import altair as alt
from vega_datasets import data
cars = data.cars.url
mpg_slider = alt.binding_range(min=0, max=50, step=1)
rule_mpg = alt.variable(bind=mpg_slider, value=0.2, name="mpg_rule")
dots= alt.Chart(cars).mark_circle().encode(
x='Miles_per_Gallon:Q',
y='Horsepower:Q'
)
yrule = alt.Chart().mark_rule(strokeDash=[12, 6], size=2).encode(
y=alt.datum(rule_mpg) # <-- how to infer type? eg. alt.Y(alt.datum(rule_mpg), type='quantitative') ?
).add_variable(rule_mpg)
dots + yrule Open the Chart in the Vega Editor And: import altair as alt
from vega_datasets import data
cars = data.cars.url
x_options = alt.binding_select(options=["Horsepower", "Acceleration"])
x_var = alt.variable(bind=x_options, value="Horsepower", name="var_x")
dots = alt.Chart(cars).mark_circle().encode(
x=alt.field(x_var), # <--- not existing yet on altair side and how to infer type?
y='Horsepower:Q'
).add_variable(x_var) Open the Chart in the Vega Editor (not possibe, as raised in vega/vega-lite#7365) |
One piece of context: the reason we require |
Gentle ping @domoritz, we have a question regarding #2517 (comment), do you know if something happened or is missing related to vega-utils, vega-embed since VL5? |
Could you generate a standalone HTML page where you get the error with |
Thank you! Does this count as a standalone html page? link on Google drive I produced it by making a Jupyter notebook with the error and then downloading that notebook as an html file. I can definitely do more debugging myself also, now that I know it should be an out-of-date reference. |
I used this piece of code to debug. It works OK in JupyterLab (to start do from cmd It is based on the from altair.utils.html import spec_to_html
from IPython.core.display import display, HTML
import json
dct = json.loads("""{
"data": {
"values": [
{"a": "A", "b": 28},{"a": "B", "b": 55},{"a": "C", "b": 43},{"a": "D", "b": 91},
{"a": "F", "b": 53},{"a": "G", "b": 19},{"a": "H", "b": 87},{"a": "I", "b": 52}
]
},
"mark": "bar",
"encoding": {
"x": {"field": "a", "type": "nominal"},
"y": {"field": "b", "type": "quantitative"}
},
"height": 200,
"width": 200
}
"""
)
# dct # this seems ok
h = spec_to_html(
spec=dct,
mode='vega-lite',
vega_version='5.21.0',
vegaembed_version='6.20.2',
vegalite_version='5.1.1',
requirejs=False,
base_url='https://cdn.jsdelivr.net/npm',
fullhtml=True,
template='standard'
)
# print(h) # seems valid HTML
display(HTML(h)) # this doesn't work For Jupyter Notebook I get this error in the console log:
EDIT: Based on this line: https://github.com/altair-viz/altair/blob/8a8642b2e7eeee3b914850a8f7aacd53335302d9/altair/vega/v5/display.py#L73 I observe that the default parameter for
Basically meaning I can reproduce it with the following HTML string: from IPython.core.display import display, HTML
h = """<div id="vis"></div>
<script type="text/javascript">
(function(spec, embedOpt){
let outputDiv = document.currentScript.previousElementSibling;
if (outputDiv.id !== "vis") {
outputDiv = document.getElementById("vis");
}
const paths = {
"vega": "https://cdn.jsdelivr.net/npm/vega@5.21.0?noext",
"vega-lib": "https://cdn.jsdelivr.net/npm/vega-lib?noext",
"vega-lite": "https://cdn.jsdelivr.net/npm/vega-lite@5.1.1?noext",
"vega-embed": "https://cdn.jsdelivr.net/npm/vega-embed@6.20.2?noext",
};
function loadScript(lib) {
return new Promise(function(resolve, reject) {
var s = document.createElement('script');
s.src = paths[lib];
s.async = true;
s.onload = () => resolve(paths[lib]);
s.onerror = () => reject(`Error loading script: ${paths[lib]}`);
document.getElementsByTagName("head")[0].appendChild(s);
});
}
function showError(err) {
outputDiv.innerHTML = `<div class="error" style="color:red;">${err}</div>`;
throw err;
}
function displayChart(vegaEmbed) {
vegaEmbed(outputDiv, spec, embedOpt)
.catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));
}
if(typeof define === "function" && define.amd) {
requirejs.config({paths});
require(["vega-embed"], displayChart, err => showError(`Error loading script: ${err.message}`));
} else if (typeof vegaEmbed === "function") {
displayChart(vegaEmbed);
} else {
loadScript("vega")
.then(() => loadScript("vega-lite"))
.then(() => loadScript("vega-embed"))
.catch(showError)
.then(() => displayChart(vegaEmbed));
}
})({"data": {"values": [{"a": "A", "b": 28}, {"a": "B", "b": 55}, {"a": "C", "b": 43}, {"a": "D", "b": 91}, {"a": "F", "b": 53}, {"a": "G", "b": 19}, {"a": "H", "b": 87}, {"a": "I", "b": 52}]}, "mark": "bar", "encoding": {"x": {"field": "a", "type": "nominal"}, "y": {"field": "b", "type": "quantitative"}}, "height": 200, "width": 200}, {"mode": "vega-lite"});
</script>
"""
display(HTML(h)) Only once per session, I've to restart Jupyter Notebook to see it again (not just restart the kernel) |
The error is coming from |
You can see the requirement declared at the top of the vega-lite v5 source: https://cdn.jsdelivr.net/npm/vega-lite@5.1: !function(e,t){"object"==typeof exports&&"undefined"!=typeof module?t(exports,require("vega-util"),require("vega")) ... Again, this problem will only arise in an environment where requirejs is loaded. If it's intended on the Vega-Lite side, it means that an extra library is necessary when creating vega-lite charts in a requireJS context. We could add this to the universal renderer:
But since it's not mentioned anywhere in the vega-embed documentation, I suspect this is a bug in the vega-lite.js build process. @domoritz might have a more definitive answer. |
Good catch. Since Vega exports all the utilities from Vega-Util, we try to replace all imports from Vega-Util with imports from Vega in the bundle. It should happen with https://github.com/vega/vega-lite/blob/12bee8b450c1176434b5166ca5f42515c41a78db/rollup.config.js#L36 but evidently does not here. I will fix this now. |
No typescript user, but here it seems mentioned as an external dependency? https://github.com/vega/vega-lite/blob/12bee8b450c1176434b5166ca5f42515c41a78db/rollup.config.js#L81 |
I think I fixed the issue in vega/vega-lite#7823. It turns out rollup doesn't let us use both external and global. We have to alias the dependencies manually. The original goal was to reduce code duplication and import functionality from the Vega bundle instead of bundling it again. |
Can you try Vega-Lite 5.2.0? |
I checked and it's working now in Jupyter Notebook! Thanks @domoritz for the very fast work! |
Thank y'all for isolating the issue. It helped tremendously. |
@jakevdp Can I ask, how do you think Currently as @mattijn pointed out above, there are I have some ad hoc code right now to keep old examples from breaking. For example, if a |
My feeling on this is we should stick to the language of the Vega-Lite schema; that is, use the name |
That sounds good to me and it should be pretty easy to implement! |
Probably not... I'm not entirely sure about the approach in that PR. The fundamental issue is that currently for schema objects, attribute access returns the attribute. That is, import altair as alt
x = alt.X(field='x')
print(x.field)
# x In order to have the semantics proposed there, attribute access would have to return a function that sets the attribute; that is, we'd have to change the API of Altair objects to be something more like this: x = alt.X().field('x')
print(x.field)
# <bound method X.field of <__main__.X object at 0x7fbf5f349d00>>
print(x['field'])
# x That's certainly doable, but it's a fundamental departure from Altair's current object model. We could discuss whether the advantages of that outweight the disadvantages, but I think that's not something that should be done as part of this PR. |
Okay cool. I'll ask no more questions until after Thanksgiving! |
I raised it primarily because of the parameter definition. When having a chart with a few sliders, you have to define them now as follow: rad_slider = alt.binding_range(min=0, max=100, step=1)
rad_var = alt.variable(bind=rad_slider, value=0, name="radius")
rad2_slider = alt.binding_range(min=0, max=100, step=1)
rad_var2 = alt.variable(bind=rad_slider, value=50, name="radius2")
theta_slider = alt.binding_range(min=-2*np.pi, max=2*np.pi)
theta_var = alt.variable(bind=theta_slider, value=-0.73, name="theta_single_arc")
theta_slider2 = alt.binding_range(min=-2*np.pi, max=2*np.pi)
theta2_var = alt.variable(bind=theta_slider, value=0.73, name="theta2_single_arc")
corner_slider = alt.binding_range(min=0, max=50, step = 1)
corner_var = alt.variable(bind=corner_slider, value=0, name="cornerRadius")
pad_slider = alt.binding_range(min=0, max=np.pi/2)
pad_var = alt.variable(bind=pad_slider, value=0, name="padAngle") I feel it is a bit verbose and wish there is a shorthand similar to #1629, so it can be done as an one-liner, eg. pad_var = alt.param(value=0, name='padAngle').bind(alt.range(min=0, max=np.pi/2)) But maybe too much for now. |
If a parameter in Vega-Lite is defined by
what would be a good way to define that parameter in Altair? Is something like the following okay, or is it too verbose?
As far as I can tell, there's no "Parameter" defined in the Vega-Lite schema. There is an
|
Having to wrap |
Maybe |
I started a new Pull Request #2528 so that I could clean up some of the commit history. It uses the most recent suggestion of mostly following the terminology of Vega-Lite schema, but allowing I'll probably close this PR in a day or two unless someone says I should leave it open. Thanks and see you at #2528! |
(Draft version only!)
Surprisingly the biggest obstacle so far hasn't been
params
but a change tolayer
. I think in the newest Vega-Lite schema, charts in a layer are not allowed to specify height or width, which seems to break many Altair examples. Here is a minimal example that doesn't work:I don't see a good way to deal with that. Do you have a suggestion?
I've read the list of "breaking changes" for the Vega-Lite 5.0.0 release and don't see anything that seems related to this, so it does make me wonder if maybe I misunderstand the cause of the problem.
Other things:
Error loading script: Script error for "vega-util", needed by: vega-lite http://requirejs.org/docs/errors.html#scripterror
It does work in Jupyter Lab.selection
fits with the newparameter
. It might be best to redo this code later now that I see more of the big picture.