Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Null values in area charts #2069

Open
karellm opened this issue Jul 25, 2013 · 73 comments

Comments

@karellm
Copy link

commented Jul 25, 2013

Hi,

I'm a little bit confused with this behaviour and I don't know if it is expected or a bug. Let's consider the following serie of values :
70000, null, 50000, null, null, 50000, 60000

I would expect that Highcharts doesn't draw any line/area at all for the 5 first points and would draw one between the last two values. The first and third values should get a point each but no line/area.

This is what happens for line charts, but not for area charts. I've put together a JSFiddle to show this: http://jsfiddle.net/3qZFS/1/

Thanks

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Jul 26, 2013

It's because you have stacking enabled. Without stacking it does what you expect: http://jsfiddle.net/highcharts/3qZFS/2/

@karellm

This comment has been minimized.

Copy link
Author

commented Jul 26, 2013

Yes but I need stacking. I simplified my exemple with only one serie per stack but in real life my graph include stacked series. Why does it visually draw a line to 0? It can just ignore it and stack the other one if need be,

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Jul 26, 2013

See comments in
#1836

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Aug 28, 2013

I tested this with ExCel and Google Spreadsheets. ExCel handles it the same way Highcharts does, but GS handles it differently by breaking up the overlying stacked items:

8a7c5d72-da5b-11e2-9c65-bd285cb70923

@karellm

This comment has been minimized.

Copy link
Author

commented Aug 28, 2013

As a FYI, it is a change in behaviour between HC 2 and 3. I don't know if it is documented but I don't think it is a good idea as it may introduce unexpected behaviours in existing apps.

Maybe an option plotNullAsZero would be the best option.

@karellm

This comment has been minimized.

Copy link
Author

commented Sep 18, 2013

Can you please give an ETA on this. I have many people complaining about it in my app. Thanks a lot!

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Sep 20, 2013

This is quite a big task, as we have to rethink the entire way null values are handled in Highcharts. I started this work yesterday and here is what I have so far: http://jsfiddle.net/JSfBp/

Please note it is a work in progress and full of bugs so far. Areas don't work without stacking - areasplines don't work, polar charts are broken etc.

@karellm

This comment has been minimized.

Copy link
Author

commented Sep 20, 2013

Thanks for the update. It is looking good!

Please let me know when you consider it stable so I can test it.

@karellm

This comment has been minimized.

Copy link
Author

commented Oct 23, 2013

Any chance to see this release soon? Thanks!

@leanserver

This comment has been minimized.

Copy link

commented Jan 7, 2014

I second that. We have a problem with the current behavior because it shows 0 level lines at the end of available data (the remaining data is NULLs) and that is definitely misleading.

@naziD

This comment has been minimized.

Copy link

commented Mar 14, 2014

For this issue, is there any plan to resolve this bug? It's really misleading in my system for our customers.

@slggamer

This comment has been minimized.

Copy link

commented Mar 19, 2014

For Joulss' solution:

if ((connectNulls && (!pointMap[x] || pointMap[x].y === null)) || pointMap[x].y == null) { // #1836 
instead of
if (connectNulls && (!pointMap[x] || pointMap[x].y === null)) { // #1836
in http://code.highcharts.com/highcharts.src.js

It's ok if the null value at the end of the chart but it will connect two parts if the null value in the middle of the chart.
old
new

@ivanmijatovic89

This comment has been minimized.

Copy link

commented Mar 19, 2014

Slggamer, thanks a lot for last post. I have incredible luck, you just post comment 15 hours ago

@w100

This comment has been minimized.

Copy link

commented Jul 17, 2014

seems to have become a critical bug

20140717_120225

@Way

This comment has been minimized.

Copy link

commented Jul 25, 2014

+1
Any chance to see this issue(s) fix in an upcoming version?
I'm facing that problem at my company having the need of using stacked areas that my contain data gaps (null values). The current treatment of handling null values as zero values is indeed misleading and in point of visualization it's simply wrong.

@simonfox

This comment has been minimized.

Copy link

commented Jul 26, 2014

+1
The example of work in progress from your comment above (#2069 (comment)) is exactly the behavior I would like...

@ghost

This comment has been minimized.

Copy link

commented Aug 8, 2014

+1 for http://jsfiddle.net/JSfBp/ when we are stacking nulls the stacked value should be null and not 0

@resure

This comment has been minimized.

Copy link

commented Aug 8, 2014

+1
Almost blocker 😔

@kieselw

This comment has been minimized.

Copy link

commented Sep 10, 2014

+1
The unstated points are not being created in stacking positions.

As shown in the example below, the data are commented to show how example: http://jsfiddle.net/dhqyeahp/4/

In version 3.0.5 it worked: http://jsfiddle.net/dhqyeahp/5/

@andywarduk

This comment has been minimized.

Copy link

commented Sep 18, 2014

+1 big problem for us too...

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Sep 22, 2014

I've picked up again the work on this, will try to get it finished now so we can check this one up. Remaining issues are stacked areasplines, animation on addPoint and ploar areas.

@dbroten

This comment has been minimized.

Copy link

commented Sep 23, 2014

+1 from me as well.

@mcr222

This comment has been minimized.

Copy link

commented Dec 15, 2014

The unstated points in: http://jsfiddle.net/dhqyeahp/4/ should be treated as null right? I have problems when not specifying points in splines, they are not treated as null (not even as zero, connectNulls doesn't work).

+1 To Google Spreadsheets solution.

@tristil

This comment has been minimized.

Copy link

commented Jan 21, 2015

You can solve the problem Slggamer mentions by starting a new set of series on the other side of the gap FYI, although this will mess up the legend.

@mcr222

This comment has been minimized.

Copy link

commented Jan 26, 2015

Is there any way of making the default behavior of missing points as null? Currently missing points in a line chart are ignored and chart connects them with a line.

With the example, I would like the black line behave as the blue line:
http://jsfiddle.net/Ld7vxxns/

Thanks!

@sebastianbochan

This comment has been minimized.

Copy link
Contributor

commented Jan 5, 2016

Seems that this kind of chart http://jsfiddle.net/bbs3buc0/3/ is also printed incorrectly.

@japepper88

This comment has been minimized.

Copy link

commented Jan 5, 2016

@envision, @karellm - I also cannot believe this issue is still outstanding and with little acknowledgement..

Considering the price of this software at 'OEM' level it really is about time this issue was properly fixed.

We also have a product dependant on HighCharts and at the moment it's simply misleading to represent null and zero in the same manner. We are having to use a number of hacks to circumvent this and display the data properly..

@TorsteinHonsi - do you have any plans or information regarding this issue? or are you still happy to pretend it doesn't exist?

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Jan 6, 2016

@japepper88 Thanks for pushing. Just want to remind you it currently works like in Excel. But I'll see what we can do with the issue. As I commented above, the remaining issue now is null values in areasplines, but we shouldn't let that stop us.

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Jan 6, 2016

Just to sum things up, we have a brach, nosegments, where we do development on this issue. The changes are so deep that I haven't had the guts to merge it into the master yet, but it's probably about time now.

Summing up some of the latest comments above:

@japepper88, @envision: Your charts will now look something like this: http://jsfiddle.net/highcharts/z870x3ne/

As area looks better than arespline on some uses, we have gone to lengthy hacks with adding several other series to circumvent this issue like hiding series behind black "gap series"

With the current Highcharts version (v4.2.1), you can add two prototype properties to make the area stack like the areaspline: http://jsfiddle.net/highcharts/z870x3ne/1/

Another issue which has needed circumventing is that when having a line series, if there are null values before and after a point, it won't be drawn!

That's because a single point has no extent (it's a singularity), so consequently there is no area below it. This is a case where you need to enable the marker explicityly: http://jsfiddle.net/highcharts/z870x3ne/2/.

Seems that this kind of chart http://jsfiddle.net/bbs3buc0/3/ is also printed incorrectly.

With the nosegments branch it prints like this: http://jsfiddle.net/highcharts/bbs3buc0/4/

@envision

This comment has been minimized.

Copy link

commented Jan 6, 2016

Thanks @TorsteinHonsi !

About that singularity / single point issue. I was hoping you could add a new prototype property for drawing single points in a line but not all markers. That would give us more visual flexibility as you know – at times a straight solid line looks the best... Adding unnecessary markers there would be visual noise. Without such a feature we have needed to resort to lengthy hacks of adding another series with processed dataset... When it could be just one prototype property.

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Jan 6, 2016

What about extending the drawPoints method like this?

PS: Maybe you should create a separate issue for this, as it is not related to the drawing of null points in areas.

@FelixWagner

This comment has been minimized.

Copy link

commented Jan 6, 2016

@TorsteinHonsi

Seems that this kind of chart http://jsfiddle.net/bbs3buc0/3/ is also printed incorrectly.

With the nosegments branch it prints like this: http://jsfiddle.net/highcharts/bbs3buc0/4/

This is not what I would expect this chart to look like.
Neither series has null values so they shouldn't be interrupted just because another series is more detailed. Series 1 is not missing a point. The existing points are enough to accurately describe the series... unless it's stacked.
What I'd expect is a stacking of the lines/areas, not just the point values.

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Jan 6, 2016

Yes I agree. We would expect Series 1 to be draped over Series 2. What currently happens is that missing points are treated exactly like null points. What we are considering is to keep that, but to add an option, connectNulls: 'interpolate'.

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Feb 4, 2016

Fixed with Highcharts 4.2.2.

@sebastianbochan

This comment has been minimized.

Copy link
Contributor

commented Feb 5, 2016

Looks like cause an other issue: https://jsfiddle.net/wyx1k8hy/

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Feb 5, 2016

Filed at #5004

@sebastianbochan

This comment has been minimized.

Copy link
Contributor

commented Feb 17, 2016

Next demo: http://jsfiddle.net/ofwoq6j3/

Try to hover on the first tick and 1 value. Hover working, but the area shape is missing.

zrzut ekranu 2016-02-17 o 14 51 38

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Feb 26, 2016

This is how it is intended - Series 1 is missing data for 03:39:30, so it is treated as a null point.

@FelixWagner

This comment has been minimized.

Copy link

commented Feb 26, 2016

Well, here is a counter example: http://jsfiddle.net/ofwoq6j3/2/ - Series 1 is defined completely, it's definitely not missing points.

The additional x value in series 2 should not result in series 1 being interrupted.
The problem stems from specifying a stacking type. If that setting is removed, series 1 gets drawn as a single area.
This should not make such a severe difference when stacking series. At least there should be an interpolation option to make it look like http://imgur.com/zYmNUYB .

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Feb 29, 2016

Yes, I see what you mean. Currently you can set connectNulls to true, which makes the null points behave like zeroes except they don't have a line, and there is connectNulls = false, which gives the default behaviour. We should probably have something like connectNulls: 'interpolate' that would give the behaviour from your screenshot.

@NickolasLapp

This comment has been minimized.

Copy link

commented Jul 25, 2016

+1 for the idea of a connectNulls: 'interpolate' option. My dataset has series which are sampled at different intervals and then parsed to .csv. Currently have to manually interpolate the data which leads to the appearance of higher sampling than is actually being done.

Would prefer to be able to specify an interpolation type (a really fancy implementation might look like http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.interpolate.html) so that we don't give the impression that we are sampling more often than we actually are, but the line maintains continuity.

@kindy-aa

This comment has been minimized.

Copy link

commented Aug 23, 2016

create a new option "hideNulls" for area chart, demo: http://jsfiddle.net/vzcth45q/4/ .
screenshots:
image

@carlosmorantes

This comment has been minimized.

Copy link

commented Sep 8, 2016

Is this already solved? Despite that I have installed higcharts v4.2.4, when the area is stacked, it connects wrong the points: (connectNulls: true)

highcharts

@rchzh

This comment has been minimized.

Copy link

commented Sep 8, 2016

In case I unselect serie that is above serie with NULL value, I can see this empty place... Does anybody else has this problem?
image

@andrei-gheorghiu

This comment has been minimized.

Copy link

commented Mar 21, 2017

I'm not much of an expert and have only been using Highcharts for a short while.

However, it looks to me like the most appropriate behavior would be not to map null/missing values as 0, but to not draw them at all. Or draw them invisible (color:transparent), if that makes more sense from a technical point of view. If they are drawn invisible, care must be taken in tooltips so the values shown for points with null values are printed in tooltip as null/empty string rather than the interpolated value from nearby existing points.

Does this make sense?

@TorsteinHonsi

This comment has been minimized.

Copy link
Collaborator

commented Mar 22, 2017

Yes, but in the current versions, null points are not drawn as zeroes: http://jsfiddle.net/highcharts/htas7o9v/

@sebastianbochan

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2018

It looks like the issue in stacking when an yAxis is the logarithic type.

Demo:

Related topic:

@smokir

This comment has been minimized.

Copy link

commented Jul 23, 2019

Another example of incorrect area with stacking: percent and connectNulls: false: https://jsfiddle.net/q5bepj8x/ However it looks like markers are correct. Maybe it's possible to reuse their logic.
Снимок экрана 2019-07-23 в 20 41 37

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.