[0.9.2] parser incorrectly accepts NaN as numerical value, but not always

[kalidasya]

I use the collectd tail plugin, what can report nan when there were no matching data in the tailed file, what somehow causes corruption in the influxdb db. If it happens a query like:
select * from /.*/ or select * from tail_value will die with:
ERR: invalid character 'j' looking for beginning of value

the easiest way to reproduce is to add the following config to collectd:

LoadPlugin tail
<Plugin "tail">
 <File "/tmp/reproduce_it.log">
  Instance "something_tail"
  <Match>
   Regex ".*something ([0-9]+).*"
   DSType "GaugeLast"
   Type "gauge"
   Instance "good_luck"
  </Match>
 </File>
</Plugin>

most likely after collectd runs, you will see nan reported, you can always echo some stuff to it like:
echo "something blah" >> /tmp/reproduce_it.log

[PierreF]

I have the same issue, we can store NaN value, but then we could do a query that use this value.

I can reproduce this issue with 0.9.2 and master (affd0b1).

Step to reproduce:

• clean ~/.influxdb to start from fresh setup (rm -fr ~/.influxdb)
• Start influxdb
• With UI, create a database and (still with UI) write the following two points:

cpu_load_short,host=server01,region=us-west value=10.0 1434055562000000000
cpu_load_short,host=server01,region=us-west value=NaN 1434055563000000000

• Write successed, and indeed we can read the first point: select * from cpu_load_short where time = '2015-06-11T20:46:02Z';
• Reading the second point fail: select * from cpu_load_short where time = '2015-06-11T20:46:03Z';
  • Error (on UI) is : Couldn't connect to host (http://localhost:8086): OK
  • Error (in HTTP response) is : json: error calling MarshalJSON for type httpd.Response: json: error calling MarshalJSON for type *influxql.Result: json: unsupported value: NaN
• Reading all value (select value from cpu_load_short) even with aggregating (select mean(value) from cpu_load_short) fail with the same error

[beckettsean]

I'm incredibly confused now. I can recreate this, but not reliably. Sometimes the unquoted NaN field value throws a parser error, sometimes it doesn't. I can never successfully query the mistakenly written points:

> use mydb
Using database mydb
> insert nan value=NAN
ERR: unable to parse ' nan value=NAN': invalid number

> insert nan value=NaN
ERR: unable to parse ' nan value=NaN': invalid number

> insert cpu_load_short,host=server01,region=us-west value=NaN 1434055563000000000
> insert cpu_load_short value=NaN 1434055563000000000
> insert cpu_load_short value=NaN 
> insert nannan value=NaN
ERR: unable to parse ' nannan value=NaN': invalid number

> insert cpu_load_sh value=NaN 
> insert cpu_ value=NaN 
> insert c value=NaN 
> insert a value=NaN 
> insert nanananan value=NaN 
> insert nannan value=NaN 
> insert nan value=NaN
ERR: unable to parse ' nan value=NaN': invalid number

> insert nannan value=NaN 
> insert nan value=NaN
ERR: unable to parse ' nan value=NaN': invalid number

> insert nana value=NaN
ERR: unable to parse ' nana value=NaN': invalid number

> insert nanaaa value=NaN
ERR: unable to parse ' nanaaa value=NaN': invalid number

> select * from cpu_load_short
ERR: invalid character 'j' looking for beginning of value
> select value from cpu_load_short
ERR: invalid character 'j' looking for beginning of value
> select value from cpu_load_sh
ERR: invalid character 'j' looking for beginning of value
> select value from nan
> select value from nannan
ERR: invalid character 'j' looking for beginning of value
> select value from c
ERR: invalid character 'j' looking for beginning of value
> select value from a
ERR: invalid character 'j' looking for beginning of value
> select value from cpu_
ERR: invalid character 'j' looking for beginning of value

It should always throw a parser error, or we have to build in support for NaN.

[kalidasya]

I am using 0.9.2-1 and it seems to me the following case are also failing:
insert nan value=NaN
to me it seems, if the last thing in the insert is NaN it fails:
insert cpu_load_sh value=NaN is OK (note the whitespace at the end) but:
insert cpu_load_sh value=NaN is not.

[kalidasya]

@beckettsean really no rush, just to plan my tasks, what do you think will it be fixed (in an RC) in the next 3 weeks?

[beckettsean]

@kalidasya It's still unscheduled, so the earliest would be in a few weeks.

[Jan-Zeiseweis]

The bug can be found in:
https://github.com/influxdb/influxdb/blob/v0.9.3-rc1/tsdb/points.go#L577
if i+3 < len(buf) should be if i+2 < len(buf)
We tried to fix this, but even when you're able to insert the 'NaN' instead of 'NaN ' you'll have the problem that you can't json encode NaN's.
That's actually why you receive:
ERR: invalid character 'j' looking for beginning of value
Because https://github.com/influxdb/influxdb/blob/v0.9.3-rc1/influxql/result.go#L102 returns:
json: unsupported value: NaN

I hope this helps.

[kalidasya]

Probably it is related to: golang/go#3480 (sorry my go knowledge is 0) where they suggest to use custom types for floats: golang/go#3480 (comment)

[zelandoramen1]

Please add full support of NaN and +-Inf.
They are part of IEEE 754 double-precision specification
7fff ffff ffff ffff = NaN
https://en.wikipedia.org/wiki/Double-precision_floating-point_format
Sometimes NaN and InF are used in data. For example NaN may be used to show no data from sensor.

[beckettsean]

@zelandoramen1 please make a feature request in a new issue rather than commenting on a closed issue.