[0.9.3] Impossible to insert string value test\" as anything other than the last field

[yvesf]

I found no way to insert the value test\" in a string field. I think the escaping for fieldValue (in tsdb.scanFieldValue(buf []byte, i int) (int, []byte)) does not allow that.

Let's try:

$ curl 'http://localhost:8086/write?db=test' --data-binary 'test value="test"'
$ curl 'http://localhost:8086/write?db=test' --data-binary 'test value="test\""'
$ curl 'http://localhost:8086/write?db=test' --data-binary 'test value="test\\""'
unable to parse 'test value="test\\""': unbalanced quotes
$ curl 'http://localhost:8086/write?db=test' --data-binary 'test value="test\\\""'

$ echo -e "use test\n select * from test" | influx
Connected to http://localhost:8086 version 0.9
InfluxDB shell 0.9
Using database test
name: test
----------


time                value
time                value
2015-08-15T23:38:58.360766155Z  test
2015-08-15T23:39:01.708551024Z  test"
2015-08-15T23:39:08.231959134Z  test\\"

[beckettsean]

@jwilder did your recent PR around parsing escape characters fix this?

[jwilder]

@beckettsean no. Still working on this one.

[yvesf]

Hi,
please take a look on the following insert, I have the feeling that the escaping is not quite right but I'm not sure yet. I'm testing current master.

$ curl 'http://localhost:8086/write?db=test' --data-binary 'test field1="value2\\",field2="value2"'

$ echo -e "use test\n select * from test" | ./gopath/bin/influx
Using database test .. name: test
----------
time                field1
2015-09-04T22:44:25.345145995Z  value2\",field2="value2

Expected:

time                field1        field2
2015-09-04T22:44:25.345145995Z  value2\       value2

If there is a problem then I assume it's a regression of the previous bug, otherwise feel free to move it to a new issue.
Regards, Yves

[beckettsean]

@yvesf there is a bug here, but there's a trivial workaround, as well.

Inserting a single trailing backslash works as long as there are no more fields to parse. It appears that the comma separating the fields complicates the parsing.

All of this is expected behavior:

> insert escaping field="string\a"
> insert escaping field="string\"
ERR: unable to parse 'escaping field="string\"': unbalanced quotes

> insert escaping field="string\\"
> select * from escaping
name: escaping
--------------
time                field
2015-09-04T22:57:45.451812805Z  string\a
2015-09-04T22:57:50.579125528Z  string\

However, if there's a field after the one with the escaped trailing backslash the two fields get combined into one (and the second field value is silently dropped.)

> insert escaping field="string\\",value=1
> select * from escaping
name: escaping
--------------
time                field
2015-09-04T22:57:45.451812805Z  string\a
2015-09-04T22:57:50.579125528Z  string\
2015-09-04T22:58:06.124683Z string\",value=

> 

Escaping something other than backslash works as expected:

> insert escaping field="string\""
> insert escaping field="string\"",value=1
> select * from escaping
name: escaping
--------------
time                field       value
2015-09-04T22:57:45.451812805Z  string\a    
2015-09-04T22:57:50.579125528Z  string\     
2015-09-04T22:58:06.124683Z string\",value= 
2015-09-04T23:01:05.093934091Z  string"     
2015-09-04T23:01:10.158174843Z  string"     1

>

Repro'd using 0.9.3

[beckettsean]

As long as the escaped backslash is the final field value things work as expected:

> insert escaping value=1,field="string\\"
> select * from escaping
name: escaping
--------------
time                field       value
2015-09-04T22:57:45.451812805Z  string\a    
2015-09-04T22:57:50.579125528Z  string\     
2015-09-04T22:58:06.124683Z string\",value= 
2015-09-04T23:01:05.093934091Z  string"     
2015-09-04T23:01:10.158174843Z  string"     1
2015-09-04T23:03:23.55665375Z   string\     1

> 

[yvesf]

This is what I found as fix and test-case. I think it makes sense to add a test like this one to points_test.go.

diff --git a/tsdb/points.go b/tsdb/points.go
index dd8dbb6..7fd22cb 100644
--- a/tsdb/points.go
+++ b/tsdb/points.go
@@ -848,0 +849,6 @@ func scanFieldValue(buf []byte, i int) (int, []byte) {
+        // Accept escape char followed by escape char
+        if buf[i] == '\\' && i+1 < len(buf) && buf[i+1] == '\\' {
+            i += 2
+            continue
+        }
+
diff --git a/tsdb/points_test.go b/tsdb/points_test.go
index f3cf93d..7fcfcbb 100644
--- a/tsdb/points_test.go
+++ b/tsdb/points_test.go
@@ -646,0 +647,8 @@ func TestParsePointUnescape(t *testing.T) {
+    test(t, `test field1="value1\\",field2="value2"`,
+        tsdb.NewPoint(
+            "test",
+            tsdb.Tags{},
+            tsdb.Fields{
+                "field1" : "value1\\",
+            },
+                       time.Unix(0, 0)))

[marccardinal]

This is still an issue on InfluxDB v0.13.0 (git: 0.13 e57fb88a051ee40fd9277094345fbd47bb4783ce)

> drop measurement escaping;
> insert escaping field="test1_abc"
> insert escaping field="test2_abc\"
ERR: {"error":"unable to parse 'escaping field=\"test2_abc\\\"': unbalanced quotes"}

> insert escaping field="test3_abc\\"
> insert escaping field="test4_abc\",otherfield="value"
ERR: {"error":"unable to parse 'escaping field=\"test4_abc\\\",otherfield=\"value\"': unbalanced quotes"}

> insert escaping field="test5_abc\\",otherfield="value"
> insert escaping field="test6_abc\\abc",otherfield="value"
> select * from escaping
name: escaping
--------------
time                    field                           otherfield
1464732839685274409     test1_abc
1464732845820495778     test3_abc\
1464732861227535067     test5_abc\",otherfield="value
1464732869579298997     test6_abc\abc                   value

[jsternberg]

At the moment, it isn't possible for us to fix this without breaking the line protocol. I'm closing this in favor of #6037 which is a discussion about a v2 of the line protocol.