Query variables parsed differently to raw values

[josalmi]

If you suspect this could be a bug, follow the template.

• What version of Dgraph are you using?
  v1.0.10

• Have you tried reproducing the issue with latest release?
  yes

• What is the hardware spec (RAM, OS)?
  Google Kubernetes Engine using the official dgraph images

• Steps to reproduce the issue (command/config used to run Dgraph).

We are using dgraph-js-http library for communicating with dgraph server from a browser. We do the following query:

const request = `query documents($query: string) {
        documents(func: alloftext(text, $query)) {
          text
        }
      }`
    const {
      data: { documents },
    } = await client.newTxn().queryWithVars(request, {
      $query: search // input from user
    })

We found a problem with how variables are handled. The variables work fine for queries that only use US-ASCII letters like "foo bar" but won't match queries that contain letters like "öä".

We tracked this down to HTTP RFC7230 defining header values to be made of USASCII characters which don't include åäö for example. This causes browsers to encode such values as \xf6 and similar encoding.

If the query was in format:

{
  documents(func: alloftext(text, "öä")) {
    text
  }
}

and not using variables then the query would return results as expected.

You should be able to test this easily by having some text that contains non usascii letters and matching that with query that contains variables. I'm not sure whether this need to be fixed in dgraph, dgraph-http-json or both.

• Expected behaviour and actual result.
  We expect dgraph to match non usascii letters as well when doing queries with variables.

I can provide example code in GitHub repository if that helps.

[srfrog]

Thanks for reporting this. We are working to improve i18n support, and many changes have gone into 1.0.11 for this. by default Dgraph will consider all text to be English, so you need to tell it to use i18n features.

in 1.0.10 try this and see if it helps:

{
  documents(func: alloftext(text, "<öä>")) {
    text
  }
}

[josalmi]

I think I might have been a bit unclear in my first message.

I created a example application where you can see this error happening (the example app uses play.dgraph.io as backend):
https://protected-shore-55899.herokuapp.com/

The source code is available here:
https://github.com/josalmi/dgraph-aakkoset

Both examples should return same results but the one where variables are used fails to do so.

As as workaround we created a proxy that re-encodes the x-dgraph-vars header as utf-8. Source code for the proxy can be found here: https://github.com/josalmi/dgraph-proxy

A solution for this could be including the variables in the request body as some graphql clients seem to work where encoding issues should not happen. Other way could be base64 encoding the json in the header and decoding on the server.

[srfrog]

The first thing that I see is that dgraph-js-http is using toString() method which I think doesn't support UTF-8. There needs to be proper encode/decode. So that could be part of it.

On the server side, Dgraph appears to behave as expected. I use the goldendata for testing, which has plenty of unicode strings.

# alter name to use fulltext index
name: string @index(term, fulltext) @lang .

# query using vars
query test($name: string = "Beyoğlu'nun Arka Yakası") {
	me(func: alloftext(name@en, $name)) {
		name@.
	}
}

So I'm going to close the bug here a move it/reference to that repo.

Thanks for the thorough report.