Cannot select field if field name has dashes #38

ilkka · 2012-10-25T08:03:10Z

Given this JSON:

[
    {
        "my-field": "val1",
        "other_field": "valA"
    },
    {
        "my-field": "val2",
        "other_field": "valB"
    }
]

this works:

$ ./jq '[.[]|{other_field}]' < test.json
[
  {
    "other_field": "valA"
  },
  {
    "other_field": "valB"
  }
]

but this does not:

$ ./jq '[.[]|{my-field}]' < test.json
error: syntax error, unexpected '-', expecting '}'
[.[]|{my-field}]
        ^
1 compile error

I think it should since the JSON is valid. Maybe there is a way around this?

The text was updated successfully, but these errors were encountered:

stedolan · 2012-10-25T08:52:05Z

Currently, that gets parsed as a subtraction. You can always explicitly use strings for when your keys don't fit identifier syntax:

{"my-new-field": .["my-old-field"]}

ilkka · 2012-10-31T13:37:16Z

Guess that makes sense, I just didn't stumble across that syntax :)

Useful when "foo" contains unusual characters. Should help with the issues #7, #38, #40, #42.

woohgit · 2013-04-29T10:58:02Z

This means I cannot get a value from a json if the key contains a dash:

< json.json jq .stuff.thisisakey-key.identifier

because it'll throws:

error: key is not defined
.stuff.thisisakey-key.identifier
                  ^^^
1 compile error

How can I do it? I tried with "" and so on, but none of them works.

ghost · 2013-04-29T11:54:14Z

jq '.stuff["thisisakey-key"].identifier' http://stedolan.github.io/jq/manual/#foo it's similar in JavaScript

zasran · 2014-03-06T23:06:00Z

According to http://stedolan.github.io/jq/manual/ "If the key contains special characters, you need to surround it with double quotes like this: ."foo$"."

However the following does not work:

$ echo '{"a-b":"0"}' | jq '."a-b"'
error: syntax error, unexpected QQSTRING_START, expecting $end
."a-b"
^
1 compile error

This works:

$ echo '{"a-b":"0"}' | jq '.["a-b"]'
"0"

However it's not clear how to make it work if you want to get a nested value:

$ echo '{"a-b":{"c-d":0}}' | jq '.["a-b"]."c-d"'
error: syntax error, unexpected QQSTRING_START, expecting IDENT
.["a-b"]."c-d"
^
1 compile error

$ echo '{"a-b":{"c-d":0}}' | jq '.["a-b"].["c-d"]'
error: syntax error, unexpected '[', expecting IDENT
.["a-b"].["c-d"]
^
1 compile error

Two questions/possible bugs:

Should ."a-b" work (in the example above)?
How to specify c-d (i.e. nested key with special characters)?

ghost · 2014-03-08T19:19:22Z

Okay, so, to settle this question already:

If the key has no special characters, access it like .this.

If the key has special characters, access it like ["this"].

So, @zasran, what you're looking for is .["a-b"]["c-d"].

zasran · 2014-03-10T20:24:59Z

Thanks, .["a-b"]["c-d"] does work.

It still looks a bit inconsistent, maybe I don't understand what dot does or something like that (I read the docs at http://stedolan.github.io/jq/manual/).

When there are no special characters this works:

$ echo '{"ab":{"cd":319}}' | jq '.ab.cd'

But if there are special characters it works without the dot (dot actually causes error):

$ echo '{"ab":{"c-d":319}}' | jq '.ab["c-d"]'
319

This seems a bit inconsistent, why is there a dot separator there for non-special character keys and no dot separator when the ["c-d"] format is used? (maybe calling dot separator is not correct but not sure what to call it)

On a similar note, why does the documentation claim that ."foo$" would work (Basic filters section)? As far as I can tell it claims that .foo should be replaced by ."foo$" if the key contains special characters. The .["foo"] version is also mentioned. But what you are saying is that it really should be ["foo$"](no dot).

Example straight from the docs (last .foo example in Basic filters section)):

echo '{"foo": 42}' | jq '."foo"'
error: syntax error, unexpected QQSTRING_START, expecting $end
."foo"
^
1 compile error

Can you please shed some light on this, maybe change the docs so that it's more obvious? Or maybe it's actual bug and it should work as described in the docs?

nicowilliams · 2014-03-10T21:00:54Z

Dot (.) refers to "the input to the current expression", roughly.

The ."string" syntax (equivalent to .["string"]) is not in a released version, but it is in the master branch.

nicowilliams · 2014-03-10T21:01:32Z

I'll fix the docs to list what's in 1.3 and what's new since at some point. Also, I think we'll do a new release soon.

zasran · 2014-03-10T21:08:18Z

@nicowilliams thanks, that clarifies it!

akram · 2014-05-05T14:15:00Z

Hi all,
@nicowilliams
thank you for the great job, this tool really saves me a lot of time.
Unfortunately I am facing this issue, and it seems that a release was planned for soon.
correct me if I am wrong but 1.3 is 1year old.

Do you have an estimate date for 1.4 ?

By the way could you please summarise if a workaroud exist for 1.3 ?
Here is my syntax:
jq ".name.PS_Perm_Gen.peak-usage.init" < /tmp/localhost-localhost-9990-Memory

And, I've tried:
jq .["name.PS_Perm_Gen.peak-usage.init"] < /tmp/localhost-localhost-9990-Memory
error: Invalid numeric literal (while parsing '..')

and that:
jq .name.PS_Perm_Gen.["peak-usage"].init < /tmp/localhost-localhost-9990-Memory
error: syntax error, unexpected '[', expecting IDENT
.name.PS_Perm_Gen.[peak-usage].init

both does not work

greetings
Akram

nicowilliams · 2014-05-05T15:16:55Z

I didn't finish setting up build envs for Win64 in time and then business
travel intervened. I'll get to it soon, but not this week.

ghost · 2014-05-14T16:20:35Z

@akram jq .name.PS_Perm_Gen["peak-usage"].init is what you're looking for.

There are two syntaxes for element access: .foo and ["foo"]. Note the lack of leading dot in the second case. The second syntax allows for extraneous characters, the first one does not.

If you're using master, as well as in the future 1.4 version, ."foo" is a valid option as well. It also allows extraneous characters.

hughmandeville · 2014-12-02T19:59:15Z

Wrap hyphenated name in escaped quotation marks.

jq "[.[]|{\"my-field\"}]" test.json

[
  {
    "my-field": "val1"
  },
  {
    "my-field": "val2"
  }
]

ereyes01 · 2015-09-20T17:10:34Z

I think it's important to note that if you ever script the query string to jq, it would be unsafe in the generic case to not use the jq ".[$keyname]" notation if the names of your keys are not statically known/defined. A simple jq ".$keyname" would break if $keyname ever has a hyphen in it.

ereyes01 · 2015-09-20T17:22:52Z

In fact, it seems like it would be more correct if, in your parser, you would prioritize matching key names over evaluating operators.

example: echo '{"key-10": 10}' | jq ".key-10"

Ideally would print "10"... and:

echo '{"key-10": 10}' | jq ".key-10-10"

would print "0". I say this of course, after having thought about all the possibilities for a full 5 minutes or so :-)

dtolnay · 2015-09-20T17:26:25Z

Then you have a problem if someone wants to subtract 10 from a key:

echo '{"key": 11, "key-10": 12}' | jq ".${KEY}-10"

This would require the even more confusing ".${KEY}-(10)" or "(.${KEY})-10".

hexsel · 2016-03-24T20:07:57Z

Just for posterity, the following seems to work:

jq .\"a-b\"

Seems like the shortest working query string with a hyphen.

mcwumbly · 2017-07-11T17:45:55Z

I ran into this issue today and this workaround in the comment above doesn't seem to work for me in jq 1.3. Anyone have any tips for what works today when strings have dashes in them?

pkoppstein · 2017-07-11T18:12:56Z

@mcwumbly - The short of it is that, given a JSON object, it's always safe to use the form: .[ $keyname ] where $keyname is a JSON string.

Note also that the fact that .X | .Y works for some X and Y does NOT always mean that .X.Y will work.

For example, .[0] | .["foo-bar"] is valid, but .[0].["foo-bar"] is not.

Alternatively:

$echo '[{ "foo-bar":"baz" }]' | jq '.[0][ "foo-bar" ]'
"baz"

wtlangford · 2017-07-11T19:00:14Z

I should also point out that jq-1.3 is quite old. jq-1.5 is the current release, and jq-1.6 is hopefully released soon (pending mine or @nicowilliams' schedules). As for jq-1.3, is that the package provided by your system or by your sysadmin? In the latter case, I'd recommend pestering them to upgrade you to 1.5 (which is much nicer in a lot of ways, and should not have any backwards incompatibilities). If the former... I'm going to guess you're on Debian. ;)

…

On Tue, Jul 11, 2017 at 2:13 PM pkoppstein ***@***.***> wrote: Assuming the input is a JSON object, it's always safe to use the form: .[ $keyname ] where $keyname is a JSON string. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#38 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADQ4V3oOzLiAKHI6P03w7MAXr1rDQffqks5sM7stgaJpZM4AN_5L> .

mcwumbly · 2017-07-12T01:35:49Z

thanks @pkoppstein - that helps, and this works:

echo '{ "foo-bar":"baz" }' | jq '.[ "foo-bar" ]'

pierreprinetti · 2017-07-26T08:14:41Z

This works:

echo '[{ "foo-bar":"baz" }]' | jq '.[0]' | jq '.[ "foo-bar" ]'

# "baz"

This doesn't:

echo '[{ "foo-bar":"baz" }]' | jq '.[0].[ "foo-bar" ]'

# jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
# .[0].[ "foo-bar" ]
# jq: 1 compile error

nicowilliams · 2017-07-26T15:25:32Z

Instead, this does not seem to work: ```shell echo '[{ "foo-bar":"baz" }]' | jq '.[0].[ "foo-bar" ]' ```

You don't need that second period. Use `jq '.[0]["foo-bar"]'`. That is, to chain index operations you must not use a period between the index operators. Only the first index operator needs a period (so as to distinguish it from the array constructor syntax). We should probably make it so that extra period is not a syntax error, as this is a bit of an FAQ. (Replying by email, so I think the formatting won't show correctly. But I'll set the MIME type to see if I can get it to work.)

pkoppstein · 2018-01-09T16:25:26Z

key="foo-bar"; echo '[{ "foo-bar":"baz" }]' | jq '.[0].[ "$key" ]'

There are two problems with the above. First, the abbreviation .[0].["KEYNAME"] is not (currently) supported. The second has to do with the relationship between the shell or environment variable $key and the jq variable $key.

In a nutshell, here is what is recommended:

key="foo-bar"; echo '[{ "foo-bar":"baz" }]' | jq --arg key "$key" '.[0]|.[ $key ]'

For future reference, please ask usage questions at stackoverflow.com with the jq tag: https://stackoverflow.com/questions/tagged/jq

dhjw · 2019-04-18T10:25:19Z

If you need to pass a variable containing a string with a dash and save it in a variable in a bash script...

result=$(jq --arg a "$query" '.[$a]' -r some.json)

jq does not like dashes see jqlang/jq#38 changelog_begin changelog_end

ilkka closed this as completed Oct 31, 2012

stedolan added a commit that referenced this issue Dec 3, 2012

Extend {foo} syntax to allow {"foo"} as well.

a88d53d

Useful when "foo" contains unusual characters. Should help with the issues #7, #38, #40, #42.

ghost mentioned this issue Apr 2, 2014

Can dashes be escaped in entity names? #202

Closed

dtolnay added the support label Jul 27, 2015

kislyuk mentioned this issue Feb 12, 2018

Error when name contains a dash (-) kislyuk/yq#20

Closed

mzegla mentioned this issue Mar 22, 2019

Add jq version note to installer readme intel/inference-model-manager#73

Merged

johnSchnake mentioned this issue Aug 15, 2019

Add Details map to post-processed Items vmware-tanzu/sonobuoy#838

Merged

pschmitt mentioned this issue Feb 27, 2021

bashio::config does not work for options that contain dashes hassio-addons/bashio#69

Closed

cocreature added a commit to digital-asset/daml that referenced this issue Jun 23, 2021

Fix speedy perf reporting

a4c8023

jq does not like dashes see jqlang/jq#38 changelog_begin changelog_end

cocreature mentioned this issue Jun 23, 2021

Fix speedy perf reporting digital-asset/daml#10100

Merged

6 tasks

cocreature added a commit to digital-asset/daml that referenced this issue Jun 23, 2021

Fix speedy perf reporting (#10100)

5bf7d8f

jq does not like dashes see jqlang/jq#38 changelog_begin changelog_end

happz mentioned this issue Jul 14, 2023

Simplify reportportal plugin test using yq teemtee/tmt#2191

Merged

matthewfeickert mentioned this issue Sep 18, 2023

jqlang release v1.7 breaks packtivity yadage/packtivity#96

Closed

5 tasks

brucejo75 mentioned this issue Apr 29, 2024

Cannot select sub-field that has a dash in the name of the field #3100

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot select field if field name has dashes #38

Cannot select field if field name has dashes #38

ilkka commented Oct 25, 2012

stedolan commented Oct 25, 2012

ilkka commented Oct 31, 2012

woohgit commented Apr 29, 2013

ghost commented Apr 29, 2013

zasran commented Mar 6, 2014

ghost commented Mar 8, 2014

zasran commented Mar 10, 2014

nicowilliams commented Mar 10, 2014

nicowilliams commented Mar 10, 2014

zasran commented Mar 10, 2014

akram commented May 5, 2014

nicowilliams commented May 5, 2014

ghost commented May 14, 2014

hughmandeville commented Dec 2, 2014

ereyes01 commented Sep 20, 2015

ereyes01 commented Sep 20, 2015

dtolnay commented Sep 20, 2015

hexsel commented Mar 24, 2016

mcwumbly commented Jul 11, 2017

pkoppstein commented Jul 11, 2017 •

edited

Loading

wtlangford commented Jul 11, 2017 via email

mcwumbly commented Jul 12, 2017

pierreprinetti commented Jul 26, 2017 •

edited

Loading

nicowilliams commented Jul 26, 2017 via email

pkoppstein commented Jan 9, 2018

dhjw commented Apr 18, 2019

Cannot select field if field name has dashes #38

Cannot select field if field name has dashes #38

Comments

ilkka commented Oct 25, 2012

stedolan commented Oct 25, 2012

ilkka commented Oct 31, 2012

woohgit commented Apr 29, 2013

ghost commented Apr 29, 2013

zasran commented Mar 6, 2014

ghost commented Mar 8, 2014

zasran commented Mar 10, 2014

nicowilliams commented Mar 10, 2014

nicowilliams commented Mar 10, 2014

zasran commented Mar 10, 2014

akram commented May 5, 2014

nicowilliams commented May 5, 2014

ghost commented May 14, 2014

hughmandeville commented Dec 2, 2014

ereyes01 commented Sep 20, 2015

ereyes01 commented Sep 20, 2015

dtolnay commented Sep 20, 2015

hexsel commented Mar 24, 2016

mcwumbly commented Jul 11, 2017

pkoppstein commented Jul 11, 2017 • edited Loading

wtlangford commented Jul 11, 2017 via email

mcwumbly commented Jul 12, 2017

pierreprinetti commented Jul 26, 2017 • edited Loading

nicowilliams commented Jul 26, 2017 via email

pkoppstein commented Jan 9, 2018

dhjw commented Apr 18, 2019

pkoppstein commented Jul 11, 2017 •

edited

Loading

pierreprinetti commented Jul 26, 2017 •

edited

Loading