Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: add notes on command line quoting #1844

Merged
merged 5 commits into from Nov 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
34 changes: 33 additions & 1 deletion website/docs/advanced/override_grammar/basic.md
Expand Up @@ -147,6 +147,29 @@ Quoted strings can accept any value between the quotes, but some characters need
</div>
</div>

It may be necessary to use multiple pairs of quotes to prevent your
shell from consuming quotation marks before they are passed to hydra.

```shell
$ python my_app.py '+foo="{a: 10}"'
foo: '{a: 10}'

$ python my_app.py '+foo={a: 10}'
foo:
a: 10

```

Here are some best practices around quoting in CLI overrides:
- Quote the whole key=value pair with single quotes, as in the first two
examples above. These quotes are for the benefit of the shell.
- Do not quote keys.
- Only quote values if they contain a space. It will work if you always quote
values, but it will turn numbers/dicts/lists into strings (as in the first
example above).
- When you are quoting values, use double quotes to avoid collision with the
outer single quoted consumed by the shell.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are good basic rules that should cater for almost all cases.

BTW double quoting in the shell is more general than single quoting, as you can specify any character through the use of escapes (you can't have literal single quotes within single quoted args for example). But I wouldn't bother mentioning that in the doc as it's shell specific

### Whitespaces in unquoted values
Unquoted Override values can contain non leading or trailing whitespaces.
For example, `msg=hello world` is a legal override (key is `msg` and value is the string `hello world`).
Expand All @@ -164,7 +187,8 @@ $ python my_app.py 'msg= hello world '
```

### Escaped characters in unquoted values
Some otherwise special characters may be included in unquoted values by escaping them with a `\`.
Hydra's parser considers some characters to be illegal in unquoted strings.
These otherwise special characters may be included in unquoted values by escaping them with a `\`.
These characters are: `\()[]{}:=, \t` (the last two ones being the whitespace and tab characters).

As an example, in the following `dir` is set to the string `job{a=1,b=2,c=3}`:
Expand All @@ -173,6 +197,14 @@ As an example, in the following `dir` is set to the string `job{a=1,b=2,c=3}`:
$ python my_app.py 'dir=job\{a\=1\,b\=2\,c\=3\}'
```

As an alternative to escaping special characters with a backslash, the value containing the special character may be quoted:

```shell
$ python my_app.py 'dir=A[B' # parser error
$ python my_app.py 'dir="A[B"' # ok
$ python my_app.py 'dir=A\[B' # ok
```

pixelb marked this conversation as resolved.
Show resolved Hide resolved
### Primitives
- `id` : oompa10, loompa_12
- `null`: null
Expand Down
Expand Up @@ -147,6 +147,32 @@ Quoted strings can accept any value between the quotes, but some characters need
</div>
</div>

It may be necessary to use multiple pairs of quotes to prevent your
shell from consuming quotation marks before they are passed to hydra.

```shell
$ python my_app.py '+foo="{a: 10}"'
foo: '{a: 10}'

$ python my_app.py '+foo={a: 10}'
foo:
a: 10

$ python my_app.py +foo={a: 10} # Two strings are passed to Hydra: `+foo={a:` and `10}`. This is an error.
no viable alternative at input '{a:'.
...
```

Here are some best practices around quoting in CLI overrides:
- Quote the whole key=value pair with single quotes, as in the first two
examples above. These quotes are for the benefit of the shell.
- Do not quote keys.
- Only quote values if they contain a space. It will work if you always quote
values, but it will turn numbers/dicts/lists into strings (as in the first
example above).
- When you are quoting values, use double quotes to avoid collision with the
outer single quoted consumed by the shell.

### Whitespaces in unquoted values
Unquoted Override values can contain non leading or trailing whitespaces.
For example, `msg=hello world` is a legal override (key is `msg` and value is the string `hello world`).
Expand All @@ -164,7 +190,8 @@ $ python my_app.py 'msg= hello world '
```

### Escaped characters in unquoted values
Some otherwise special characters may be included in unquoted values by escaping them with a `\`.
Hydra's parser considers some characters to be illegal in unquoted strings.
These otherwise special characters may be included in unquoted values by escaping them with a `\`.
These characters are: `\()[]{}:=, \t` (the last two ones being the whitespace and tab characters).

As an example, in the following `dir` is set to the string `job{a=1,b=2,c=3}`:
Expand All @@ -173,6 +200,14 @@ As an example, in the following `dir` is set to the string `job{a=1,b=2,c=3}`:
$ python my_app.py 'dir=job\{a\=1\,b\=2\,c\=3\}'
```

As an alternative to escaping special characters with a backslash, the value containing the special character may be quoted:

```shell
$ python my_app.py 'dir=A[B' # parser error
$ python my_app.py 'dir="A[B"' # ok
$ python my_app.py 'dir=A\[B' # ok
```

### Primitives
- `id` : oompa10, loompa_12
- `null`: null
Expand Down