Skip to content

Commit

Permalink
allow to retrieve multi objects in a single one command...
Browse files Browse the repository at this point in the history
... by introducing the additional-objects parameter

Can be tested with:

```
niet ".configuration.object_type | keys(@)[0]"
https://gist.githubusercontent.com/4383/9538953081f2e475903a8208e834b954/raw/97ffc80acc6d9a9293cae44334707419e27fd98f/config
-a ".configuration.object_type.schema.[keys(@)[0], values(@)[0]]"
```

The previous command will output:

```
schema
schema1
/path/to/schema1.sql
```
  • Loading branch information
4383 committed Apr 21, 2023
1 parent 6e96fe7 commit 9caa788
Show file tree
Hide file tree
Showing 3 changed files with 93 additions and 25 deletions.
5 changes: 5 additions & 0 deletions ChangeLog.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
CHANGES
=======

Not yet released
----------------

* [feature] Introduce the `additional-objects` parameter to allow multi researches

3.1.0
-----

Expand Down
77 changes: 57 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,17 +38,13 @@ needed to install it.

## Main Features
- Extract elements by using xpath syntax
- Extract values from JSON format
- Extract values from YAML format
- Extract values from TOML format
- Extract values from JSON, YAML, and TOML format
- Automaticaly detect format (json/yaml)
- Read data from a web resource
- Read data from file or pass data from stdin
- Format output values
- Format output to be reused by shell `eval`
- Convert YAML to JSON, or TOML
- Convert JSON to YAML, or TOML
- Convert TOML to YAML, or JSON
- Convert YAML to JSON, or TOML and vice versa

## Install or Update niet

Expand All @@ -75,29 +71,27 @@ higher version.

```shell
$ niet --help
usage: niet [-h] [-f {json,yaml,toml,eval,newline,ifs,squote,dquote,comma}] [-s] [-v]
object [file]
usage: niet [-h] [-a ADDITIONAL_OBJECTS [ADDITIONAL_OBJECTS ...]] [-f {json,yaml,toml,eval,newline,ifs,squote,dquote,comma}] [-i] [-o OUTPUT_FILE] [-s] [-v] [--debug] object [file]

Read data from YAML or JSON file

positional arguments:
object Path to object separated by dot (.). Use '.' to get
whole file. eg: a.b.c
file Optional JSON or YAML filename. If not provided niet
read from stdin
object Path to object. Based on jsmespath identifiers (https://jmespath.org/specification.html#identifiers) Use '.' to get whole file. (eg: a.b.c)
file Optional JSON or YAML local filename or distant web resource at raw format. If not provided niet read from stdin

optional arguments:
options:
-h, --help show this help message and exit
-a ADDITIONAL_OBJECTS [ADDITIONAL_OBJECTS ...], --additional-objects ADDITIONAL_OBJECTS [ADDITIONAL_OBJECTS ...]
Path to additional objects to search. Here you can pass a list of additional researchs. Allow you to combine researchs into the same command call. The researchs will be made on the original file as with the
`object` parameter. Niet will output the results sequentially without delimiter between the results. If the `--output` argument is given by user, the results are appended at the end of the file sequentially. Based
on jsmespath identifiers (https://jmespath.org/specification.html#identifiers) Use '.' to get whole file. (eg: a.b.c)
-f {json,yaml,toml,eval,newline,ifs,squote,dquote,comma}, --format {json,yaml,toml,eval,newline,ifs,squote,dquote,comma}
output format
-i, --in-place Perform modification in place. Will so alter read file
-o OUTPUT_FILE, --output OUTPUT_FILE
Print output in a file instead of stdout (surcharged
by infile parameter if set)
-s, --silent silent mode, doesn't display message when element was
not found
-v, --version print the Niet version number and exit (also
--version)
Print output in a file instead of stdout (surcharged by in-place parameter if set)
-s, --silent silent mode, doesn't display message when element was not found
-v, --version print the Niet version number and exit (also --version)
--debug Activate the debug mode (based on pdb)
output formats:
Expand Down Expand Up @@ -237,7 +231,7 @@ $ echo $?
See the [related section](#deal-with-errors) for more info on how to manage
errors with `niet`.
Niet is based on `jmespath` to find results so for complex research you can
Niet is based on `jmespath` to find results so for complexe research you can
refer to the [jmespath specifications](http://jmespath.org/specification.html#identifiers)
to use identifiers properly.
Expand Down Expand Up @@ -293,6 +287,49 @@ niet project.'"test-dash"' tests/sample/sample.json
Further examples with [`jmespath` identifiers](http://jmespath.org/specification.html#examples).
### Additional objects
Additional objects allow you to combine more than one query.
The `--additional-objects` parameter accept a list of objects strings.
These objects are the same thing that the base object used to query
your inputs.
This parameter allow to generate advanced output from your input, let see an
example:
Consider the following yaml example:
```
configuration:
warehouse: warehouse-name
database: database-name
object_type:
schema:
schema1: "/path/to/schema1.sql"
schema2: "/path/to/schema2.sql"
```
The following command will allow us to generate an output constitued from the
results of the two objects used as query:
```
$ niet ".configuration.object_type | keys(@)[0]" config.yaml
-a ".configuration.object_type.schema.[keys(@)[0], values(@)[0]]"
```
The previous command will output:
```
schema
schema1
/path/to/schema1.sql
```
This output wouldn't be possible without combining the result of two queries,
the additional objects are made for that.

Outputs of these additional objects are printed sequentially in the order they
are given in the command line.

### Output

#### Stdout
Expand Down
36 changes: 31 additions & 5 deletions niet/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,10 @@ def argparser():
parser.add_argument(
"object",
type=str,
help="Path to object separated by dot (.). \
help="Path to object. Based on jsmespath identifiers \
(https://jmespath.org/specification.html#identifiers) \
Use '.' to get whole file. \
eg: a.b.c",
(eg: a.b.c)",
)
parser.add_argument(
"file",
Expand All @@ -98,6 +99,24 @@ def argparser():
distant web resource at raw format. \
If not provided niet read from stdin",
)
parser.add_argument(
"-a",
"--additional-objects",
nargs="+",
help="Path to additional objects to search. Here you can pass a list \
of additional researchs. Allow you to combine \
researchs into the same command call. \
The researchs will be made on the original file as with \
the `object` parameter. \
Niet will output the results sequentially without \
delimiter between the results. \
If the `--output` argument is given by user, the results \
are appended at the end of the file sequentially. \
Based on jsmespath identifiers \
(https://jmespath.org/specification.html#identifiers) \
Use '.' to get whole file. \
(eg: a.b.c)",
)
parser.add_argument(
"-f",
"--format",
Expand Down Expand Up @@ -196,7 +215,7 @@ def get(data, keywords, silent=False):
sys.exit(1)


def print_result(res, out_format, in_format, search, out_file):
def print_result(res, out_format, in_format, search, out_file, mode="w+"):
if out_format is None:
if isinstance(res, (list, str, int, float)) or in_format is None:
out_format = "newline"
Expand All @@ -213,8 +232,8 @@ def print_result(res, out_format, in_format, search, out_file):
print("Supported formats are: {format}".format(",".join(VALID_PRINTERS)))
else:
if out_file:
with open(out_file, "w+") as f:
f.write(output)
with open(out_file, mode) as f:
f.write(f"{output}\n")
else:
print(output)

Expand Down Expand Up @@ -266,3 +285,10 @@ def main():
out_file = infilename

print_result(result, out_format, in_format, search, out_file)

additional_objects = args.additional_objects
if additional_objects:
for el in additional_objects:
result = get(data, el, silent)

print_result(result, out_format, in_format, search, out_file, mode="+a")

0 comments on commit 9caa788

Please sign in to comment.