separate interface from data

[callowaylc]

Let's say I want to determine the entry node ip address and country, using the cli, our starting point looks something like (shortened for brevity):

$ carml circ --list --verbose | tee /tmp/out | head -n5
Circuits:
  ID   | Age   | Path (router names, ~ means no Named flag) | State    | Purpose
 ------+-------+--------------------------------------------+----------+------------
    31 |  177s | ~niftyshrew->~setsun->~tor01aditaa         | BUILT    | GENERAL
         BUILD_FLAGS=NEED_CAPACITY, PURPOSE=GENERAL, TIME_CREATED=2020-05-26T18:08:52.059798
...

To get the entry node ip address and country code, from a cli, I could do:

$ cat /tmp/out | grep -Eo '[0-9.]+\s[A-Z]{2}' | head -1
12.34.56.78 FR

Which is, of course, insane - the table layout is fine and all, but the purpose of the cli is provide convenient interface BETWEEN PROCESSES, so that we can programmatically interface-with and build pipelines-around expected data.

It would be nice if i could just list the circuits and nodes, as space and newline separated, which empowers the end user to do what they wish with the data; instead I have to work to strip out all those pretty formatting concerns.

Cheers

[callowaylc]

And to just to intercept an expected response, which would go something like "just use the sdk and add as small python script".. That's fine, except for the dev, commit, codify, orchestrate and management of an additional concern, when that concern can be addressed with a single statement on the command line.

Cheers again

[meejah]

Yeah, for sure some of this is more suited for "human reading the command-line" rather than "some other process".

At some point, I had thought a global --json option might be good; would this solve your problem, or just make it hard "because now there's json"?

Meantime, you might get slightly-more-pipe-able output by using the "raw" control-protocol commands, like carml cmd getinfo circuit-status

[callowaylc]

literally the fastest response ever on a submitted issue, which I mean genuinely (lol, though it looks sarcastic as hell)

Yeah, for sure some of this is more suited for "human reading the command-line" rather than "some other process".

Yeah, I get that, but everytime I see it, I feel like its reflexive solution (humans need interfaces) to a problem that doesn't exist (eg, we use something like grafana to visualize data, and the command line to build ETL pipelines).

At some point, I had thought a global --json option might be good; would this solve your problem, or just make it hard "because now there's json"?

Personally speaking, I love jq, and I think json makes for a more "approachable" option for your end users (ie, easier to market), but if it takes 2 months for what should be 30 minutes of work, I would just opt for space, newline and null character delimited STDOUT.

Meantime, you might get slightly-more-pipe-able output by using the "raw" control-protocol commands, like carml cmd getinfo circuit-status

What, learn the low level interface, that your sdk is intended to abstract in the first place - that's crazy talk mijo; that reminds me of a guy who spent a multi-sprint cycle on building tests for already existing tests, while telling an anxious project manager that he hadn't committed any "actual" work on the requested feature. The end result, like in that situation, is that people tend to move on to the next thing.

Sorry, I am clearly bored, but yeah, sed/grep provide enough to strip unwanted tokens, and please, take the above as drumbeat around philosophical use of command line as opposed to personal invective. Outside of that, it's good work; I chose this over stem and txtorcon for reasons other than cli.

cheers

[meejah]

probably worth noting that this uses txtorcon under the hood ;)

JSON is pretty easy to output from Python. I assume "one JSON object per line, per circuit" is going to be the easiest to pass onwards in the pipeline..?

[callowaylc]

probably worth noting that this uses txtorcon under the hood - yep, I know; my concern as an end-user is what your interface provides, not what engine your encapsulates. It could be built on top of wordpress - if the tool fit my needs or if I am feeling particularly sadistic, its its own atomic concept.

JSON is pretty easy to output from Python. I assume "one JSON object per line, per circuit" is going to be the easiest to pass onwards in the pipeline..?

Up to you, but that would be my preference - stream of json objects, compressed and newline seperated, as opposed to an array of json objects, because it makes streaming problematic and because everything in shell world is organized as record-per-line.