Skip to content

Add support for JSON formatting / Always use stderr for error messages#5

Merged
egrtechno merged 5 commits intoTechnolution:masterfrom
cipriancraciun:master
Jul 16, 2018
Merged

Add support for JSON formatting / Always use stderr for error messages#5
egrtechno merged 5 commits intoTechnolution:masterfrom
cipriancraciun:master

Conversation

@cipriancraciun
Copy link
Copy Markdown
Contributor

@cipriancraciun cipriancraciun commented Jul 12, 2018

The first small patch replaces println! with eprintln! in main so that errors are printed to stderr, and thus allowing one to better integrate rustig into other pipelines.

The other four patches add support for JSON formatting of the analysis.
The format used is in fact a "JSON-stream" (i.e. one object after another separated by empty lines), which are perfect for consumption by the jq tool.

For example if one dumps the contents of the analysis of rustig itself into a JSON file, then one can execute the following jq query that prints all the "top" functions that might cause a panic, but whose "pattern" is not trivial (like unwrap, indexing, or airthmetic):

./target/release/rustig --json --binary ./target/debug/rustig > /tmp/rustig.json
jq --slurp '
    map(select(.pattern != "unwrap" and .pattern != "arithmetic" and .pattern != "indexing"))
    | map([.pattern, .backtrace[0].procedure.linkage_name_demangled])
    | unique
    | .[]
' < /tmp/rustig.json

@cipriancraciun cipriancraciun changed the title Always use stderr for error messages Add support for JSON formatting / Always use stderr for error messages Jul 12, 2018
@egrtechno
Copy link
Copy Markdown
Contributor

Thanks for your pull request. I'm very happy that you find our tool useful and that you added a feature we also wanted. I'll take a look at it shortly!

Comment thread src/output.rs
impl OutputStream for JsonConsoleOutputStream {
fn print_output(&self, panic_calls: &PanicCallsCollection) {
let stream = io::stdout();
for (i, trace) in panic_calls.calls.iter().enumerate() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are missing an outer Json Array here. Some tools consider the output invalid JSON.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be considered as a separate option.

For example the best way to handle large JSON files, especially when the "root" object is a large array of similar objects, is to render them as a "stream" of JSON objects. (For example the jq tool works like this natively, and there is also an RFC describing this https://tools.ietf.org/html/rfc7464 )

In the end the difference is minimal, and I can provide the patch for this extra version:

--json would create that large array;
--json-sequence would create the format I've already proposed;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I was not aware of the spec. Does the current implementation indeed implement the spec, which says: JSON-sequence = *(RS JSON-text LF)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately no, it doesn't use the record separator character, mainly because I thought having this "in-between" format is "good-enough", especially since works by default with jq, and it can easily be changed with a sed script into an actual JSON-object or JSON-sequence.

Moreover fully implementing that RFC would require one to lose all the pretty-printing.

Therefore perhaps the option --json-sequence should output a format compliant with that RFC, and another one --json-stream could retain the current "good-enough" format.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your reasoning. Let's just change to option name and be done with it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then just call it --json-stream which I guess it's the best "description".

Comment thread src/output.rs
let json = json!({
"index" : i,
"pattern" : match trace.pattern.borrow().deref() {
PanicPattern::Unrecognized => "unrecognized",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is this the best place to do this translation, or can we use the Display trait for this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say it's better to have all JSON-specific formatting explicit, especially since it allows the Display format to change without breaking the JSON rendering.

(I.e. you could view JSON as an "external" machine-readable format, that shouldn't change in backward-compatible ways.)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

@egrtechno
Copy link
Copy Markdown
Contributor

Sorry, wrong button. Did not want to approve the entiry PR. I'm new to using GitHub.

Copy link
Copy Markdown
Contributor

@egrtechno egrtechno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to fix some Json output details.

@egrtechno egrtechno merged commit b9c4a05 into Technolution:master Jul 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants