Skip to content

Conversation

@silverjam
Copy link
Contributor

@silverjam silverjam commented Mar 27, 2020

  • Formalize the Rust variant of the sbp2json, json2sbp, and json2json tools, once merged we'll be able to install these tools with cargo via cargo install --git https://github.com/swift-nav/libsbp.git --bins.

  • The tools are tested with various "round trip" tests that ensure that data converted into and out of the JSON representation of the tool is bit-for-bit identical.

  • Introduce a benchmark framework for the Haskell, Python and Rust versions of these tools using hyperfine.

  • Adds builds on Mac OS X and Windows so that we can provide more platform specific builds of the sbp2json, json2sbp, and json2json tools.

  • Behavior difference with Haskell: the float formatting of Haskell's sbp2json uses GHC's showFloat behavior for formatting float values, which is explained in Python sbp2json tool source -- the Python sbp2json tool matches this formatting exactly, the Rust tool does not. The Rust tool attempts to provide some compatibility with a --float-compat switch but falls short of the Python variant. This behavior difference does not impact actual float values stored in SBP binary (for example when converting Rust SBP JSON output back to SBP binary with json2sbp).

  • Behavior difference with Haskell: the sbp2json tool in Haskell would pass through "messages" that failed CRC validation. It's unclear of this choice was intentional but it makes the Haskell tool not "bit-for-bit" compatible with the Rust tool if the data stream includes "junk". The Rust tool will drop the invalid data. The Haskell tool will pass through the junk data, but as a JSON blob with just common fields such as "message type", "payload" and "crc"-- even if these fields represent junk data.

@silverjam silverjam changed the title rust: implement sbp2json, json2sbp, json2json rust: implement sbp2json, json2sbp, json2json [INFRA-125] Mar 27, 2020
Copy link
Contributor

@jbangelo jbangelo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of changes here, so it's difficult to review everything in depth. Looks good to me. I did have suggestions for some minor changes.

I'd like to come to a consensus on the Parser object as well, but that's probably less urgent until we start deploying releases to crates.io

///
/// ```
///
pub fn sbp2json_read_loop(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be handling the same kind of logic that I meant the Parser struct to in the main library. I don't think the Parser is particularly ergonomic as it current stands, and the fact that you had to implement something similar seems to confirm that.

Do you think we should reconsider the Parser object design? I think this is as good a time as any, and the longer we leave it as is makes it more difficult to change it later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are definitely things that could be more ergonomic, but the buffer logic here is a pretty verbatim copy of sbp2json in Python: https://github.com/swift-nav/libsbp/blob/master/python/sbp/sbp2json.py#L187 -- are you envisioning that most of this logic would live inside the parser object?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, there's already some buffering logic in the parser object but it could be changed to mirror the python logic and the interface could probably made to be more rusty.

Jason Mobarak added 2 commits March 31, 2020 12:39
Also fix the template for the addition of the test deps.
Copy link
Contributor

@benjaminaltieri benjaminaltieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built and ran on my machine super easy. Enjoyed a chance to look over the code and see how things are organized. Overall great stuff, can't say I had much to comment on in terms of design choices but it's an extensive solution with a lot of great tooling. Thanks for the hard work @silverjam!

@silverjam silverjam merged commit c8d25c2 into master Apr 1, 2020
@silverjam silverjam deleted the silverjam/sbp2json-rs branch April 1, 2020 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants