should rust be abandoned for fix protocol?

[PipsqueakH]

hi, @jbendig 
the current status of fix-rs is suspended, as " long compile times, poor networking libraries, and difficulty eliminating allocations while remaining ergonomic."
actually, i am looking for some solution for high performance fix protocal implementation for some financial system, do you think i should use rust to develop such kind of system? and could you give some more details about the difficulties in this project?

[jbendig]

Hello @PipsqueakH! If performance is your biggest concern, I recommend using one of the existing open source C++ libraries which have been optimized and battle tested.

I don't know if Rust should be abandoned for FIX but I don't think it's the right choice today. The first two problems I listed are being worked on but the third is a problem with Rust itself.

1. long compile times: I've implemented FIX messages using macros because so much of the code is boilerplate for defining traits, implementing serialization, querying fields, etc. But only a small amount of messages are currently defined because the compile times are already so slow. Waiting tens of seconds to several minutes every time I want to run a test or benchmark was slowing down development drastically.

2. poor networking libraries: Right now, fix-rs uses mio for networking and does all of its processing in a single thread. The performance is reasonable but I have concerns about scalability and ease of also polling asynchronous IO for disk or database access.

   It originally looked to me like tokio might make these problems disappear but that's not the case. In its current state, tokio is slower (in terms of bandwidth and latency) than using mio directly. It's very hard to use and produces cryptic error messages. Arguably worst of all, the resulting code becomes an unmaintainable mess of spaghetti.

3. difficulty eliminating allocations: This just comes down to the borrow checker trying to control everything in a way that it understands. It's awesome when it works but sometimes it doesn't. For example, I've found that it forces needless cloning instead of safely casting and it's difficult to reclaim memory that has been sent over threads.

I think today's Rust fits into a niche where you want safety above all else, at least ~95% of C++'s performance, and don't depend on third-party libraries.

[PipsqueakH]

thank you so much for your most valuable information and good luck to u.

[mjmeehan]

Do you feel the issues are well understood/communicated to the Tokio and Rust core teams?

[jbendig]

@mjmeehan Yes, I do. There was a recent discussion where people covered most of the problems I've had: https://users.rust-lang.org/t/what-does-rust-need-today-for-server-workloads/11114

I feel like tokio is in a situation similar to what happened with KDE 4.0. It was pushed too early as an already usable platform. In reality, it's more of an experiment that needs many iterations and will hopefully work completely differently some day.

[emileindik]

@jbendig Even though development has been halted, thank you for this library.
I'm wondering if you think it is in a usable state. I'd be using it as a client with FIX 4.2, looking to send and receive fairly standard messages. Do you think the library, in its current state, is still able to support that kind of client functionality?

[jbendig]

@emileindik I would not recommend using this library in its current state. It has been awhile since I last worked on fix-rs but two really big hang-ups come to mind regarding FIX 4.2.

1. Some edge cases at the administrative level are not handled properly. This is reason enough to not run this on a network you don't entirely control.
2. Not all valid messages types are defined. I left them out during development to help keep the compile times down. You can define the ones you need using the provided macros but it's very repetitive. (Messages are implemented as structs for compile time safety and performance. Large complex macros are used to generate the appropriate Rust code. However, I definitely should have written a tool to generate the code automatically from the XML documents provided as part of the FIX standard.)

[jakeschurch]

hi @jbendig !

I know that you halted on development a while ago on this - but have you looked at the recent state of the rust ecosystem to determine whether or not you feel that it is right to begin development on this again? Do you plan to pick up this crate in the future at any time?

Thanks!

[freetopos]

hello @jbendig,

I was going to ask the same question as @jakeschurch. Are there any plans to pick up on the development of this crate? I am seriously considering using Rust for a project and a library like yours would be really useful.

Thanks.

[rucoder]

And here we are again in April 2020. Today's state of Async IO in Rust is way better. IMO async_std or Tokio 0.2 is a way to go. Moreover, Rust 2018 gives you much more power in controlling memory allocations. I'm going to work on FIX/FAST anyways regardless of the state of this library but It would be nice to revisit it

[Boscop]

@rucoder Nice. I think Rust is a good fit for this domain.

@jbendig Btw, you can use something like Arc or ArcSwap to reclaim memory of things you send to other threads!

Btw, I've been using Rust since 2014 and I haven't found that it leads to unnecessary cloning at all. Usually you can avoid cloning by adding references to your types (OR using clone-on-write OR using something like Rc).

[neysofu]

Given the state of this crate, about a year ago I started experimenting on my own to lay the foundations for another FIX implementation in Rust. I recently resumed development and I've got a generous sponsor which helps greatly. The name of the project is FerrumFIX.

Mind you, it's in a very early stage. It kinda works, but you should refrain from using it in any meaningful way for now. I'm just posting this in case someone wants to take a look at the APIs or offer suggestions and get valuable feedback.

Hopefully Rust will have a nice and up-to-date FIX library in a few months :)

[cjdsellers]

Hi all,

So some more time has passed and I see @neysofu has started their own project which is looking quite good indeed.

I'm interested to hear if @jbendig has any updates to their opinions about the state of the Rust ecosystem - both the language itself, and the tokio project?

I'm quite new to the language myself and see its massive potential.