Rust port of Drain (drainrs)

[haydenflinner]

Hi, the code for this is early, for example the usual parameters aren't configurable. But the core algorithm was ported over 1:1 and then cleaned up to be a little more zero-copy/type-correct. For example there shouldn't be any issue caused by "<*>" occurring in the logs. The focus was on performance, and without an LruCache it measures as considerably faster in my early tests, despite having some definitely unnecessary copying going on.

https://crates.io/crates/drainrs

[Superskyyy]

This is really cool, would you like to contribute and maintain the project in this repository as an alternative/faster implementation?

[haydenflinner]

I wouldn't mind it; I wasn't sure at first on being able to maintain a Rust create as a subdir of a git repo, but it seems it's well-supported.

I also just found this better implementation this morning; I renamed mine from rdrain at the last moment before uploading and had never seen this impl!
https://github.com/benwtrent/drain-rs

[Superskyyy]

I wouldn't mind it; I wasn't sure at first on being able to maintain a Rust create as a subdir of a git repo, but it seems it's well-supported.

I also just found this better implementation this morning; I renamed mine from rdrain at the last moment before uploading and had never seen this impl!

https://github.com/benwtrent/drain-rs

Oh wow, neither have I found there were other rust drain implements out there. But still, as an open source project, being actively maintained is better than a good implementation without maintenance(it only got one release). So if you feel like, I welcome you to join this repository and keep on growing and maintaining a new version as users would love a high performance alternative to Python.

[haydenflinner]

True! Plus this one's API is more suitable to streaming logs that don't fit in memory or live-streaming logs, whereas that one seems more tailored to a batch use-case. I will probably copy the configuration API and grokking filter from it though, that's good stuff.

There isn't much value in the current git history so I'll upload a PR here with my next refactor and archive my repo. Decided to switch off of "iterator-style" and over to callbacks, as it should be easier to use, easier on lib developer, and faster (fewer allocations).

[Superskyyy]

True! Plus this one's API is more suitable to streaming logs that don't fit in memory or live-streaming logs, whereas that one seems more tailored to a batch use-case. I will probably copy the configuration API and grokking filter from it though, that's good stuff.

There isn't much value in the current git history so I'll upload a PR here with my next refactor and archive my repo. Decided to switch off of "iterator-style" and over to callbacks, as it should be easier to use, easier on lib developer, and faster (fewer allocations).

Great! I will be expecting your PR and looking forward to working with you on this project and maybe beyond if you are also interested in the Observability domain or AIOps (I'm a contributor at Apache SkyWalking, an open source APM platform).