Survey: Crates / libraries / tools you need / want

[japaric]

Let's try to find out what libraries the embedded ecosystem lacks with a survey!

Please request one library / tool per comment. You can comment several times though. I suggest sticking to the guidelines below but feel free to ignore some of them if you think it makes sense to do so:

• Microcontroller and development board specific crates are out of scope for this survey. The requested crate must be general enough so that it can be useful to a non-trivial fraction of the embedded Rust community.

"I want a crate to program the e.g. STM32F4DISCOVERY development board" is out scope, otherwise we'd get a very long list of similar requests.

• Check the issue Survey: Crates you use for embedded development #12 first.

You may be asking for something that already exists! If a crate an crates.io or GitHub doesn't work for you, please tell us why that's the case.

• Please be as specific as possible and elaborate on your use case.

Don't just say "an Arduino-like framework that supports my microcontroller", which BTW would be out of scope because of point 1. That's too vague. Instead say "I need a low latency mechanism to send data from an interrupt handler to the main loop without disabling interrupts because I do $HARD_REALTIME_STUFF".

• If someone has already posted here a library that you also need then upvote their answer so we can get an idea of how much demand there is for such library / functionality.

• Bonus points if you can point to an existing C library that solves your problem. That way we can inform ourselves about prior art.

[genbattle]

It would be good to see statically allocated data structures like those implemented by ETL for C++. The biggest blocker for the creation of data structures with a similar API in Rust at the moment is type-level integers for Generics. There may be a way around this using a different approach in Rust, but there's also various hacks that can provide something like type-level integers.

[japaric]

I want a no-std binary serialization crate that doesn't require memory allocations and that uses fixed size buffers in its API to eliminate the possibility of runtime errors. I'm thinking of some API like this:

#[derive(BinSerialize, BinDeserialize)]
struct Foo {
    x: i32,
    y: i32, 
    z: i32,
}

// then I get methods like these
impl Foo {
    fn serialize(&self, buffer: &mut [u8; 12]) { .. }
    fn deserialize(buffer: &[u8; 12]) -> Self { .. }
}

Currently I'm using the byteorder for this task and it works wondefully but I'd like to avoid having to write the serialization / deserialization code myself.

[japaric]

statically allocated data structures

I got some (2) in the heapless crate. You can workaround the lack of integer generics by including the type of the array buffer, instead of just its size, in the data structure. So for example an std::Vec<T> backed by an array looks like Vec<T, [T; 8]> where [T; 8] is the statically allocated storage (capacity = 8).

[m4b]

Yea afaics I've literally already done this some time ago:

https://GitHub.com/m4b/scroll for the base reading traits of various flavors. Basically a pread style version where you specify an offset, and another version which takes a mutable offset and when it successfully reads it increments the offset for you.

Each trait has different methods which have all proved extremely useful for different kinds of binary reading, that I've come across in various contexts. Please see documentation.

It implements various basic types, as well as leb128 integers and allows safe, zero copy reading of strings. I've even considered adding safe marshalling from &[u8] to arrays of fixed size, e.g. [u8; N] for some N. All this is possible from the design.

For custom types, just as you have above, I have
https://GitHub.com/m4b/scroll_derive for the proc macros

You can also manually implement with whatever parsing context you want, in case endianness isn't sufficient. Again see documentation for examples (there should be a lot of it)

Nothing allocates. Errors also don't allocate.

I've used it for various tasks and some projects not online and I just find it superior to every alternative. Refactoring byteorder methods is literally horrible if something changes, and was the source of numerous manual errors in my experience.

Also I wanted to read without the bytes being mutable which I've used for great effect in parallel operations with sharded byte slices, but that isn't necessarily relevant here.

Lastly, if there's interest I can add a result less version for only a small class (e.g. numbers) when that is desired. There's also the unsafe versions which don't return a result either.

[fluffysquirrels]

@japaric wrote:

I want a no-std binary serialization crate that doesn't require memory allocations and that uses fixed size buffers in its API to eliminate the possibility of runtime errors. I'm thinking of some API like this:

For those that find their way to this thread, ssmarshal seems to this job nicely, as mentioned elsewhere.

[fluffysquirrels]

I want no_std-friendly std::io::Read and std::io::Write traits. It should be easy to write a crate that use std::io::{Read, Write} when using std and has some no_std alternative when no_std.

Repository https://github.com/QuiltOS/core-io has some nice ideas. It's not on crates.io. It has std::io-inspired Read and Write traits, but with a generic associated type as an error type. In practice I found the error type made it difficult to create an implementation of a module that worked with std and with no_std.

The core_io crate doesn't compile out of the box. It's a set of scripts that fetches the source to std::io and applies some patches to remove the requirement on the standard library.

Crate embedded-serial has similar ideas, providing traits with generic error type parameters designed for embedded peripherals to implement. It doesn't aim to be drop-in replacement for std::io::{Read, Write}, so it's hard to use in projects that must support both no_std and std.

I just did a quick search in the rust-lang/rust and rust-lang/rfc repositories and can't actually find any issues around moving std::io::{Read, Write} into core for no_std use. I'll do that shortly.

I'm working on ergonomics for framed now, and if no good options present themselves I will probably make my own crate (no_std-io or similar) to supply Read and Write traits, with the hope that several no_std crates end up supporting them.

[whitequark]

Please do something about core::io! It's been bothering me for over a year at this point, with no resolution. I've tried to do something with the core_io crate but the model seems a dead end to me.

[fluffysquirrels]

More searching for core Read, Write traits found these links:

• Error is crippled jethrogb/rust-core_io#3
• Move ::std::error to ::core::error rust-lang/rust#33149
• Port libstd to "no operating system" target rust-lang/rust#37133
• https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301
• Read Write for core BurntSushi/byteorder#51
• https://github.com/rust-lang/rfcs/blob/master/text/0517-io-os-reform.md#coreio
• RFC: io and os reform: initial skeleton rust-lang/rfcs#517
• no_std support (via an enabled-by-default "std" feature) rust-lang-deprecated/error-chain#157
• https://internals.rust-lang.org/t/crate-evaluation-for-2017-06-27-error-chain/5362/25

[fluffysquirrels]

@whitequark wrote:

Please do something about core::io! It's been bothering me for over a year at this point, with no resolution. I've tried to do something with the core_io crate but the model seems a dead end to me.

I was thinking it should be possible to have some core_io2 crate that supports compiling with and without std with some feature flags, std itself re-exports the traits from that, and no_std crates can use it directly. The problem with core_io was it keeps breaking and needing more patches whenever std::io changes, whereas with core_io2 the upstream project would have that responsibility.

I'm happy to work on a proof of concept or RFC for this if it stands a reasonable chance of getting into rust-lang/rust, but I'm not sure if it does. rust-lang/rust maintainers may not be interested in the approach or the maintenance burden. I'll post a quick issue on rust-lang/rfcs.

[pftbest]

There is also genio crate by @Kixunil. I think it has some compatibility with std::io traits, but it's using associated types for errors.

[fluffysquirrels]

I added this issue: rust-lang/rfcs#2262

[fluffysquirrels]

@pftbest wrote:

There is also genio crate by @Kixunil. I think it has some compatibility with std::io traits, but it's using associated types for errors.

Thanks for the link! I'll take a look.

[fluffysquirrels]

Wow, genio looks really good. I created a PR to add keywords to make it more visible.

[blanham]

What about bare metal support for full processors (ie x86(_64), SPARC, MIPS, etc)? They are used in the embedded space, and the support code could also be used as the basis for unikernels, hypervisors, and full kernels. There's currently a hodgepodge of x86, x86_64, and x86/x86_64 support libraries of varying quality and completeness, and I think collecting them under the rust-embedded banner would focus development.

[whitequark]

"Full processors"? x86 is just a microcontroller architecture like AVR ;P It even comes on a SoC these days.

[blanham]

I'd also love to have a tool or DSL for specifying binary file and wire formats. Kaitai Struct is an existing format in this space, though I'm not the biggest fan of it's specification language. I believe the maintainers attempted a Rust target/runtime, but gave up.

And really better support for bit access/twiddling/masking. I've never been a fan of having to write such code from scratch, and often leads to subtle errors. I think this fits Rust's goal for zero-cost abstractions, and the improved ergonomics and safety will make it much more enjoyable to write embedded code. I believe @Ericson2314 once proposed improvements to Rust's low-level handling of bits/bitfields and would love to hear his take on this.

[Kixunil]

I'd like to see an abstraction for generating waveforms. I'd love to port some of my Arduino libraries to use such abstraction and work on RPi or other stuff.

[puzrin]

Keyboard crate

A lot of devices have buttons, attached to MCU pins. Common problem is to filter jitter. Also, it would be more convenient for application to work with keyup/keydown events instead of on/off state.

• Must have: jitter filter
• Optional: auto repeat, double click
• Advanced inputs support: EC11 encoders are very popular, it would be nice to support those with jitter filter (translate inputs to left/right + button press events).

LCD displays graphics library

• Something like u8g2, preferably with more liberate license
• Some efforts been done by @jamwaffles in embedded-graphics but it's in very-very basic state.

[jamwaffles]

@puzrin What features would you like to see from embedded-graphics? It's definitely early days for the library so it would be great to get some input from the community in what direction to take it in.

[puzrin]

At first glance:

• The most demanded, more font sizes (up to 24-30px), with tools to extract required char subsets. +more compact font store format.
• (probably) More compact framebuffer format for monochrome display (1 byte per 8 pixels, in your ssd1306)
• (probably) Async inteface support

If you wish, i can analyze situation more deep and create issue with details in your repo.

[jamwaffles]

Thanks, that's really useful!

The most demanded, more font sizes (up to 24-30px)

Definitely! I just haven't got round to adding them yet.

with tools to extract required char subsets.

Interesting idea. I'll give it some thought.

+more compact font store format.

The fonts are stored as 1 byte per 8 pixels, so I'm not sure how much more compact they could get without using some sort of runtime decoding which I'd like to steer away from.

(probably) More compact framebuffer format for monochrome display

I'm not sure what you mean for embedded-graphics as it doesn't use a framebuffer at all. Can you elaborate?

If you wish, i can analyze situation more deep and create issue with details in your repo.

That would be really helpful if you don't mind spending the time doing it. Thanks!

[RandomInsano]

SPI abstraction that allows for differenet enable lines / device configurations per device. I think it may be tricky with the current API...

[RandomInsano]

Also SPI-related, bit-banged crate.

One of my single board computing devices has a completely broken hardware implementation.

[aep]

Crates for interfacing are easy to make and the community will come up with them. A major roadblock right now is the size of the existing core/jmalloc/unwind/..
If we somehow managed to have an out of the box small runtime without doing the no-std dance, this would be gold

[Kixunil]

9-bit serial (abstraction and implementation)

[Kixunil]

Basic HTTP client implementation with TLS and websockets.

[jamesmunns]

@japaric what do you think about this issue? I think we may want to refocus this issue, and form some kind of way to collect "wants" like this somewhere more visible. Maybe something like "not-yet-awesome-embedded-rust"?

[japaric]

Maybe something like "not-yet-awesome-embedded-rust"?

Sounds good to me, in principle.

[flip111]

I would like a tool like svd2rust for register maps that are on other devices (not in the memory mapped IO of the MCU). Would be nice if this could be generic over some protocol. For instance if the connected device supports I2C and SPI to just say which protocol needs to be used + register map and out comes a low level driver. I don't expect this to be the final driver because usually to do anything more meaningful it needs a sequence of commands beyond the context of the memory map alone. If this is not possible it would already be helpful to have generated some stub code which needs to be finished manually.

Note: i don't mean that the SVD format should be used to specify the register map of devices outside of the MCU. That's why i ask for a tool "like" svd2rust for devices.

https://arm-software.github.io/CMSIS_5/SVD/html/index.html

[osterwood]

@flip111 - I’ve been thinking a bit about something like this myself. I’ve used Kaitai Struct to recreate register maps from chip datasheets, and a bit of custom python to emit Contruct allowing serialization / deserialization from dicts to binary payloads. I’m interested in doing the same thing for Rust code, but haven’t tried anything yet.

One potential issue with Kaitai is that it’s designed primarily for binary format decoding, and as such doesn’t have fields for RO, WO, RW - but I believe you can add fields to the YAML input without disrupting existing tooling.

[flip111]

Looks interesting Katai Struct.

By the way not directly related, but i found this https://www.microsoft.com/en-us/microsoft-365/blog/2019/02/28/new-to-microsoft-365-in-february-advance-security-and-empower-a-modern-workplace/#excel Can be handy to scan datasheet, put them into a spreadsheet, mangle the columns a bit and then process into rust types.

[sajattack]

Also SPI-related, bit-banged crate.

One of my single board computing devices has a completely broken hardware implementation.

@RandomInsano It's not really in a good state yet, but I'm working on this. https://github.com/sajattack/bitbang-hal

[japaric]

Update: we now have a not yet awesome embedded Rust (NYAER) list. If you have posted some request in this thread you should submit it to NYAER for higher visibility. And if you agree with one of the requests here but didn't submit it yourself you should consider submitting it to NYAER too!

[kevswims]

I would really like the ability to run cargo test tests on the embedded device and have it report results back over the debugger. This could really help with certain driver testing in particular.

[therealprof]

@kevswims Please check out the latest probe-rs developments and more specifically these two recent blog posts: https://ferrous-systems.com/blog/probe-run/ and https://ferrous-systems.com/blog/21st-century-embedded-tooling/

Please feel free to add missing tooling to https://github.com/rust-embedded/not-yet-awesome-embedded-rust and join us https://matrix.to/#/#rust-embedded:matrix.org for ongoing live discussion.

As discussed in the meeting we're closing this old issue now.