Any document on type conversions between Rust and Elixir?

[vfsoraki]

I'm writing a NIF to expose some Rust library to Elixir world. Rustler is awesome, I can tell. I'm just beginning Rust, and I've been able to expose what I want easily in a day. Totally awesome!

I'm struggling to find what is the equivalent types between the two. Some types may be obvious, like integer to i32 or some other integer types.

Specifically, I can't find how to decode bitstrings (non-UTF-8 strings). A Vec<char> or Vec<u8>, or maybe both depending on what I expect from Elixir? There is no clearance, as I take it (at least from my perspective, maybe I'm wrong).

Also, how to decode structs? Maps?

Also, I want to return some opaque type from Rust, like a resource that only my library knows what's inside. Can it be done? Or should I use some kind of map/struct/record?

I also saw serde being mentioned for this kind of encoding/decoding job. Is it a necessity? Or can I do it without it?

I may sound like a noob. That's totally correct!

[Qqwy]

Hi there!
There are some answers that can be found in the documentation, but it does seem to be a little 'all over the place'.

To answer some of your questions:

BEAM -> Rust

• Elixir/Erlang terms are given to the NIF as an opaque Term type, (which cannot outlive this call to the NIF, so if the NIF want to keep it for later, it needs to be copied elsewhere or converted).
• Depending on the signature of the function annotated with #[rustler::nif], Rustler will attempt to convert these terms for you.
  • If this conversion fails, an ArgumentError will be raised at the Elixir/Erlang side.
  • Conversion BEAM -> Rust is done using the Decoder trait implementations. You can see a list of supported Rust result types here. Most are obvious, I will comment on a few below.

There is however one other scenario: What if you want to be able to handle multiple kinds of terms? Or what if you want to write your own other kind of decoder for your custom Rust datastructure?
In that case, you can use Term.get_type() and/or the Term.is_*-functions to check what kind of term it is, and write your logic based on that.

Rust -> BEAM

This works similarly as above, but here we uset the Encoder trait implementations.
Here, no failure can happen, because Rust prevents you from using it with an unsupported type already at compile-time.

Passing arbitrary Rust terms to the BEAM

To be able to work with types not directly supported by Elixir/Erlang (one of the common reasons to write a NIF), we can return a ResourceArc.
In Elixir/Erlang, this will just look like a Reference (just like one created with make_ref/0).
References in general are an 'opaque handle'. When you obtain a references from a NIF, you can do only two things with it:

• Pass it on to another compatble NIF later.
• Forget about it, in which case it will be garbage collected at some point.

Converting your Rust structure to an Elixir structure is definitely possible, but only useful if you want the structure to be edited on the Elixir side.

Dealing with bitstrings/binaries in Rust

There is extra supports for binaries given by Erlang's NIF wrappers, and Rustler exposes these through the Binary type and related functionality.
Do note that IIRC only binaries whose size is a multiple of 8 (expressable in bytes) are supported by the NIF API.

On the Rust side, as you can already see from the documentation, binaries are usually an &[u8] (for reading) or a Vec<u8> (for modification).

Storing arbitrary Elixir terms in Rust

Now this is where it gets tricky: We need to copy or convert aTerm before the NIF is over because we have no guarantees that the terms are not moved or removed by the garbage collector afterwards. (Rust prevents us from storing Terms directly with its lifetime system).
However, not all Elixir/Erlang terms are representable in Rust!
Or, to be more precise: the Erlang NIF code does expose conversion functionality for the following (and so Rustler can also not support it):

• Ports
• References
• Functions
• Integers which are too big to fit in an i64
• Erlang exceptions

However, there is one trick: While we cannot convert the terms to Rust-specific terms, we can create an 'owned environment' and copy the terms (without knowing what exactly they are) to there.

You can see a simple implementation I made for this here. There probably is room for improvement.

About serde

serde is a library which allows Rust datastructures to be (de)serialized easily from elsewhere, with little to no boilerplate code.
serde_rustler is a wrapper that tries to do this for the Erlang NIF datatypes.
serde_rustler is however not very mature, and has not been updated to the latest version of rustler and as such does not currently run on the newest OTP versions.

So at least currently, it's better to not rely on it, and write the Encoder and Decoder trait implementations for your Rust structs yourself.
Also, see NifStruct which already works good enough for many common situations.

How to decode maps and structs?

These become a HashMap<Term, Term> in Rust, and you can specify the constraints on the key and value types even further. So if we have an Elixir struct, it might be a HashMap<Atom, Term>, for instance.
(Atom is one of the other types Rustler implements, as dealing with atoms is something the Erlang NIF interface also exposes).

Specifically for structs, there also is [NifStruct]https://docs.rs/rustler/0.22.0/rustler/derive.NifStruct.html).

[hansihe]

We actually have started working on some better docs at https://rustler-web.onrender.com/docs. Specifically there is a very early WIP type mapping cheat sheet at https://rustler-web.onrender.com/docs/cheat-sheet.

@Qqwy Really nice summary, maybe it would fit into our docs site as an article which goes though type conversions? You are free to open a PR with what you wrote here if you want to.

[praveenperera]

A question about NifStruct for the example:

#[derive(Debug, NifStruct)]
#[module = "AddStruct"]
struct AddStruct {
   lhs: i32,
   rhs: i32,
}

Is the AddStruct module supposed to be automatically created in the Elixir side, or is that left up to the user?

[evnu]

Is the AddStruct module supposed to be automatically created in the Elixir side, or is that left up to the user?

That is left up to the user.

[praveenperera]

Is the AddStruct module supposed to be automatically created in the Elixir side, or is that left up to the user?

That is left up to the user.

Thanks I figured in the end. But I was a little confused by this wording in the docs

This would be translated by Rustler into:

defmodule AddStruct do
  defstruct lhs: 0, rhs: 0
end

https://docs.rs/rustler/0.23.0/rustler/derive.NifStruct.html

I can make a PR for that little line to make it a bit clearer.