Serializable Lambda expression

[hyunsik]

Marshalling/unmarshalling rust lambda expression like Python pickle is possible? If not now, technically it can be possible later?

If this feature is supported in Rust, Rust would be very strong for distributed frameworks too.

[shepmaster]

It appears that Python doesn't support pickling a lambda without the help of an external library. Since the Rust standard library doesn't have serialization built-in, this may be a feature request for one of the serialization libraries.

[hyunsik]

Besides, Java also seems to have some similar feature.
https://docs.oracle.com/javase/8/docs/api/java/lang/invoke/SerializedLambda.html

Yes, you are right. In the case where all necessary external libraries are already deployed in all nodes that execute the serialized lambda expression, it would be available.

As far as I know, Python and Java internally uses byte codes over some kinds of virtual machine, and they appear to achieve this feature.

I'm newbie for Rust, and I still don't know the internal of Rust. But, Rust is compilation. Probably, it may be hard to implement this feature. Is it technically possible? If possible, I'd like to investigate it more.

[Diggsey]

@hyunsik I'm fairly sure neither python nor java send the bytecode for the lambda over the network: in java's case it requires the anonymous class to be findable on the class-path when it is deserialised. The deserialiser just creates a new instance of the lambda class and then deserialises any captured values into that instance. I'm sure python will do something similar.

All of this is doable in rust, with on exception: there's no good way to identify and then recreate a particular lambda uniquely. Java for example, gives even anonymous classes a unique, fully qualified name such as "package.class$1", and this is what actually gets sent over the network. In rust, nothing like this is built-in AFAIK, so there'd be no way to find and recreate the lambda on the other end, as function pointers aren't necessarily going to be consistent across runs, or across different machines (when using position-independent code). You could get around that with platform-specific hacks to find the relative offset of a lambda from its module start, but it's still going to be incredibly brittle, with even the slightest change to the application breaking it entirely.

The best option would be using reflection to generate a unique "path" into the program to the lambda, but that would require much more powerful reflection capabilities than rust currently has.

[hyunsik]

@Diggsey Thank you for your detailed comments. I exactly understood the limitation and the current status.

[steveklabnik]

This issue would be a good candidate for the RFCs repo: https://github.com/rust-lang/rfcs/

Adding something like this will require a significant amount of design work. Please open something there if you want to pursue this further! :)

[hyunsik]

Thank you! I'll do that.