This repository has been archived by the owner. It is now read-only.

Integration with other languages #14

Open
aturon opened this Issue Jan 31, 2017 · 15 comments

Comments

Projects
None yet
10 participants
@aturon
Member

aturon commented Jan 31, 2017

Overview

Other languages here includes "low-level" cases like C/C++, and "high-level" cases like JavaScript, Ruby, Python, Java and C#. Rust adoption often depends on being able to start using it incrementally, and language integration is often a key to doing so -- an intuition substantiated by data from the survey and commercial outreach.

Rust's core support for interfacing with C is fairly strong, but wrapping a C library still involves tedious work mirroring declarations and writing C shims or other glue code. Moreover, many projects that are ripe for Rust integration are currently using C++, and interfacing with those effectively requires maintaining an alternative C wrapper for the C++ APIs. This is a problem both for Rust code that wants to employ existing libraries and for those who want to integrate Rust into existing C/C++ codebases.

For interfacing with "high-level" languages, there is the additional barrier of working with a runtime system, which often involves integration with a garbage collector and object system. There are ongoing projects on these fronts, but it's early days and there are still a lot of open questions.

Some potential avenues of exploration include:

  • Continuing work on bindgen, with focus on seamless C and eventually C++ support. This may involve some FFI-related language extensions (like richer repr).
  • Other routes for C/C++ integration.
  • Continued expansion of existing projects like Helix and Neon, which may require some language enhancements.
  • Continued work on GC integration hooks
  • Investigation of object system integrations, including DOM, COM and GObject.

RFCs

  • repr(transparent), for allowing newtypes to appear as primitive data for ABI purposes.
    • Status: undergoing final stages of specification
  • Extern types, for talking about totally opaque types
    • Status: early discussion
@retep998

This comment has been minimized.

Show comment
Hide comment
@retep998

retep998 Jan 31, 2017

Member

While having more automatic binding to C would be nice, it would be much more useful if we could bind to C at all. The lack of repr(align(N)) and repr(packed(N)) along with unions still being unstable really hamstrings the ability of developers to represent many types. Language support for bitfields could also be useful as well. Also, Rust still needs a calling convention for C++ methods on Windows, because it has a slightly different ABI than C/C++ functions (rust-lang/rfcs#1342). repr(transparent) would improve the ability of libraries to provide safe wrappers around C/C++ APIs. Don't forget COM for object system integrations.

Member

retep998 commented Jan 31, 2017

While having more automatic binding to C would be nice, it would be much more useful if we could bind to C at all. The lack of repr(align(N)) and repr(packed(N)) along with unions still being unstable really hamstrings the ability of developers to represent many types. Language support for bitfields could also be useful as well. Also, Rust still needs a calling convention for C++ methods on Windows, because it has a slightly different ABI than C/C++ functions (rust-lang/rfcs#1342). repr(transparent) would improve the ability of libraries to provide safe wrappers around C/C++ APIs. Don't forget COM for object system integrations.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Jan 31, 2017

Member

@retep998 Note, a plan for union stabilization is here.

Member

aturon commented Jan 31, 2017

@retep998 Note, a plan for union stabilization is here.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment

Ericson2314 commented Feb 1, 2017

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Feb 1, 2017

Member

@Ericson2314 Added, thanks!

I also added repr(transparent).

Member

aturon commented Feb 1, 2017

@Ericson2314 Added, thanks!

I also added repr(transparent).

@Ericson2314

This comment has been minimized.

Show comment
Hide comment

Thanks!

@patlecat

This comment has been minimized.

Show comment
Hide comment
@patlecat

patlecat Feb 9, 2017

I understand that to offer a C API is still necessary to reach many other languages who have not yet moved to C++. But it would be a wasted chance to not create a symbiosis between C++ and Rust as 1st class partners. Rust needs a proper C++ interface lib to gain momentum for the tool builders and millions of application developers out there who would like to just try Rust i their existing C++ project for a small task. This interface has to be comfortable, complete and rock solid and always up to date.

patlecat commented Feb 9, 2017

I understand that to offer a C API is still necessary to reach many other languages who have not yet moved to C++. But it would be a wasted chance to not create a symbiosis between C++ and Rust as 1st class partners. Rust needs a proper C++ interface lib to gain momentum for the tool builders and millions of application developers out there who would like to just try Rust i their existing C++ project for a small task. This interface has to be comfortable, complete and rock solid and always up to date.

@Kimundi

This comment has been minimized.

Show comment
Hide comment
@Kimundi

Kimundi Mar 2, 2017

Member

Another route for C/C++ integration that I have seen is to use Rusts macro system to allow somewhat seamless context switching between code in different languages. Notable example: https://github.com/google/rustcxx

A possible avenue for developing that approach would be to offer some specific language syntax integration to make it less verbose than in that crate, and usable with arbitrary languages.

Member

Kimundi commented Mar 2, 2017

Another route for C/C++ integration that I have seen is to use Rusts macro system to allow somewhat seamless context switching between code in different languages. Notable example: https://github.com/google/rustcxx

A possible avenue for developing that approach would be to offer some specific language syntax integration to make it less verbose than in that crate, and usable with arbitrary languages.

@Kimundi

This comment has been minimized.

Show comment
Hide comment
@Kimundi

Kimundi Mar 2, 2017

Member

To elaborate on my previous comment, I think there are two main ways to offer Rust integration with a different language:

  1. Offering a Rust API for an API in a different language.
  2. Offering a way to call API of a different language.

Something like bindgen would fall under 1, while something like rustcxx would fall under 2.

Approach 1 would be the more ergonomic option from a user-of-the-API point of view, as you only need to know Rust to call functionality provided by a library in a different language. However, depending on language it might not be possible to generate the necessary Rust bindings automatically, requiring a lot of effort for the initial API translation and for maintaining the binding code over time. Examples here are dynamic languages where the interface is defined by runtime code, and C/C++, where the interface depends on things like which header files are included, and how much of it is actually documented as a public interface.

Approach 2 on the other hand would probably require to know both Rust and the language you are calling into, but be generally easier to implement, as there would only need to be support for the "context switching" into the other language, and not for every library written in it you want to call.

Approach 2 would also probably be a good basis to implement Approach-1 Rust bindings themself, as you would just wrap the language switching behind an API.

It seems to me that a lot of the "we use language X, and want to partially migrate to Rust" scenarios would be well served by approach 2, as you would be able to gradually move the API and dependency barrier between the two code bases instead of needing to maintain an explicit Rust API upfront or migrating all at once.

Member

Kimundi commented Mar 2, 2017

To elaborate on my previous comment, I think there are two main ways to offer Rust integration with a different language:

  1. Offering a Rust API for an API in a different language.
  2. Offering a way to call API of a different language.

Something like bindgen would fall under 1, while something like rustcxx would fall under 2.

Approach 1 would be the more ergonomic option from a user-of-the-API point of view, as you only need to know Rust to call functionality provided by a library in a different language. However, depending on language it might not be possible to generate the necessary Rust bindings automatically, requiring a lot of effort for the initial API translation and for maintaining the binding code over time. Examples here are dynamic languages where the interface is defined by runtime code, and C/C++, where the interface depends on things like which header files are included, and how much of it is actually documented as a public interface.

Approach 2 on the other hand would probably require to know both Rust and the language you are calling into, but be generally easier to implement, as there would only need to be support for the "context switching" into the other language, and not for every library written in it you want to call.

Approach 2 would also probably be a good basis to implement Approach-1 Rust bindings themself, as you would just wrap the language switching behind an API.

It seems to me that a lot of the "we use language X, and want to partially migrate to Rust" scenarios would be well served by approach 2, as you would be able to gradually move the API and dependency barrier between the two code bases instead of needing to maintain an explicit Rust API upfront or migrating all at once.

@antonkulaga

This comment has been minimized.

Show comment
Hide comment
@antonkulaga

antonkulaga Mar 5, 2017

It would be nice to have some integration with https://github.com/scala-native/scala-native as both Rust and ScalaNative comple to LLVM

It would be nice to have some integration with https://github.com/scala-native/scala-native as both Rust and ScalaNative comple to LLVM

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Mar 6, 2017

Member

@eddyb raised https://reviews.llvm.org/rL294291 on the IRC channel; maybe he can elaborate on his thoughts here.

Member

aturon commented Mar 6, 2017

@eddyb raised https://reviews.llvm.org/rL294291 on the IRC channel; maybe he can elaborate on his thoughts here.

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Mar 6, 2017

Member

Eventually, (perhaps though not this year) clangd might offer enough C++ introspection for us to be able to dynamically query it from, say, rustc, without dealing with clang's internal (and unstable!) C++ API.

Member

eddyb commented Mar 6, 2017

Eventually, (perhaps though not this year) clangd might offer enough C++ introspection for us to be able to dynamically query it from, say, rustc, without dealing with clang's internal (and unstable!) C++ API.

@jonathandturner

This comment has been minimized.

Show comment
Hide comment
@jonathandturner

jonathandturner Mar 8, 2017

Collaborator

Apologies if this was mentioned elsewhere, but just saw it pop up on my twitter feed:

https://github.com/rust-qt/cpp_to_rust

Collaborator

jonathandturner commented Mar 8, 2017

Apologies if this was mentioned elsewhere, but just saw it pop up on my twitter feed:

https://github.com/rust-qt/cpp_to_rust

@ctaggart

This comment has been minimized.

Show comment
Hide comment
@ctaggart

ctaggart Apr 21, 2017

I've explored both MATLAB integration and GNU Octave integration. Rust is a great fit for writing extensions. I'm currently hacking bindgen for Octave and could use some help.

I've explored both MATLAB integration and GNU Octave integration. Rust is a great fit for writing extensions. I'm currently hacking bindgen for Octave and could use some help.

@fafhrd91

This comment has been minimized.

Show comment
Hide comment
@fafhrd91

fafhrd91 Jul 6, 2017

I am working on python binding - PyO3

fafhrd91 commented Jul 6, 2017

I am working on python binding - PyO3

@ctaggart

This comment has been minimized.

Show comment
Hide comment
@ctaggart

ctaggart Oct 9, 2017

I explored Rust taking advantage of TypeScript.
https://twitter.com/cmr0n/status/917424453830762497
http://blog.ctaggart.com/2017/10/creating-and-printing-typescript-ast.html

Being able to pass in a trait reference into a function requiring a supertrait would be extremely helpful here. rust-lang/rust#5665

cc @jonathandturner

ctaggart commented Oct 9, 2017

I explored Rust taking advantage of TypeScript.
https://twitter.com/cmr0n/status/917424453830762497
http://blog.ctaggart.com/2017/10/creating-and-printing-typescript-ast.html

Being able to pass in a trait reference into a function requiring a supertrait would be extremely helpful here. rust-lang/rust#5665

cc @jonathandturner

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.