thirdparty libs strategy and support

[brenzi]

I'm trying to port rust-base58 library. I see that you've ported quite a few libs in third_party but I don't see how these will be supported and updated in the long run. Wouldn't it be better to fork these libs separately and reference them by git path in Cargo.toml?

Is there a recipie about what to do to port a lib?

[dingelish]

Good question!

For the recipe, I have a blog article (but written in Chinese) about the porting process. Basically, if you use cargo instead of xargo:
(1) Check if the current crate supports #![no_std]. If yes, use it directly without any change.
(2) If not, recursively port it's dependencies and then
(3) Make the current crate a #![no_std] crate and add the dependency of sgx_tstd as follows:

[dependencies]
sgx_tstd = { path = "......" }

or

[dependencies]
sgx_tstd = "1.0.5"

And add #![no_std] at the head of lib.rs. Then use this trick to replace the default std:

#[macro_use]
extern crate sgx_tstd as std;
use std::prelude::v1::*;

The prelude import is done by default in a normal environment (with default libstd). You can see the difference here.

(4) Add the import std::prelude::v1::*; on every .rs file included in this project.
(5) Adjust the uncompatbile usages. Such as Mutex => SgxMutex. Tweakstd::fs to either std::untrusted::fs or std::sgxfs based on your file access policy (unprotected UNIX file system or trusted SGX file system).
(6) Test with cargo build

For the maintenance of popular crates, currently I don't have a clear idea. Some crates are very useful in SGX but less maintained such as bit-vec. I can maintain a fork on it. Some other crates such as ring are well maintained and change rapidly. I can try to work with the authors to support rust-sgx-sdk directly, so that we can use it without any modification. For #![no_std] crates, such as parity-wasm, we can use it directly without any modification.

Currently I think you can port it by yourself, or I can help you with rust-base58 since it's simple. I'm happy to add your ported version into the third_party directory :-)

[brenzi]

Just added a PR: #61 for base58
However, I still get errors

[dingelish]

Pushed 477219f. I think it's working :-)

[brenzi]

You're quick! Thanks a lot. I've tried it but I'm still missing something. When building my project
(https://github.com/brenzi/rust-ipfs-verifier/tree/master/sgx)
I get

error: the #[global_allocator] in sgx_tstd conflicts with this global allocator in: sgx_tstd

warning: unused import: `FromBase58`
  --> src/lib.rs:44:29
   |
44 | use rust_base58::{ToBase58, FromBase58};
   |                             ^^^^^^^^^^
   |
   = note: #[warn(unused_imports)] on by default

error: duplicate lang item in crate `sgx_tstd`: `f32_runtime`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_tstd`: `f64_runtime`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_tstd`: `panic_impl`.
  |
  = note: first defined in crate `sgx_tstd`.

error: duplicate lang item in crate `sgx_trts`: `oom`.
  |
  = note: first defined in crate `sgx_trts`.

error: aborting due to 5 previous errors

error: Could not compile `IpfsVerifierEnclave`.

To learn more, run the command again with --verbose.
Makefile:42: recipe for target 'libenclave.a' failed
make[1]: *** [libenclave.a] Error 101
make[1]: Leaving directory '/home/abrenzikofer/rust-ipfs-verifier/sgx/enclave'
Makefile:162: recipe for target 'enclave' failed
make: *** [enclave] Error 2

what am I missing?

[elichai]

@brenzi It sounds like you're using part of the sgx libraries from crates.io and part from github.

If you're using @dingelish's third_party libraries you must use the sgx_* libs from the github too, otherwise the third_party libs will take sgx_* from github and you'll take from crates.io and they will conflict

[brenzi]

@elichai Thanks, that was it!

[brenzi]

but to come back on the topic: wouldn't it be more sustainable to fork all these libraries individualy and submit PR's (that might never be merged...), but it would be easier to rebase on top of upstream improvements.
The question also touches the issue how to package a project that is based on rust-sgx-sdk. As it is now, the easiest might be to have the sdk as a git submodule in my project. So I can have all these relative paths.....

[brenzi]

As a follow-up: I've filed an issue with rust-lang:
rust-lang/rust#60561 (comment)

Please go vote for this issue, if you're suffering from no_std trouble as well.