High level thinking from Textile

[asutula]

Very excited to see this repo take shape! At Textile, we've done a lot of the work that will be done here, but instead of wrapping the go-ipfs API, we've wrapped the go-textile API (which wraps the go-ifps API). I think we can transfer a lot of what we've done to this repo, and ultimately build the Textile mobile SDKs on top of gomobile-ipfs.

I want to start some general conversation and mention some high-level ideas that could point to more specific issues we could create and work on.

• HTTP API vs native API - We went quite far down the path of running the HTTP API, but ultimately found it be very insecure. Any method of securing the API seemed very complicated, especially on Android. It would be interesting to see if anyone figured out something here that we couldn't figure out.
• Golang to native data serialization - In the case of using native (non-HTTP) methods, we found it necessary to serialize data for passing back and forth from go to native and back. This is because gobind supported data types are limited and changes in object interfaces are hard to deal with. For these reasons, we went with Protobuf, but maybe there are other ideas.
• Node lifecycle - The app that the IPFS node is embedded within can't be running all the time. The OS will suspend and kill the app on a regular basis. If you start the node and the OS suspends the app, it's not clear what happens to the IPFS node, but when the app is un-suspended, it seems the node isn't functioning properly. Perhaps network connections were closed, etc. For this reason, we found it necessary to explicitly control the starting and stopping of the node using app lifecycle events.
• Maximize runtime in background - We're able to take advantage of native APIs to run the host app in the background opportunistically so that the IPFS node runs as much as possible.
• Repo/project structure - As you can see, the iOS and Android native-layer code gets complex and potentially should be managed as a separate projects. We settled on a nice setup where we have separate repos for the Go, iOS, Android, and React Native libraries. The structure of the code and APIs within each librarary is similar to provide a consistent developer experience and consistent API.
• CI integration - We have the Go, iOS, Android, and React Native libraries all building in CI and publishing release artifacts to Cocopods, Maven, and NPM.
• App Store compliance - Go 12 doesn’t build the framework in a way that passes the Apple App Store review, so there is a workaround we’ve patched together on our CI server to use a working combination of Go an Gobind.

Those are some of the areas that come to mind where I think we can contribute a lot. Let me know if any of those seem particularly important or not important at all. We can break things down into smaller pieces and get to work!

[aeddi]

Hello @astula and Textile team,

Sorry for the response time but we had a lot of thought about your proposal and we took the time to look at what you were doing on Textile.

We totally agree that the goal of gomobile-ipfs is to create a package that is easily usable for mobile developers. The best in our opinion would be to instanciate an optimized core.IpfsNode that will use a more suitable default configuration and some mobile specific features/optimizations through native drivers, that gives the developer the choice to use it via Java, ObjC/Swift, Javascript (with React-Native) or Golang.

We've been thinking about a proposal and we'd like to have your feedback on it.

Init and Binding

We think that the best approach would be to have a lib that is imported, inited and lifecycle managed from native languages as you do with Textile.

The init would create a new IPFS-node exposing an HTTP API over Unix domain socket. The advantage of UDS compared to TCP/IP is that you can restrict its access to your own process.

So here is the pseudo code of this first step:

node = "github.com/ipfs/go-ipfs/core".NewNode()
unix_listener = "net".Listen("unix", unix_socket_path)

"github.com/ipfs/go-ipfs/core/corehttp".Serve(node, unix_listener)

Native languages

For Java/Swift/ObjC/JS(RN) to be able to talk with the Node, it would only be necessary to:

• Expose a Shell from Golang:

httpclient := "http".Client(unix_socket_path)
shell := "github.com/ipfs/go-ipfs-api".NewShellWithClient(httpclient)

• Write Native -> Go binding that allow mobile dev to compose a request that way:

# example: https://docs.ipfs.io/reference/api/http/#api-v0-dht-findpeer

request = shell.NewRequestBuilder("/dht/findpeer")
request.Argument("<peerID>")
request.BoolOption("verbose", true)

response = request.Exec() // returns JSON

• And even that way:

# example: https://docs.ipfs.io/reference/api/http/#api-v0-dht-findpeer

response = shell.Request("/dht/findpeer?arg=<peerID>&verbose=true") // returns JSON

Pros:

• Easy to implement for us compared to write bindings for the different languages to a complete CoreAPI and the complex types that come with it.
• Easy to use for a mobile dev that will only need to refer to the IPFS HTTP API documentation to write requests and interpret JSON responses. No need for him to bother to learn golang basics and understand the godoc.
• Easy to maintain: even if the CoreAPI changes, we won't need to update the bindings in the different languages.
• Secure thanks to UDS.

Cons:

• Less integrated with the native languages so no autocompletion in IDE, probably less practical in a program that massively uses IPFS-related objects, etc...

Golang

In case a developer would like to write go code using a mobile IPFS node, we think it would be better if they did so in a separate gomobile root package because:

• When trying to modify the lib's source code and then recompile it, it might be complicated to add custom code in a clean way.
• It is likely that the added golang code is not to be managed by the same lifecycle management system as the node.

This implies that this second root package must talk with gomobile-ipfs through the same channel as native languages. However, it would still be much better if a golang developer could use the node directly through the CoreAPI. It would be simpler for him, more powerful and its code would be almost identical on mobile and on another platform (we plan to make a Windows, macOS and Linux version of Berty so this issue interests us a lot).

To achieve this result, it would be sufficient to use this package and to plug it to the gomobile-ipfs HTTP API over UDS as follows:

httpclient := "http".Client(unix_socket_path)
coreapi := "github.com/ipfs/go-ipfs-http-api".NewApiWithClient(httpclient)

Packages and lifecycle management

Node lifecycle - The app that the IPFS node is embedded within can't be running all the time. The OS will suspend and kill the app on a regular basis. If you start the node and the OS suspends the app, it's not clear what happens to the IPFS node, but when the app is un-suspended, it seems the node isn't functioning properly. Perhaps network connections were closed, etc. For this reason, we found it necessary to explicitly control the starting and stopping of the node using app lifecycle events.

Maximize runtime in background - We're able to take advantage of native APIs to run the host app in the background opportunistically so that the IPFS node runs as much as possible.

We believe that these are two of the top priorities. We've had hard time to manage this on Berty v1 and we saw that you managed to do something very clean on Textile. It would be really great if it could be managed directly by the lib with as little effort and reflection on the part of the user as possible.

Your help would be greatly appreciated to make this works! :)

Repo/project structure - As you can see, the iOS and Android native-layer code gets complex and potentially should be managed as a separate projects. We settled on a nice setup where we have separate repos for the Go, iOS, Android, and React Native libraries. The structure of the code and APIs within each librarary is similar to provide a consistent developer experience and consistent API.

CI integration - We have the Go, iOS, Android, and React Native libraries all building in CI and publishing release artifacts to Cocopods, Maven, and NPM.

We really like the way you managed to do that! It is clean, modular and easy to use. We never had to think about these problems on our side because our initial code was not intended to be generic and reusable. Again, we would very much like your help on these points.

However, we would like to make a proposal for an alternative to multi-repo. We could have a mono-repo (except for the react-native package) with a structure as follows:

gomobile-ipfs/
  Makefile
  example/
  android/
    ...maeven package
  ios/
    ...cocoapods package
  go/
    ...gomobile-ipfs core

The main advantage would be to manage PRs from contributors that modify both Android, iOS and Go code in one place and without the risk of conflicts/incompatibilities due to the delay between merges on different repos.

App Store compliance - Go 12 doesn’t build the framework in a way that passes the Apple App Store review, so there is a workaround we’ve patched together on our CI server to use a working combination of Go an Gobind.

We didn't have the chance to publish our app on the App Store yet, so we trust you on this one. The trick is to disable DTrace, right?

BTW, we started setting up Bazel for the build of our current version of Berty, it's really powerfull when it comes to manage cross-platform application builds in different languages, its blazing fast but it can be a pain to setup and maintain. It is not a priority and we will probably be more effective in implementing it when we have more experience on the subject.

Native drivers

We think that one of the best ways to improve gomobile-ipfs would be to optimize it and extend its capabilities using native drivers.

On Berty v1 we had a connectivity driver that notified the node of connectivity changes (internet connection yes/no, wifi or cellular, bluetooth yes/no, multicast address available, etc...).

In this way, we ensured that unnecessary services were cut and requests were filtered according to the context (mainly to save resources). It was easy to do on our libp2p custom network and we are still thinking about how to integrate it into a complete IPFS node.

We also had native drivers used for BLE transports and we plan to add Apple MultipeerConnectivity and Android Nearby transports.

It took us a long time to find out how to make the Java/ObjC drivers code totally independent of the Android/IOS project (so only imported and managed by the golang package, without importing it into the main project). Another concern was to have no circular dependencies so we implemented everything using gomobile reverse binding on Android and cgo on iOS. We wanted to make it easy for a gomobile user to import the go package directly with the least setup to do to make it work.

After seeing your package and lifecycle management system, we thinking about changing that. But it's complicated to find a solution to do things properly by respecting the following rules:

• Modular and easily importable in a project that uses only libp2p without gomobile-ipfs.
• Without circular dependency / easy to build (knowing that calls are bidirectional Java driver <-> go and gomobile Java reverse bindings are very limited in term of features).
• Can be initiated and managed by the Android project even if only used by the go module.

So we need more time to reflect on this last point. This does not prevent us from starting to implement the rest when we have defined a plan together. :)

By the way, as soon as we agree on a plan, we will modify the initial PR #5 to stick to it and then merge it. All that remains is to decide whether to open-source the repo immediately or wait until lifecycle management is implemented. Would your team be willing to implement this part when the initial PR is merged @astula?

It would be great to have the opinion of the IPFS team in addition to Textile and ours. @Stebalien @parkan :)

[sanderpick]

Hi @aeddi,

I just wanted to quickly flag https://github.com/hsanjuan/ipfs-lite. Have you played with this at all? We've been considering migrating to this lighter client for mobile applications. It is quite nice to build up the stack as needed on top of libp2p, e.g., clients just need to get+put blocks, and find local peers with mdns / other future plugable libp2p modules.

As in your proposal, an HTTP/gRPC interface could be built around this lighter API. Perhaps there's a way to reuse some infrastructure for both a full and lite node (lifecycle, etc.).

The UDS path sounds very interesting. Are you able to have multiple connections open?

Thanks for looping us in!

GitHub

hsanjuan/ipfs-lite

IPFS-Lite is an embeddable, lightweight IPFS-network peer for IPLD applications - hsanjuan/ipfs-lite

[asutula]

Thank you for the thorough and thoughtful response!

• Very cool about using http over uds with that shell client. Seems like a good and secure way to quickly have access to all the IPFS api.
  • Does UDS change anything about the behavior or capabilities of the http client and/or server or is it something you can just swap out and forget about?
• We'd be happy to contribute work on the lifecycle management. I'd like to write that in a way that could be plugged into anything that needs to be started and stopped and run opportunistically in the background.
• I agree the mono-repo could be nice, sounds good to me.
• Bazel sounds slick. I'm sure we can cherry pick bits of knowledge from our CircleCI setup as needed and apply them in Bazel.
• The work you're doing on native drivers is great. Looking forward to learning more about how that works. Textile will definitely be interested in using those drivers, and helping out however we can.

[aeddi]

@sanderpick
We took a quick look at IPFS-lite a while ago but it seemed too limited for our needs so we implemented a custom libp2p network for Berty v1 (basically a heavier IPFS-lite ^^).

As in your proposal, an HTTP/gRPC interface could be built around this lighter API. Perhaps there's a way to reuse some infrastructure for both a full and lite node (lifecycle, etc.).

It could have been great to reuse exactly the same system but unfortunately IPFS-lite does not offer a Shell nor implement the CoreAPI interface, so the "bridge" defined in our proposal won't be compatible with it.

However, the developers behind IPFS-lite could be interested in porting those features to make it compatible with gomobile-ipfs bridge/lifecycle management. But I have the impression that making IPFS-lite compatible with CoreAPI would go against what the creators of the project are trying to do.

Another solution could be to implement another version of the bridge and lifecycle management system which is compatible with IPFS-lite.

The UDS path sounds very interesting. Are you able to have multiple connections open?

Do you mean having an HTTP server that listens on multiple UDS paths or having multiple HTTP clients making requests on the same UDS path? I think both are possible without any problem.

@asutula

Does UDS change anything about the behavior or capabilities of the http client and/or server or is it something you can just swap out and forget about?

At the go-ipfs level you just instantiate a connection over UDS instead of TCP/IP and any difference between these two transports are abstracted by golang net. So yes I think you can just swap them without seeing any difference.
At the OS level, unless I miss something, the differences are: more performance with UDS (even on localhost the TCP/IP protocol remains more complex, with more check, ACK, etc... than simple read/write) and UDS takes advantage of file permissions so no need to set up an authentication system.

We are thinking of allowing developers to init the HTTP API of the node over TCP/IP, so they could why not make the node accessible by other applications on their smartphone by making requests on localhost.

We'd be happy to contribute work on the lifecycle management. I'd like to write that in a way that could be plugged into anything that needs to be started and stopped and run opportunistically in the background.

Awesome! :)

I agree the mono-repo could be nice, sounds good to me.

Great, so we'll do that!

Bazel sounds slick. I'm sure we can cherry pick bits of knowledge from our CircleCI setup as needed and apply them in Bazel.

The setup we have of Bazel today gives us incredible performance in terms of build time but it's complicated to set up and maintain.

The work you're doing on native drivers is great. Looking forward to learning more about how that works. Textile will definitely be interested in using those drivers, and helping out however we can.

As soon as we can free up some time, we will ensure that our work on native drivers is open-source and documented so that it can serve the community and help contributors to further improve the project.