Add peer-to-peer support into Dendrite via libp2p and fetch #880
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, all events would come down redacted because the hash checks would fail. They would fail because sjson.DeleteBytes didn't remove keys not used for hashing. This didn't work because of a build tag which included a file which no-oped the index returned. See tidwall/gjson#157 When it's resolved, let's go back to mainline.
Caused when sending events over federation.
Partly so I don't forget myself!
This is important or else the normal build of dendrite will fail because the p2p libraries depend on syscall/js which doesn't work on normal builds. Also, clean up main.go to read a bit better.
We actually need this for P2P because otherwise we can *segfault* with things like: "runtime: unexpected return pc for runtime.handleEvent" where the event is a `syscall/js` event, caused by spamming sql.js caused by "Checking event signatures for 14 events of room state" which hammers the key DB repeatedly in quick succession. Using a cache fixes this, though the underlying cause is probably a bug in the version of Go I'm on (1.13.7)
Defaults to false, which is why this is a breaking change. We need this flag because WASM builds cannot do opentracing.
The general idea here is to have the wasm build have a `NewXXXDatabase` that doesn't import any postgres package and hence we never import `lib/pq`, which doesn't work under WASM (undefined `userCurrent`).
Instead, the dendrite-sw.js must mock out `process.pid` and
`fs.stat` - which must invoke the callback with an error (e.g `EINVAL`)
in order for it to work:
```
global.process = {
pid: 1,
};
global.fs.stat = function(path, cb) {
cb({
code: "EINVAL",
});
}
```
kegsay
changed the title
neilalexander
approved these changes
Mar 6, 2020
|
The failing tests are SQLite only - which is a known issue I'm going to look into today! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds in support for running peer-to-peer Dendrite instances.
Overview
$ GOOS=js GOARCH=wasm go build -o dendrite.wasm ./cmd/dendritejsKeyDB. This is required for p2p (though all implementations benefit) in order to keep federation traffic below the threshold which would cause us to be blacklisted (denied) on the relay server.p2p.mdwhich contains instructions on how to set up a complete peer (including a Riot instance).Breaking changes
Previously, tracing defaulted to on. Now it is off by default. Set
tracing.enabled = truein the config YAML to re-enable it.Notes
storage.goentry point for all microservices. These files only contain a singleNewWhateverDatabasefunction, so the code duplication is minor. For WebAssembly, this means runningstorage_wasm.go. This is required in order to remove a hard dependency onlib/pqwhich does not compile under WASM. Around 700 lines of this diff is doing this refactoring.pq.StringArrayto do this serialisation, which as the above point makes, we cannot do anymore.