Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Yamux and TLS and fix multiaddr (WIP) #273

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
21273fd
First attempt at yamux implementation
ianopolous Feb 10, 2023
7ac831c
Fix deadlock in yamux window update
ianopolous Feb 10, 2023
4d51150
Fix condition to send yamux window updates
ianopolous Feb 10, 2023
8126c23
Remove debug
ianopolous Feb 10, 2023
2528bb2
Add yamux test. Fix new stream bug in yamux
ianopolous Feb 12, 2023
ec46f8c
remove guava - part 1
kevodwyer Feb 10, 2023
5f363bb
remove log4j
kevodwyer Feb 10, 2023
c0c8ad7
remove apache commons-codec
kevodwyer Feb 10, 2023
28610f3
more cleanup
kevodwyer Feb 10, 2023
7842239
replace TestLogAppender
kevodwyer Feb 13, 2023
2260bcc
use multibase from jitpack
kevodwyer Feb 13, 2023
30ede81
uncomment tests
kevodwyer Feb 13, 2023
9685293
Fix lint errors
ianopolous Feb 13, 2023
633bb5b
Remove dependency on consensys server
ianopolous Feb 13, 2023
74d17ac
Only import what we need from netty
ianopolous Feb 13, 2023
85f9ac6
Fix linter
ianopolous Feb 14, 2023
ee7b562
Fix linter
ianopolous Feb 14, 2023
7b3fb39
Randomise ports in test
ianopolous Feb 14, 2023
b72d9c0
Add key type tests
ianopolous Feb 14, 2023
e05b0e9
Remove unused bouncy castle dep
ianopolous Feb 14, 2023
e767a00
Update builds to java 11
ianopolous Feb 14, 2023
4247928
Add jitpack config to use jdk 11
ianopolous Feb 14, 2023
b33a272
Support quic-v1 and wss multiaddr
ianopolous Feb 20, 2023
d7ad326
Initial attempt at libp2p-tls implementation
ianopolous Feb 22, 2023
8a723ea
Update to Java 17 minimum, which Ed25519 asn1 needs
ianopolous Feb 22, 2023
89de5ee
Improve tls cert test
ianopolous Feb 22, 2023
d3c1321
Linting
ianopolous Feb 22, 2023
fe59132
More linting
ianopolous Feb 22, 2023
cc780b4
Lint tests
ianopolous Feb 22, 2023
a668590
First successful libp2p-tls connections!! (Java-Java)
ianopolous Feb 23, 2023
e45eb49
Add TLS tests
ianopolous Feb 23, 2023
5b63435
Linting
ianopolous Feb 23, 2023
c3eba45
Use jdk17 in jitpack
ianopolous Feb 23, 2023
15e5d90
Implement early muxer negiation in TLS using ALPN
ianopolous Feb 24, 2023
903ee57
Linting
ianopolous Feb 24, 2023
48a182b
Support ECDSA certificate keys in TLS as well as Ed25519
ianopolous Feb 24, 2023
640dfde
Remove UShortLengthCodec in tls.
ianopolous Feb 24, 2023
752f314
Hooray first successful TLS handshake with Kubo!
ianopolous Feb 25, 2023
e6d47a7
Push muxer channel initializer rather than directly initialise channe…
ianopolous Feb 27, 2023
03fc260
Use context allocator in TLS
ianopolous Feb 27, 2023
addcd10
Ping over TLS working to kubo!!
ianopolous Mar 3, 2023
d9bc5f2
Reenable early muxer negotiation in TLS (It works!)
ianopolous Mar 3, 2023
2311d00
Change test to use tls and yamux
ianopolous Mar 3, 2023
f34dbde
Support webtransport multiaddrs
ianopolous Mar 20, 2023
b7aca17
Implement dns multiaddr
ianopolous Apr 4, 2023
bbe648a
Expose EventLoop on Stream
ianopolous Apr 5, 2023
d3c528c
Expose EventLoop on Kotlin Stream
ianopolous Apr 5, 2023
4dd4d50
Fix yamux bug opening reverse stream on existing connection!
ianopolous Apr 18, 2023
bd1a478
linting
ianopolous Apr 24, 2023
4942c94
fix rebased build file
ianopolous May 10, 2023
e3ee729
fix rebased build file
ianopolous May 10, 2023
45eef14
Move sources to new structure
ianopolous May 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
- uses: actions/setup-java@v3
with:
distribution: temurin
java-version: 11
java-version: 17

- name: Setup Gradle
uses: gradle/gradle-build-action@v2
Expand Down
166 changes: 96 additions & 70 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -36,27 +36,40 @@ allprojects {
apply(plugin = "io.spring.dependency-management")
apply(from = "$rootDir/versions.gradle")


repositories {
mavenCentral()
maven("https://artifacts.consensys.net/public/maven/maven/")
maven("https://jitpack.io")
}

sourceSets.create("jmh") {
compileClasspath += sourceSets["main"].runtimeClasspath
compileClasspath += sourceSets["testFixtures"].runtimeClasspath
runtimeClasspath += sourceSets["main"].runtimeClasspath
runtimeClasspath += sourceSets["testFixtures"].runtimeClasspath
}

dependencies {
api("io.netty:netty-buffer:4.1.88.Final")
api("io.netty:netty-codec-http2:4.1.88.Final")
api("io.netty:netty-transport:4.1.88.Final")
api("io.netty:netty-transport-classes-epoll:4.1.88.Final")
api("com.google.protobuf:protobuf-java:3.21.9")

implementation("com.github.peergos:noise-java:22.1.0")

implementation(kotlin("stdlib-jdk8"))
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core")
implementation("org.bouncycastle:bcprov-jdk15on:1.70")
implementation("org.bouncycastle:bcpkix-jdk15on:1.70")

implementation("com.google.guava:guava")
implementation("org.apache.logging.log4j:log4j-api")
implementation("com.github.multiformats:java-multibase:v1.1.1")

testFixturesImplementation("org.apache.logging.log4j:log4j-api")
testFixturesImplementation("com.google.guava:guava")
testImplementation("org.junit.jupiter:junit-jupiter-api:5.9.1")
testImplementation("org.junit.jupiter:junit-jupiter-params:5.9.1")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.9.1")

testImplementation("org.junit.jupiter:junit-jupiter")
testImplementation("org.junit.jupiter:junit-jupiter-params")
testImplementation("io.mockk:mockk")
testImplementation("org.assertj:assertj-core")
testImplementation("org.apache.logging.log4j:log4j-core")
testImplementation("io.mockk:mockk:1.12.2")
testImplementation("org.assertj:assertj-core:3.23.1")
}

java {
Expand All @@ -72,88 +85,101 @@ allprojects {
freeCompilerArgs = listOf("-Xjvm-default=all")
}
}

tasks.withType<Copy> {
duplicatesStrategy = DuplicatesStrategy.INCLUDE
}

// Parallel build execution
tasks.test {
description = "Runs the unit tests."
java {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}

useJUnitPlatform {
excludeTags("interop")
tasks.withType<KotlinCompile> {
kotlinOptions.jvmTarget = "17"
kotlinOptions {
freeCompilerArgs = listOf("-Xjvm-default=all")
}

testLogging {
events("FAILED")
exceptionFormat = TestExceptionFormat.FULL
showCauses = true
showExceptions = true
showStackTraces = true
}
// Parallel build execution
tasks.test {
description = "Runs the unit tests."

useJUnitPlatform {
excludeTags("interop")
}

// disabling the parallel test runs for the time being due to port collisions
// If GRADLE_MAX_TEST_FORKS is not set, use half the available processors
testLogging {
events("FAILED")
exceptionFormat = TestExceptionFormat.FULL
showCauses = true
showExceptions = true
showStackTraces = true
}

// disabling the parallel test runs for the time being due to port collisions
// If GRADLE_MAX_TEST_FORKS is not set, use half the available processors
// maxParallelForks = (System.getenv("GRADLE_MAX_TEST_FORKS")?.toInt() ?:
// Runtime.getRuntime().availableProcessors().div(2))
}
}

kotlinter {
disabledRules = arrayOf("no-wildcard-imports", "enum-entry-name-case")
}
kotlinter {
disabledRules = arrayOf("no-wildcard-imports", "enum-entry-name-case")
}

val sourcesJar by tasks.registering(Jar::class) {
archiveClassifier.set("sources")
from(sourceSets.main.get().allSource)
}
val sourcesJar by tasks.registering(Jar::class) {
archiveClassifier.set("sources")
from(sourceSets.main.get().allSource)
}

tasks.dokkaHtml.configure {
outputDirectory.set(buildDir.resolve("dokka"))
dokkaSourceSets {
configureEach {
jdkVersion.set(11)
reportUndocumented.set(false)
externalDocumentationLink {
url.set(URL("https://netty.io/4.1/api/"))
tasks.dokkaHtml.configure {
outputDirectory.set(buildDir.resolve("dokka"))
dokkaSourceSets {
configureEach {
jdkVersion.set(17)
reportUndocumented.set(false)
externalDocumentationLink {
url.set(URL("https://netty.io/4.1/api/"))
}
}
}
}
}

val dokkaJar: TaskProvider<Jar> by tasks.registering(Jar::class) {
group = JavaBasePlugin.DOCUMENTATION_GROUP
val dokkaJavadocTask = tasks.getByName("dokkaJavadoc")
dependsOn(dokkaJavadocTask)
archiveClassifier.set("javadoc")
from(dokkaJavadocTask.outputs)
}
val dokkaJar: TaskProvider<Jar> by tasks.registering(Jar::class) {
group = JavaBasePlugin.DOCUMENTATION_GROUP
val dokkaJavadocTask = tasks.getByName("dokkaJavadoc")
dependsOn(dokkaJavadocTask)
archiveClassifier.set("javadoc")
from(dokkaJavadocTask.outputs)
}

publishing {
repositories {
maven {
name = "cloudsmith"
url = uri("https://api-g.cloudsmith.io/maven/libp2p/jvm-libp2p")
credentials {
username = findProperty("cloudsmithUser") as String?
password = findProperty("cloudsmithApiKey") as String?
publishing {
repositories {
maven {
name = "cloudsmith"
url = uri("https://api-g.cloudsmith.io/maven/libp2p/jvm-libp2p")
credentials {
username = findProperty("cloudsmithUser") as String?
password = findProperty("cloudsmithApiKey") as String?
}
}
}
}
if (hasProperty("mavenArtifactId")) {
publications {
register("mavenJava", MavenPublication::class) {
from(components["java"])
artifact(sourcesJar.get())
artifact(dokkaJar.get())
groupId = "io.libp2p"
artifactId = project.property("mavenArtifactId") as String
if (hasProperty("mavenArtifactId")) {
publications {
register("mavenJava", MavenPublication::class) {
from(components["java"])
artifact(sourcesJar.get())
artifact(dokkaJar.get())
groupId = "io.libp2p"
artifactId = project.property("mavenArtifactId") as String
}
}
}
}
}

detekt {
config = files("$rootDir/detekt/config.yml")
buildUponDefaultConfig = true
detekt {
config = files("$rootDir/detekt/config.yml")
buildUponDefaultConfig = true
}
}
}
2 changes: 2 additions & 0 deletions jitpack.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
jdk:
- openjdk17
11 changes: 5 additions & 6 deletions libp2p/src/main/java/io/libp2p/core/dsl/HostBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,15 @@
import io.libp2p.core.Host;
import io.libp2p.core.crypto.PrivKey;
import io.libp2p.core.multistream.ProtocolBinding;
import io.libp2p.core.mux.StreamMuxerProtocol;
import io.libp2p.core.mux.*;
import io.libp2p.core.security.SecureChannel;
import io.libp2p.core.transport.Transport;
import io.libp2p.transport.ConnectionUpgrader;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;
import java.util.function.Supplier;
import java.util.function.*;

public class HostBuilder {
public HostBuilder() { this(DefaultMode.Standard); }
Expand Down Expand Up @@ -41,7 +40,7 @@ public final HostBuilder transport(

@SafeVarargs
public final HostBuilder secureChannel(
Function<PrivKey, SecureChannel>... secureChannels) {
BiFunction<PrivKey, List<String>, SecureChannel>... secureChannels) {
secureChannels_.addAll(Arrays.asList(secureChannels));
return this;
}
Expand Down Expand Up @@ -76,7 +75,7 @@ public Host build() {
b.getTransports().add(t::apply)
);
secureChannels_.forEach(sc ->
b.getSecureChannels().add(sc::apply)
b.getSecureChannels().add((k, m) -> sc.apply(k, (List<String>)m))
);
muxers_.forEach(m ->
b.getMuxers().add(m.get())
Expand All @@ -91,7 +90,7 @@ public Host build() {

private DefaultMode defaultMode_;
private List<Function<ConnectionUpgrader, Transport>> transports_ = new ArrayList<>();
private List<Function<PrivKey, SecureChannel>> secureChannels_ = new ArrayList<>();
private List<BiFunction<PrivKey, List<String>, SecureChannel>> secureChannels_ = new ArrayList<>();
private List<Supplier<StreamMuxerProtocol>> muxers_ = new ArrayList<>();
private List<ProtocolBinding<?>> protocols_ = new ArrayList<>();
private List<String> listenAddresses_ = new ArrayList<>();
Expand Down
Loading
Loading