Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go-libp2p-kad-dht version 2 #864

Closed
wants to merge 65 commits into from
Closed

go-libp2p-kad-dht version 2 #864

wants to merge 65 commits into from

Conversation

dennis-tra
Copy link
Contributor

This is a draft pull request that shows the changes of version 2 to the go-libp2p-kad-dht repository.

@iand
Copy link

iand commented Aug 14, 2023

This looks pretty clean. Does it build after go mod tidy?

// DefaultBootstrapPeers returns hard-coded public DHT bootstrap peers operated
// by Protocol Labs. You can configure your own set of bootstrap peers by
// overwriting the corresponding Config field.
func DefaultBootstrapPeers() []peer.AddrInfo {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we should rename this to <NETWORK_NAME>BootstrapPeers(), e.g., AminoBootstrapPeers

@iand
Copy link

iand commented Sep 8, 2023

Test in coord failing (see job)

  === RUN   TestBootstrap
      coordinator_test.go:251: saw event: *coord.EventRoutingUpdated
      coordinator_test.go:251: saw event: *coord.EventBootstrapFinished
      coordinator_test.go:270: 
          	Error Trace:	/home/runner/work/go-libp2p-kad-dht/go-libp2p-kad-dht/v2/coord/coordinator_test.go:270
          	Error:      	Received unexpected error:
          	            	node not found
          	Test:       	TestBootstrap
  --- FAIL: TestBootstrap (0.00s)

This run is from the Musa Bootstrapper branch. That test failure was fixed in #880

Simplify usage of kadtest.CtxShort
@iand iand added the v2 All issues related to the v2 rewrite label Sep 11, 2023
* Give dht and coordinator their own telemetry instances

* Fix some references to old code
* Migrate go-kademlia state machines

* Clean up naming of events

* Add Node generic parameter

* Remove unused helpers

* Remove unused fields

* Add peer addresses to peerstore

* go fmt

* Remove some more usages of NodeID fields and args

* Replace usage of key.Key256 by kadt.Key

* Use kadt.PeerID rather than peer.ID in many places

* Remove CloserNodeIDs
iand and others added 21 commits September 16, 2023 13:41
* v2: upgrade to go1.21

* Add uci config

* Use newer uci actions

* Use v2 working directory in actions

* Set go-version input in actions

* Set go-version input in actions

* Use go 1.20.8 in actions

* Use go 1.21.1 and relative working directory

* Try default working directory on job

* Remove uci.yaml which is not supported yet

* Try default working directory on job

* Try default working directory as input

* Restore uci.yaml

* Restore uci.yaml

* Use modified go-check

* Use modified go-test

* Fix go-test

* Fix go-test

* Fix go-test

* Restore libp2p 0.30.0

* go mod tidy

* Remove nil error return from DefaultConfig

* use mock clock for IPNS record generation (#894)

* Use MapDatastore for provider backend tests instead of leveldb (#896)

* revert some merge residuals

* style: minor coding clean up (#898)

* remove superfluous type conversion

* add tiny example test

* unexport type conversion helpers

* Target go language version 1.20 and add 1.20.8 to build matrix

* Target go language version 1.20 and add 1.20.8 to build matrix

* WIP

---------

Co-authored-by: Dennis Trautwein <git@dtrautwein.eu>
* Test query interactions with routing table

* v2: upgrade to go1.21

* Add uci config

* Use newer uci actions

* Pass clock to coordinator

* Use v2 working directory in actions

* Set go-version input in actions

* Set go-version input in actions

* Use go 1.20.8 in actions

* Use go 1.21.1 and relative working directory

* Try default working directory on job

* Remove uci.yaml which is not supported yet

* Try default working directory on job

* Try default working directory as input

* Restore uci.yaml

* Restore uci.yaml

* Use modified go-check

* Use modified go-test

* Fix go-test

* Fix go-test

* Fix go-test

* Update go-kademlia

* Add more tracing

* Use go-kademlia trie fix
This PR changes tracing to use the OpenTelemetry protocol which is also recommended by Jaeger.

[`go.opentelemetry.io/otel/exporters/jaeger`](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/jaeger):

> Deprecated: This module is no longer supported. OpenTelemetry dropped support for Jaeger exporter in July 2023. Jaeger officially accepts and recommends using OTLP. Use [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp) or [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc) instead.

Works with the following docker command:

```go
docker run --rm --name jaeger -p 16686:16686 -p 4317:4317 jaegertracing/all-in-one:1.49
```

Note that `jaegertracing/all-in-one` version `1.49` is **newer** than version `1.6` which we often used in the past (5 years old).
* Decouple coord package from addressing

* Go fmt

* fix: garbage collection test race condition (#904)

Moved ticker initialization outside the garbage collection goroutine. There was a race condition between advancing the mocked time in TestProvidersBackend_GarbageCollection and the initialization of the ticker that triggers the garbage collection runs. It happened that we were advancing the time without the ticker being initialized. Hence, advancing the time hasn't had any effect.

In this PR, I moved the ticker initialization outside the garbage collection goroutine. This means the ticker will be registered with the MockClock object after StartGarbageCollection returns. Calls to mockClock.Add will therefore trigger the ticker.

* Fix test flakes that wait for routing events (#905)

* Increase test iterations to trigger flake

* Add failfast

* Replace routing notification channel with a RoutingNotifier type

* Remove unused code

* Remove test count

---------

Co-authored-by: Dennis Trautwein <git@dtrautwein.eu>
* Implement GetValue

* Add failing TestGetValueOnePeer test

* Unexport methods
* Improve query capabilities

* go mod tidy

* Review feedback

* go mod tidy

* Move coord packages to internal (#933)

* Move coord and kadt packages to internal

* go mod tidy

* go fmt

* Move kadt out of internal and add RoutingTable interface
Co-authored-by: Ian Davis <18375+iand@users.noreply.github.com>
* Improve query capabilities

* go mod tidy

* Review feedback

* go mod tidy

* Move coord packages to internal (#933)

* Move coord and kadt packages to internal

* go mod tidy

* go fmt

* Move kadt out of internal and add RoutingTable interface

* Add explore state machine to expand population of routing table

* Refactor schedule into separate type

* Add generation of random peer id for a given cpl

* go mod tidy

* Add prefixmap generator

* Use constants for various query ids

* go mod tidy

* Wire explore state machine into routing behaviour

* Remove some unnecessary conversions

* PR review updates
* Expose behaviour and state machine configs

* Adjust test

* Revert adjust test
* feat: findProvidersAsync

* add: find providers tests

* add config tests
* Add metrics to routing state machines

* Simplify use of gauges with atomics
* Loggiing improvements

* Add more debug logging

* Add more debug logging

* Use logging helpers
@iand
Copy link

iand commented Oct 2, 2023

Once #954 is complete this PR will be closed and the v2-develop branch deleted.

@guillaumemichel
Copy link
Contributor

Closing as v2 has moved to Zikade

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v2 All issues related to the v2 rewrite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants