Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embed Hubble #10238

Merged
merged 7 commits into from Mar 4, 2020
Merged

Embed Hubble #10238

merged 7 commits into from Mar 4, 2020

Conversation

michi-covalent
Copy link
Contributor

@michi-covalent michi-covalent commented Feb 19, 2020

7 commits:

  1. listener refactoring, there is a separate PR here. Rebased
  2. vendor cilium/hubble.
  3. expose ipcache.GetK8sMetadata method.
  4. add service.GetDeepCopyServiceByAddr.
  5. add a monitor listener to send events to hubble.
  6. implement getters to access local cilium state.
  7. add daemon config options to run hubble.

This change is Reviewable

@michi-covalent michi-covalent requested a review from a team February 19, 2020 02:08
@michi-covalent michi-covalent requested review from a team as code owners February 19, 2020 02:08
@michi-covalent michi-covalent requested a review from a team February 19, 2020 02:08
@michi-covalent michi-covalent requested a review from a team as a code owner February 19, 2020 02:08
@maintainer-s-little-helper
Copy link

Release note label not set, please set the appropriate release note.

pkg/monitor/agent/agent.go Show resolved Hide resolved
pkg/ipcache/ipcache.go Show resolved Hide resolved
pkg/service/service.go Outdated Show resolved Hide resolved
pkg/monitor/agent/agent.go Show resolved Hide resolved
@michi-covalent michi-covalent changed the title Pr/michi/hubble3 Embed Hubble Feb 19, 2020
@michi-covalent michi-covalent force-pushed the pr/michi/hubble3 branch 2 times, most recently from 70ffdfc to de770f7 Compare February 19, 2020 02:17
@coveralls
Copy link

coveralls commented Feb 19, 2020

Coverage Status

Coverage decreased (-0.004%) to 45.628% when pulling 8dffb1f on pr/michi/hubble3 into 647a43c on master.

@aanm aanm added pending-review release-note/major This PR introduces major new functionality to Cilium. labels Feb 19, 2020
daemon/daemon_main.go Outdated Show resolved Hide resolved
@aanm aanm added dont-merge/waiting-for-upstream Only merge once upstream library or kernel features have landed dont-merge/needs-rebase This PR needs to be rebased because it has merge conflicts. labels Feb 19, 2020
@aanm
Copy link
Member

aanm commented Feb 19, 2020

waiting on #9924 to be merged

Copy link
Member

@tgraf tgraf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing. Most comments are minor nits that can be sorted out pretty quickly. The main discussion point is likely how we serve the API without introducing a security concern.

pkg/service/service.go Outdated Show resolved Hide resolved
pkg/service/service.go Outdated Show resolved Hide resolved
pkg/service/service.go Outdated Show resolved Hide resolved
pkg/hubble/listener.go Show resolved Hide resolved
daemon/daemon_main.go Outdated Show resolved Hide resolved
pkg/hubble/listener_test.go Show resolved Hide resolved
daemon/daemon_main.go Outdated Show resolved Hide resolved
pkg/hubble/getters.go Outdated Show resolved Hide resolved
pkg/hubble/getters.go Outdated Show resolved Hide resolved
@michi-covalent
Copy link
Contributor Author

thanks for your feedback @tgraf , i'll rebase and then address your comments.

@michi-covalent michi-covalent force-pushed the pr/michi/hubble3 branch 2 times, most recently from 2f0b4a6 to d6e18c8 Compare February 19, 2020 19:18
@michi-covalent
Copy link
Contributor Author

test-me-please

@michi-covalent
Copy link
Contributor Author

test-me-please

@michi-covalent
Copy link
Contributor Author

test-me-please

@michi-covalent
Copy link
Contributor Author

test-me-please

@michi-covalent
Copy link
Contributor Author

test-me-please

@michi-covalent
Copy link
Contributor Author

test-me-please

1 similar comment
@michi-covalent
Copy link
Contributor Author

test-me-please

go.mod Outdated Show resolved Hide resolved
Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
Hubble needs to access this function to implement the IPGetter interface:

https://github.com/cilium/hubble/blob/04ab72591faca62a305ce0715108876167182e04/pkg/parser/getters/getters.go#L47

Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
Hubble needs to be able to look up service namespace/name by address to
implement the ServiceGetter interface:

https://github.com/cilium/hubble/blob/04ab72591faca62a305ce0715108876167182e04/pkg/parser/getters/getters.go#L52

Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
- Implement a monitor listener that sends monitor events to Hubble.
- Export RegisterNewListener and Monitor so that Daemon can register
  Hubble as a monitor listener.

Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
Implement all the getter interfaces that Hubble uses to access Cilium
state:

https://github.com/cilium/hubble/blob/04ab72591faca62a305ce0715108876167182e04/pkg/parser/getters/getters.go

Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
Add options to launch Hubble with the following configuration parameters:

- hubble-listen-addresses:
    List of IP addresses for Hubble server to listen to. Hubble is disabled
    if this list is empty. It also limits the addresses to be Unix domain
    socket for now.
- hubble-flow-buffer-size:
    Maximum number of flows in Hubble's buffer.
- hubble-event-queue-size:
    Buffer size of the channel to receive monitor events.
- hubble-metrics-server:
    Address to serve Hubble metrics on. The metrics server is disabled if
    this string is empty.
- hubble-metrics:
    List of Hubble metrics to enable.

Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
Ref #9925

Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>
@michi-covalent
Copy link
Contributor Author

test-me-please

@michi-covalent
Copy link
Contributor Author

fixed the etcd version in go.mod

@aanm aanm merged commit 8d28ab1 into master Mar 4, 2020
1.8.0 automation moved this from In progress to Merged Mar 4, 2020
@aanm aanm deleted the pr/michi/hubble3 branch March 4, 2020 20:57
@rolinh rolinh added the sig/hubble Impacts hubble server or relay label Mar 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note/major This PR introduces major new functionality to Cilium. sig/hubble Impacts hubble server or relay
Projects
No open projects
1.8.0
  
Merged
Development

Successfully merging this pull request may close these issues.

None yet

9 participants