Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AuthN Support #782

Closed
srijs opened this issue Oct 20, 2015 · 17 comments

Comments

@srijs
Copy link
Contributor

@srijs srijs commented Oct 20, 2015

In our deployment of Zipkin, some internal customers are asking for authentication support, such that we can restrict the access on a per-service level. The reason for this is mostly that binary annotations may contain security-sensitive information.

What I would be interested in is securing the query service via some form of auth, and then modifying web to prompt and forward that information.

Thoughts?

@srijs srijs changed the title AuthN/Z Support AuthN Support Oct 20, 2015
@adriancole

This comment has been minimized.

Copy link
Contributor

@adriancole adriancole commented Oct 20, 2015

The zipkin-web terminates all traffic from the UI. One way to reduce surface area is to only expose zipkin-web. There's work to do in order to lock that down, and bear in mind I've not seen all things possible in finagle-land.

zipkin-web is a TwitterServer, and very similar to Finatra. Most patterns of Finatra could be applied to zipkin-web. To do authentication, you'd likely end up writing a filter or explicit code. I've found two examples: finatra-angular-example and finatra-tweet-example

The subject of authorization would be slightly more complex. There's a mix of responsibilities inside zipkin-web, presumably folks optimizing for how much data is going to the browser, but honestly, I've no idea. This mix presents itself as some javascript components calling json endpoints, and mustache template rendering doing other things. This scatters the authorization concern a bit, as if there was only a "normal" json query api, you could just secure that. For example, to authorize by service, code would likely need to wrap the underlying SpanStore object as that's the choke-point for data.

The work involved in unwinding this problem is essentially refactoring the javascript UI to only use the query server's api. Authorization code could be much simpler, even at the http abstraction. That said, there would be a caveat, which is that you need to inspect the json to ensure a call to getTraces (by id) doesn't accidentally return data the user shouldn't see. That's because it isn't likely all storage implementations will be able to support transparent authorization.

Regardless, once the UI only makes calls to a narrowly defined set of endpoints, you'd have relatively straight-forward authorization. This would be whatever you can code in finatra, or any number of established patterns supported by Spring Boot, such as OAuth (as zipkin-java is a boot app)

@dsyer

This comment has been minimized.

Copy link
Contributor

@dsyer dsyer commented Oct 21, 2015

All I can add to that is that, while securing the Java query server (and also collector now) will be very straightforward, you would probably have to write your own UI in order to deal with the user experience of being denied access (assuming authentication could be done as a separate service, which is fairly easy if you have one already doing oauth etc.).

@schrepfler

This comment has been minimized.

Copy link
Contributor

@schrepfler schrepfler commented Oct 21, 2015

Finatra being based on Finagle, I imagine it would be possible to make some sort of a Filter to perform authentication and resource protection scheme but UI work would be needed and probably some way to model the security model.

@eirslett

This comment has been minimized.

Copy link
Contributor

@eirslett eirslett commented Mar 16, 2016

Closing due to lack of activity. Zipkin is no longer based on Finagle directly, but on Finatra.
I think you're left with 2 options:

  1. put nginx/apache/something in front of zipkin, to handle authentication (cannot be fine-grained)
  2. fork zipkin, to make a version with authentication support

We're quite limited on resources, so I don't think autentication is something we would prioritise. (Feel free to reopenthis issue if any project members disagree!)

@eirslett eirslett closed this Mar 16, 2016
adriancole added a commit to openzipkin-attic/docker-zipkin that referenced this issue Sep 16, 2016
This container doubles as a skeleton for creating proxy configuration around
Zipkin like authentication, dealing with CORS with zipkin-js apps, or
terminating SSL.

It can also be adapted to test new UIs, such as https://github.com/openzipkin/zipkin-ui

Related issues which having an example NGINX config could help with:

See openzipkin/zipkin#782
See openzipkin/zipkin#1135
See openzipkin/zipkin#1171
See openzipkin/zipkin#974
@adriancole adriancole reopened this Mar 13, 2017
@adriancole adriancole assigned adriancole and unassigned adriancole Mar 13, 2017
@adriancole

This comment has been minimized.

Copy link
Contributor

@adriancole adriancole commented Mar 13, 2017

Re-opening for @devinsba who has restarted an investigation on this (#1537). I'm adding some notes to help frame thing, and thankful for the help.

The current topic is custom auth (also how any instrumentation might produce it). This will be easier for something like sleuth (as it uses the same language and library), so particularly important to consider non-jvm instrumentation, too.

To frame things, I think the idea is to understand what's sustainable and sensible to support wrt auth either directly (some auth module that covers all http), or indirectly (swap out http ui or collector for something secured). This is important as folks have different security contexts (some api key, some basic auth, some oauth). Also, afiak, the only http span reporters that support auth only do so because their underlying libraries do.

To set this up for success, I'm pinging some folks who have custom http collectors for insight on what auth means for them (wrt at least collector, but perhaps ui if willing to share)

@trustin and @anuraaga (armeria collector)
@marcingrzejszczak (some folks do custom auth in sleuth, presumably via custom servers?)
@kevinmdavis @denyska (stackdriver zipkin)
@nicmunroe riposte
@pavolloffay hawkular

then other folks who have requested auth, but not on this thread, yet (at least @jquatier)

One thing of note is that while you can add transports to the zipkin-server (like sqs, etc), you currently cannot modify the http configuration. I think this is an artificial constraint, which we can likely change. For example, zipkin-ui autoconfiguration exists independently, so it should be possible to change UI authorization without affecting the http transport. However, it should also be possible to change the http transport for reasons including auth (ex performance which I know others have done in the past).

@marcingrzejszczak

This comment has been minimized.

Copy link

@marcingrzejszczak marcingrzejszczak commented Mar 13, 2017

@marcingrzejszczak (some folks do custom auth in sleuth, presumably via custom servers?)

That's more related to securing connection between Sleuth and Zipkin. What I think would need to be done here is more related to wrapping the Tracer. In the Tracer you have the addTag method which should be used to add tags. You could write an aspect or just provide your own implementation that extends the DefaultTracer that would check some security details when somebody calls addTag.

Another approach would be to provide your custom implementation of the SpanReporter that would first e.g. remove the tags that shouldn't be there prior to sending them to Zipkin.

@devinsba

This comment has been minimized.

Copy link
Member

@devinsba devinsba commented Mar 13, 2017

So it seems there are 2 slightly different concerns here for people:

  • authentication: securing the webapp, including the span reporting endpoint
  • authorization: securing the data from specific services to only certain users

Neither of these are straightforward as many orgs will have different ways of authenticating applications to each other and authorizing users

Securing the data seems to be more involved change though from securing the endpoints. As @dsyer noted, there are UI/UX implication to hiding data from certain users and showing it to others, and there is the question of what kinds of data can be hidden. @srijs maybe you can give more clarity to which types of things (tags? logs? certain tags/logs? entire spans?) you would like to have require authorization?

It seems to me that any authorization strategy will require authentication so these two components can be designed separately from each other as long as any design for authorization lays out its assumptions about what the authentication component will provide it

@anuraaga

This comment has been minimized.

Copy link
Contributor

@anuraaga anuraaga commented Mar 13, 2017

Authentication during collection is probably pretty easy as long as the storage is only accessible from zipkin (a precondition for any authorization I think) - SSL certificates work well for providing an identity in internal RPCs and seem to be the standard in linkerd, kubernetes, etc (I use it with armeria too for RPCs, though not with zipkin which I'd rather be free as in beer ;) ). After that, it's just up to the data model to store this restriction some way - a somewhat complicated way that doesn't require changing the model would probably be some binary annotation like secure_annotations = "adriancole;key1,key2,key3" (not really binary at this point...). The collector would validate this annotation against the provided identity (SSL certificate). The web UI could then trust this annotation to authorize when returning spans.

Authenticating and authorizing requests to a web UI tends to be more complicated - if I was a Google Apps user I would expect authenticating to be Google Login Oauth and authorizing a lookup of Google groups. Other organizations might have different methods for employee authentication like LDAP.

I guess the three components needed would be

  1. What are the authorized parts of a span (specific annotations, entire span, etc)
  2. Given a span-writing request, what's it's identity (pluggable, default could be SSL certs)
  3. Given a request to the web ui, what identities can it access (pluggable, not sure of a good default)

Edit: I recalled vanilla zipkin might not support SSL - I guess that would be a prerequisite to any work on security.

@pbadenski

This comment has been minimized.

Copy link

@pbadenski pbadenski commented Oct 16, 2017

For all who just need a simple oauth2 in front of zipkin web ui we used nginx + oauth2_proxy: https://gist.github.com/pbadenski/10069aaab5d8f0a2b2f8e943c2502e0e

@ajessup

This comment has been minimized.

Copy link

@ajessup ajessup commented Oct 26, 2017

SPIFFE (https://github.com/spiffe/spiffe) could be a good fit here. If Zipkin supported the SVID X.509 certificate (or even better, the SPIFFE Workload API) then it would (a) be able to automatically extract the service name from the certificate's SAN, and (b) would be able to use the associated PK to establish an mTLS connection to the Zipkin server. The SPIFFE infrastructure (eg. https://github.com/spiffe/spire) would automatically issue PKs to each client, and certificate bundles to the server, rotating them as necessary.

@adriancole

This comment has been minimized.

Copy link
Contributor

@adriancole adriancole commented Nov 17, 2017

FYI, (and not about SPIFFE, rather related to a gitter chat)

We all have different ideas on what authn/authz needs to be, I'd welcome folks to describe what they must have in order for this to be solved. Bear in mind that this may impact the http collector endpoint, but not others (ex kafka and rabbit have their own auth mechanisms). It is important to enumerate because we might end up otherwise burning time on a solution no-one can use and no sender supports.

TL;DR; mention what you need or want, and how if at all it impacts your apps (ex do your apps need to share a token or key or password? who manages these?). Mention any workarounds you use today, if you can. Appreciated!

@adriancole

This comment has been minimized.

Copy link
Contributor

@adriancole adriancole commented Oct 26, 2018

For those who really really want basic auth and no proxy, here's a way to bolt-on BASIC auth to an existing zipkin server without modifying it.

Once you have it together, the below shows how to change password (though there's an example too) https://docs.spring.io/spring-boot/docs/2.1.0.RELEASE/reference/htmlsingle/#boot-features-security

This requires that you have maven installed. You don't need to do any custom code as it is 100% packaging.

# get normal zipkin server
$ curl -sSL https://zipkin.io/quickstart.sh | bash -s
# build the security module
$ mvn clean install
# rename the jar so it is easier
$ mv ./target/security-1.0-SNAPSHOT-module.jar security.jar
# start zipkin which now has BASIC auth on all endpoints
$ java -Dloader.path='security.jar,security.jar!/lib' -cp zipkin.jar  org.springframework.boot.loader.PropertiesLauncher --spring.security.user.name=zipkin --spring.security.user.password=pipkin

Here's the only file you need:

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>io.zipkin.custom</groupId>
  <artifactId>security</artifactId>
  <version>1.0-SNAPSHOT</version>

  <description>Example module that adds basic auth to an existing Zipkin</description>

  <properties>
     <!-- make sure this matches zipkin-server's spring boot version -->
    <spring-boot.version>2.1.1.RELEASE</spring-boot.version>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <!-- This makes sure versions are aligned properly -->
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-dependencies</artifactId>
        <version>${spring-boot.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <!-- this is the thing that adds basic auth -->
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-security</artifactId>
      <version>${spring-boot.version}</version>
      <exclusions>
        <!-- zipkin already has this -->
        <exclusion>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.springframework</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-jar-plugin</artifactId>
        <executions>
          <execution>
            <id>default-jar</id>
            <configuration>
              <archive>
                <!-- prevents huge pom file from also being added to the jar under META-INF/maven -->
                <addMavenDescriptor>false</addMavenDescriptor>
              </archive>
            </configuration>
            <goals>
              <goal>jar</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>${spring-boot.version}</version>
        <executions>
          <execution>
            <goals>
              <goal>repackage</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <layoutFactory implementation="zipkin.layout.ZipkinLayoutFactory">
            <name>custom</name>
          </layoutFactory>
          <classifier>module</classifier>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>io.zipkin.layout</groupId>
            <artifactId>zipkin-layout-factory</artifactId>
            <version>0.0.4</version>
          </dependency>
        </dependencies>
      </plugin>
    </plugins>
  </build>
</project>
@adriancole adriancole closed this Oct 26, 2018
@adriancole

This comment has been minimized.

Copy link
Contributor

@adriancole adriancole commented Oct 26, 2018

the above workaround was ported from earlier work by @devinsba .. thanks! #1537

@jbedalov

This comment has been minimized.

Copy link

@jbedalov jbedalov commented Jan 14, 2019

@adriancole, we tested the auth configurations you recommended above and they work nicely but for only the GET APIs and WebUi however there are a issues with securing the POST APIs.

  1. The zipkin server's POST APIs use undertow, so they are missed in the Spring Security configuration https://github.com/openzipkin/zipkin/blob/master/zipkin-server/src/main/java/zipkin2/server/internal/ZipkinHttpCollector.java

  2. Clients cannot inject any auth headers, they only add content type https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-zipkin/src/main/java/org/springframework/cloud/sleuth/zipkin2/sender/RestTemplateSender.java#L125.

  3. ZipkinRestTemplate is package private so it would be a bad code smell to inject auth headers there.

Just a heads up for those interested in pursuing an auth approach. For the most part, there's no way to lock down POST without some sort of private network approach. Maybe the other transports Kafka or Scribe may have a way to auth writing.

@eirslett

This comment has been minimized.

Copy link
Contributor

@eirslett eirslett commented Jan 14, 2019

You could deploy the POST servers separately from the GET servers, behind a network firewall, if that's sufficient?

@adriancole

This comment has been minimized.

Copy link
Contributor

@adriancole adriancole commented Jan 15, 2019

@jbedalov

This comment has been minimized.

Copy link

@jbedalov jbedalov commented Jan 15, 2019

You could deploy the POST servers separately from the GET servers, behind a network firewall, if that's sufficient?

@eirslett, yeah basically, that's what we will have to do. Thanks for the replies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.