Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

cognito/oidc auth support #754

Merged
merged 1 commit into from Jan 14, 2019
Merged

Conversation

@M00nF1sh
Copy link
Collaborator

M00nF1sh commented Nov 28, 2018

Proposal for Auth Support

Requirement

  1. Support auth provider of OIDC and AWS Cognito
  2. Support configure auth per rule(host/path)
  3. Support load OIDC clientSecrets from kubernetes secrets

Designs

Alternative1: Use dedicated auth annotation (prefer this with 馃憤 )

We can use dedicated set of annotations to provide authentication support.

annotation description example
auth-type authentication type(required for express non-auth while specify default auth cfg at controller level) oidc|cognito|none
auth-idp-cognito Cognito specific configuration '{'UserPoolArn': 'xxx', 'UserPoolClientId': 'xxx', 'UserPoolDomain':'xxx'}'
auth-idp-oidc OIDC specific configuration '{'Issuer':'xxx', 'ClientID':'xxx', 'ClientSecret':'xxx', 'AuthorizationEndpoint':'xxx', 'TokenEndpoint':'xxx', 'UserInfoEndpoint':'xxx'}
auth-scope The set of user claims to be requested from the IdP openid
auth-session-cookie name of session cookie AWSELBAuthSessionCookie
auth-session-timeout time of session 86400

How to configure auth per rule?

The above set of annotations can be applied to both ingress / service. If it's applied to ingress, all host/path under that ingress will be impacted. If it's applied to service, only backend to such service will be impacted.
However, the above approach still have below limitations:

  • We cannot use different auth config for same service under different ingress(If we don't want to apply global auth annotation to these ingresses)
  • We cannot use different auth config for same service but different port(if we don't want to apply global auth annotation to these ingresses)
  • We cannot specify auth config for redirect/fixed-response action introduced by the use-annotation targets(if we don't want to apply global auth annotation to these ingresses)

The above limitation come from a fundamental problem that we cannot customize ingress behavior for per rule(cannot specify annotations that only apply to specific rules). This can be solved by introduce support for ingress group as describe in #688

Pros

  • Users don't need to deal with the complicated action annotation configuration
  • User can apply global auth config at ingress level
  • Common auth config like scope/session-cookie that are shared by OIDC or Cognito is abstracted out
  • We can limiting the action annotation usage to only for redirect/fixed-response

Cons

  • The above limitation exists until we implemented ingress group support.

Alternative2: Extend current alb.ingress.kubernetes.io/actions annotation. (prefer this with 馃槂 )

The actions annotation is introduce to support redirect and fixed-response action and it's an single entry. However in ALB API, the actions is the combination of auth action(optional) and response action(forward to targetGroup/redirect/fixed-response).
To extend this, we need to allow specify an array of actions(while also accept an single action).
Also, we need to represent forward to targetGroup action(need to encode serviceName and servicePort in this action config)

An sample action that enables OIDC support can be like below:

alb.ingress.kubernetes.io/actions: >-
[{
          "Type": "authenticate-oidc",
          "AuthenticateOidcConfig": {
              "Issuer": "https://idp-issuer.com",
              "AuthorizationEndpoint": "https://authorization-endpoint.com",
              "TokenEndpoint": "https://token-endpoint.com",
              "UserInfoEndpoint": "https://user-info-endpoint.com",
              "ClientId": "abcdefghijklmnopqrstuvwxyz123456789",
              "ClientSecretSecretName": "",     // -< name of kubernetes secret, we'll load the content from it.
              "SessionCookieName": "my-cookie",
              "SessionTimeout": 3600,
              "Scope": "email",
              "AuthenticationRequestExtraParams": {
                  "display": "page",
                  "prompt": "login"
              },
              "OnUnauthenticatedRequest": "deny"
          },
}, {
       "Type": "forward",
       "ServiceName": "xxx",
       "ServicePort": "xxxx"
}]

Pros:

  • It's an extension to existing action annotation

Cons:

  • The action annotation become more complicated.
  • The action annotation won't be a direct mapping to ALB actions, since
    • we need to include serviceName & servicePort in forward action. And the targetGroup component should honor it when creating targetGroups.
    • introduce of ClientSecretSecretName which will be the k8s secret name to load clientSecret.

Alternative3:(prefer this with 鉂わ笍 )

Help contribute your preferred alternative as comments .

References

#592

@M00nF1sh M00nF1sh changed the title design proposal for auth support [WIP] design proposal for auth support Nov 28, 2018
@hluchej

This comment has been minimized.

Copy link

hluchej commented Nov 28, 2018

Hi, the proposal looks nice and I voted for Alternative 1.

I am just thinking about a possible extension of this proposal regarding injection of the OIDC client secret.

We are currently using AWS Secrets Manager for many purposes and we would likely prefer to fetch the secret value from there instead of from K8s Secret. In our applications, we are not really forced to use k8s secrets because we can provide just ARN of the secret in Secrets Manager via e.g. env property and apps can fetch the value from there.

We cannot use this approach with ingress controller because there are no "our" services where we would implement the fetching so the fetching has to be part of the ingress controller.

So I was thinking about extending the auth-oidc-cfg annotation. Example:

{
'Issuer':'xxx', 
'ClientID':'xxx', 
'ClientSecretSecretsManagerId':'xxx', 
'AuthorizationEndpoint':'xxx', 
'TokenEndpoint':'xxx', 
'UserInfoEndpoint':'xxx'
}

where ClientSecretSecretsManagerId is basically this: https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html#SecretsManager-GetSecretValue-request-SecretId - therefore can contain full ARN of the secret or "friendly name" of the secret.
Obviously only one of the ClientSecretSecretManagerId and ClientSecretSecretName can be specified, otherwise configuration error should be raised.

Pros:

  • Adds option to use AWS Secrets Manager for OIDC credentials storage

Cons:

  • additional complexity

What do you think?

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Nov 28, 2018

  • AWS Secrets Manager

The use case make sense to me. However it's indeed more complexity added(and more iam permissions).
I think ideally there should be a separate component/controller that map secret manager into k8s secret...

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Dec 4, 2018

i have some concerns around the usage of
SecretName. Since this introduced a privilege escalation, someone creating ingress can stolen secrets that don't belong to him=.=

There is two way for this:

  1. Via documentation to tell users to only grant controller permission to read specific secrets.
  2. Using a central managed secret(configurable via controller flag), so only certain k8s admin can manage that secret. The format of that secret can be like this:
{
     "ingresses": {
           "namespace/name": secretForThatIngress
      },
     "services":  {
           "namespace/name": secretForThatService
     }
}

Which one do you guys prefer? 馃槃
For me, i'd prefer the first one, and will produce the PR based on it by tomorrow. We can do following to improve it in the future:

  • We can limiting only certain user to create ingress with secret via admission webhook.
  • We can require that secret to be annotated with special annotation like "alb.k8s.io/acl-ingress: namespace/name", to denote that certain ingress can access it 馃槃
@hluchej

This comment has been minimized.

Copy link

hluchej commented Dec 4, 2018

I guess option 1 could be sufficient now.

@JakubJecminek

This comment has been minimized.

Copy link

JakubJecminek commented Dec 4, 2018

+1 for option 1

@M00nF1sh M00nF1sh force-pushed the M00nF1sh:cognito-support branch from 088c752 to c41d86e Dec 4, 2018
@k8s-ci-robot k8s-ci-robot added size/L and removed size/XS labels Dec 4, 2018
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Dec 4, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: M00nF1sh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Dec 5, 2018

For OIDC, users can specify clientId & clientSecret directly as json

{  
   "Issuer":"xxx",
   "AuthorizationEndpoint":"xxx",
   "TokenEndpoint":"xxx",
   "UserInfoEndpoint":"xxx",
   
   "ClientID":"xxx",
   "ClientSecret":"xxx"
}

Or specify a secret that contains it.

{  
   "Issuer":"xxx",
   "AuthorizationEndpoint":"xxx",
   "TokenEndpoint":"xxx",
   "UserInfoEndpoint":"xxx",
   
   "SecretNamespace":"xxx",
   "SecretName":"xxx"
}

should SecretNamespace be forced to ingress's namespace?Currently i'm a bit uncomfortable from security perspective =.=

The format of secret is as follows:

apiVersion: v1
kind: Secret
metadata:
  name: test-secret
data:
  clientId: base64 version of clientId
  clientSecret: base64 version of clientSecret

current code works with some imperfection:

  • the listener actions will be updated every time when sync events happens(ingress/service changes or
    every 60minutes by default). This is due to we cannot retrieve clientSecret via API on aws, so we cannot compare).

I can change it to be:

  1. when compare the actions, don't check for clientSecret. (users will need to delete the auth-type annotation and add it back, if only clientSecret are changed).
    To avoid extra work above, Another change could be record the desired status in last sync, and compare them, and only sync when anything changed. But this will be more complicated.

some considerations during implementation:

  • I created a separate component to generate authConfig, this field will become a field of our "enhanced ingress" model, which contains list of routes, and each route(host&ingressPath) have it's own authConfig. This will be needed to support ingress group, since different path can have different auth requirement)
@hluchej

This comment has been minimized.

Copy link

hluchej commented Dec 5, 2018

Hi, I am not a security expert so I cannot judge regarding forcing the SecretNamespace...

I can change it to be: when compare the actions, don't check for clientSecret. (users will need to delete the auth-type annotation and add it back, if only clientSecret are changed)

I would not like that change. Seems to require external action which is error prone and it will be really hard to realize why the change of clientSecret did not propagate... We sometimes regenerate secrets (e.g. because employee offboarding) and clientId does not change in these cases. So at least Okta does not create a new clientId when you want to rotate the secret.

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Dec 5, 2018

@hluchej Thanks for your comments, these points are pretty valuable. I'll add watch on the secrets referenced by ingresses to trigger sync when they changed.

@M00nF1sh M00nF1sh force-pushed the M00nF1sh:cognito-support branch from c41d86e to 775603f Dec 13, 2018
@k8s-ci-robot k8s-ci-robot added size/XL and removed size/L labels Dec 13, 2018
@goruha

This comment has been minimized.

Copy link

goruha commented Dec 22, 2018

@M00nF1sh when do you expect to have the PR merged?
I'd like to test it asap )

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Dec 22, 2018

@M00nF1sh when do you expect to have the PR merged?
I'd like to test it asap )

Hi, it's currently under an internal sec review, i'll update the status here once it's finished. (it should went through smoothly)
a few things have changed from this PR so far

  1. I removed the support for specify OIDC clientID&clientSecret directly and the ability to specify secretNamespace to be secure by default 馃槃
  2. added support to rotating secrets

I can push the latest change to a personal branch if you'd like to test.

@goruha

This comment has been minimized.

Copy link

goruha commented Dec 26, 2018

@M00nF1sh it would be perfect.
I would be happy to help you with beta testing.

@M00nF1sh M00nF1sh force-pushed the M00nF1sh:cognito-support branch from 775603f to 996d628 Jan 3, 2019
@k8s-ci-robot k8s-ci-robot added size/XXL and removed size/XL labels Jan 3, 2019
@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 3, 2019

@M00nF1sh it would be perfect.
I would be happy to help you with beta testing.

Hi, sorry for the late reply, just get back from holiday trip 馃槂
I have pushed the changes to https://github.com/M00nF1sh/aws-alb-ingress-controller/tree/cognito-support.
You can build it yourself by

export PREFIX=your_docker_hub_account/aws-alb-ingress-controller 
export TAG=PR-754
make clean && make container && make push

or using my image: m00nf1sh/aws-alb-ingress-controller:PR-754

Sample ingress I have tested:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: testcase
  name: echoserver
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: echoserver
    spec:
      containers:
      - name: echoserver
        image: gcr.io/google_containers/echoserver:1.4
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  namespace: testcase
  name: echoserver
spec:
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  type: NodePort
  selector:
    app: echoserver
---
apiVersion: v1
kind: Secret
metadata:
  namespace: testcase
  name: odic-secret
data:
  clientId: xxxx
  clientSecret: xxxxx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  namespace: testcase
  name: echoserver
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: xxxxx
   alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/auth-type: oidc
    alb.ingress.kubernetes.io/auth-idp-oidc: '{"Issuer":"https://albingress.auth0.com/","AuthorizationEndpoint":"https://albingress.auth0.com/authorize","TokenEndpoint":"https://albingress.auth0.com/oauth/token","UserInfoEndpoint":"https://albingress.auth0.com/userinfo","SecretName":"odic-secret"}'
spec:
  rules:
    - http:
        paths:
           - backend:
               serviceName: ssl-redirect
               servicePort: use-annotation
          - path: /normal-path
            backend:
              serviceName: echoserver
              servicePort: 80

Some tip for testing,

  1. clientId&clientSecret can be generated via echo -n "clientIdFromOIDC" | base64
  2. auth action can only(and will) be applied to https listeners, so you should have only https listener(by omit the alb.ingress.kubernetes.io/listen-ports and specify alb.ingress.kubernetes.io/certificate-arn, or redirect http to https)
@goruha

This comment has been minimized.

Copy link

goruha commented Jan 7, 2019

@M00nF1sh thanks. Will try on this week

@M00nF1sh M00nF1sh force-pushed the M00nF1sh:cognito-support branch from 996d628 to 659556d Jan 10, 2019
@M00nF1sh M00nF1sh changed the title [WIP] design proposal for auth support cognito/oidc auth support Jan 10, 2019
@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 11, 2019

@M00nF1sh thanks. Will try on this week

@goruha Have you got time to test it out?
I have tested both OIDC & Cognito works 馃槃 , and plans to merge it and release a new version.

@goruha

This comment has been minimized.

Copy link

goruha commented Jan 11, 2019

@M00nF1sh testing now.
Will report feedback today.

@goruha

This comment has been minimized.

Copy link

goruha commented Jan 11, 2019

@M00nF1sh
I have

This site can鈥檛 be reached The webpage at https://example.com/ might be temporarily down or it may have moved permanently to a new web address.
ERR_SPDY_PROTOCOL_ERROR

In browser.
Did I config OIDC app wrong?

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 11, 2019

@M00nF1sh
I have

This site can鈥檛 be reached The webpage at https://example.com/ might be temporarily down or it may have moved permanently to a new web address.
ERR_SPDY_PROTOCOL_ERROR

In browser.
Did I config OIDC app wrong?

it's strange to get SPDY Error..what's your configuration looks like?
For my auth0 based OIDC, it's works fine

alb.ingress.kubernetes.io/auth-type: cognito
alb.ingress.kubernetes.io/auth-idp-oidc: '{"Issuer":"https://albingress.auth0.com/","AuthorizationEndpoint":"https://albingress.auth0.com/authorize","TokenEndpoint":"https://albingress.auth0.com/oauth/token","UserInfoEndpoint":"https://albingress.auth0.com/userinfo","SecretName":"odic-secret"}'
apiVersion: v1
kind: Secret
metadata:
  namespace: testcase
  name: odic-secret
data:
  clientId: N014SzFiY0RSbmZyblFBd05CRGdyTWlVN0J2TG9kVXY=
  clientSecret: TnlubjBxVGxkc3FYV2ZucFgxNW1pYVpadXhsQWYtM1gzY3JwXzA4bDlOcGZ4YkdQVTZCd2oyZnlnN1hLZWpLdQ==

The secret(clientId&clientSecret) is generated by echo -n "xxxxx" | base64

@goruha

This comment has been minimized.

Copy link

goruha commented Jan 12, 2019

@M00nF1sh Confirm OIDC work as a charm.
I guess we can merge th PR

@goruha

This comment has been minimized.

Copy link

goruha commented Jan 12, 2019

/lgtm

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jan 12, 2019

@goruha: changing LGTM is restricted to assignees, and only kubernetes-sigs/aws-alb-ingress-controller repo collaborators may be assigned issues.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@marcust

This comment has been minimized.

Copy link

marcust commented Jan 12, 2019

Hey, I stumbled accross this because I wanted to do Cognito auth on my ingress alb, and after fighting with all the other issues with an alb ingress I can confirm that Cognito also looks good, I get a 401, don't really have the time right now to actually get past it, but in the AWS console it looks good

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 14, 2019

Hey, I stumbled accross this because I wanted to do Cognito auth on my ingress alb, and after fighting with all the other issues with an alb ingress I can confirm that Cognito also looks good, I get a 401, don't really have the time right now to actually get past it, but in the AWS console it looks good

My working cognito setup is like below:

alb.ingress.kubernetes.io/auth-idp-cognito: '{"UserPoolArn":"arn:aws:cognito-idp:us-west-2:xxxx:userpool/xxxxx", "UserPoolClientId":"xxxxx", "UserPoolDomain":"m00nf1sh"}'

I can help debug your issue if you can share more details, like config and error message if any.

@bigkraig

This comment has been minimized.

Copy link
Member

bigkraig commented Jan 14, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Jan 14, 2019
@k8s-ci-robot k8s-ci-robot merged commit 49e66e5 into kubernetes-sigs:master Jan 14, 2019
6 checks passed
6 checks passed
cla/linuxfoundation M00nF1sh authorized
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls First build on cognito-support at 44.746%
Details
pull-aws-alb-ingress-controller-lint Job succeeded.
Details
pull-aws-alb-ingress-controller-test Job succeeded.
Details
tide In merge pool.
Details
@AdeOpe

This comment has been minimized.

Copy link

AdeOpe commented Jan 15, 2019

Hi,
i just tried this from your repo, its seems to work good. thank you :)

Is there a way to specify:

  • Path-A to have oidc
  • and Path B to not authenticate.

E.g. grafana has direct integration with oidc & oauth2 so there's no need to challenge before landing on the grafana page.
whilst other paths, i want to protect before landing on the destination

(apologies, its probably something i have missed)
any ideas?

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 15, 2019

Hi,
i just tried this from your repo, its seems to work good. thank you :)

Is there a way to specify:

  • Path-A to have oidc
  • and Path B to not authenticate.

E.g. grafana has direct integration with oidc & oauth2 so there's no need to challenge before landing on the grafana page.
whilst other paths, i want to protect before landing on the destination

(apologies, its probably something i have missed)
any ideas?

Hi,
@AdeOpe these annotations can be applied to service objects as well.
so if your ingress looks like

/path1: service1
/path2: service2

You can add these auth annotation on service1, to get /path1 authenticated while path2 not 馃槃

@AdeOpe

This comment has been minimized.

Copy link

AdeOpe commented Jan 15, 2019

Hi,
i just tried this from your repo, its seems to work good. thank you :)
Is there a way to specify:

  • Path-A to have oidc
  • and Path B to not authenticate.

E.g. grafana has direct integration with oidc & oauth2 so there's no need to challenge before landing on the grafana page.
whilst other paths, i want to protect before landing on the destination
(apologies, its probably something i have missed)
any ideas?

Hi,
@AdeOpe these annotations can be applied to service objects as well.
so if your ingress looks like

/path1: service1
/path2: service2

You can add these auth annotation on service1, to get /path1 authenticated while path2 not 馃槃

thanks for the reply @M00nF1sh , this should like what i am looking for.

I just need some direction with the implementation.
sorry to be a pain, but this is the part where i am confused.

can this all be done directly in the ingress.yaml?
or shall i add the annotation directly to the service1.yaml?

e.g. so i currently have the following in my ingress.yaml

  - path: /path1/*
    backend:
      serviceName: app-1-service
      servicePort: 80
  - path: /path2/*
    backend:
      serviceName: app-2-service
      servicePort: 80

with the above and with oidc everything path is now integrated with oidc.
how do instruct the ingress.yaml to only use oidc for path1 and not path2?

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 15, 2019

@AdeOpe
Hi, currently, you have to add these annotations to the service.yaml of app-1-service.

@AdeOpe

This comment has been minimized.

Copy link

AdeOpe commented Jan 15, 2019

ah ok got it. I will give it a try :)
thank you again

@AdeOpe

This comment has been minimized.

Copy link

AdeOpe commented Jan 15, 2019

@M00nF1sh - i am just reporting back after my test.
this works brilliantly.
I decided to use it on the service.yaml instead of the ingress.yaml

thank you again 馃憤

@AdeOpe

This comment has been minimized.

Copy link

AdeOpe commented Jan 22, 2019

Hi @M00nF1sh ,
can you please confirm if you have uploaded this?

I get the following error with v1.1.0 & v1.1.1

Warning Failed 10s kubelet, ip-10-1-6-205.eu-west-1.compute.internal Failed to pull image "894847497797.dkr.ecr.us-west-2.amazonaws.com/aws-alb-ingress-controller:v1.1.0": rpc error: code = Unknown desc = Error response from daemon: manifest for 894847497797.dkr.ecr.us-west-2.amazonaws.com/aws-alb-ingress-controller:v1.1.0 not found

@M00nF1sh

This comment has been minimized.

Copy link
Collaborator Author

M00nF1sh commented Jan 22, 2019

Hi @M00nF1sh ,
can you please confirm if you have uploaded this?

I get the following error with v1.1.0 & v1.1.1

Warning Failed 10s kubelet, ip-10-1-6-205.eu-west-1.compute.internal Failed to pull image "894847497797.dkr.ecr.us-west-2.amazonaws.com/aws-alb-ingress-controller:v1.1.0": rpc error: code = Unknown desc = Error response from daemon: manifest for 894847497797.dkr.ecr.us-west-2.amazonaws.com/aws-alb-ingress-controller:v1.1.0 not found

@AdeOpe , sorry for the confusing.
The image have moved to docker.io/amazon/aws-alb-ingress-controller:v1.1.1 since v1.0.1.
(since the helm e2e tests cannot be run with ecr images, so we moved to company's dockerhub).

BTW, the release notes contains the image repo, and required upgrade actions(e.g. new iam permissions), be sure to check it out before doing upgrades 馃槃 .

@AdeOpe

This comment has been minimized.

Copy link

AdeOpe commented Jan 23, 2019

ah great!
thanks @M00nF1sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can鈥檛 perform that action at this time.