Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

METRON-2074: Script to handle TGT renewal with Storm and Kerberos enabled #1382

Closed
wants to merge 3 commits into from

Conversation

@mmiklavc
Copy link
Contributor

mmiklavc commented Apr 13, 2019

Contributor Comments

https://issues.apache.org/jira/browse/METRON-2074

Pull Request Checklist

For all changes:

  • Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
  • Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
  • Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

  • Have you included steps to reproduce the behavior or problem that is being changed or addressed?

  • Have you included steps or a guide to how the change may be verified and tested manually?

  • Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:

    mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh 
    
  • n/a Have you written or updated unit tests and or integration tests to verify your changes?

  • n/a If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

  • Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:

    cd site-book
    mvn site
    
mmiklavc added 2 commits Oct 8, 2018
…s documentation. Add script to metron-common section in metron.spec.
@mmiklavc

This comment has been minimized.

Copy link
Contributor Author

mmiklavc commented Apr 13, 2019

Test Instructions

  1. Run up full dev
  2. Setup Kerberos per the instructions in https://github.com/apache/metron/blob/master/metron-deployment/Kerberos-ambari-setup.md
  3. Modify the maxlife and maxrenewlife for your TGT. Here we're setting it to 3 and 4 minutes, respectively, but you can set the values even shorter for testing purposes.
    kadmin.local -q "modprinc -maxlife 3minutes krbtgt/EXAMPLE.COM@EXAMPLE.COM"
    kadmin.local -q "modprinc -maxrenewlife 4minutes krbtgt/EXAMPLE.COM@EXAMPLE.COM"
    
  4. Restart the bro parser topology. We'll focus on one topology for this exercise.
  5. Allow your max renew time to pass. If you look at your Storm bolt logs, you should see at least one renewal prior to expiration, e.g.
    2018-09-17 06:52:54.506 o.a.s.s.o.a.z.Login Thread-1 [INFO] Initiating logout for metron@EXAMPLE.COM
    2018-09-17 06:52:54.507 o.a.s.s.o.a.z.Login Thread-1 [INFO] Initiating re-login for metron@EXAMPLE.COM
    2018-09-17 06:52:54.521 o.a.s.s.o.a.z.Login Thread-1 [INFO] TGT valid starting at:        Mon Sep 17 06:53:17 UTC 2018
    2018-09-17 06:52:54.527 o.a.s.s.o.a.z.Login Thread-1 [INFO] TGT expires:                  Mon Sep 17 07:03:17 UTC 2018
    2018-09-17 06:52:54.527 o.a.s.s.o.a.z.Login Thread-1 [INFO] TGT refresh sleeping until: Mon Sep 17 07:01:29 UTC 2018
    
  6. You should also start to see error messages like the following in the Nimbus logs:
    2018-09-17 07:12:52.064 o.a.s.s.a.k.AutoTGT timer [INFO] Renewing TGT for metron@EXAMPLE.COM
    2018-09-17 07:12:52.064 o.a.s.s.a.k.AutoTGT timer [WARN] Failed to refresh TGT javax.security.auth.RefreshFailedException: This ticket is past its last renewal time.
    
  7. Run the tgt_renew.py script per the Kerberos README instructions in this PR and verify that you again start to see TGT login/renewal for metron@EXAMPLE.COM
@mmiklavc

This comment has been minimized.

Copy link
Contributor Author

mmiklavc commented Apr 15, 2019

Appears to be an intermittent test failure. I've logged it in this Jira - https://issues.apache.org/jira/browse/METRON-2077. Re-running Travis.

Copy link
Contributor

anandsubbu left a comment

Thanks @mmiklavc for the script ! Works as advertised. I was able to validate the script on a kerberized multi-node CentOS7 cluster with the testing steps noted in the PR description.

+1 from me, pending a minor change to the README


```
su - metron
for item in epel-release centos-release-scl "@Development tools" python27 python27-scldevel python27-python-virtualenv libselinux-python; do yum install -y $item; done

This comment has been minimized.

Copy link
@anandsubbu

anandsubbu Apr 17, 2019

Contributor

This needs to be run with sudo to be consistent with the steps that follow.

Run the following on a node with a Storm and Metron client installed. We need python 2.7 via virtualenv for this to work correctly.

```
su - metron

This comment has been minimized.

Copy link
@anandsubbu

anandsubbu Apr 17, 2019

Contributor

Do you think it would be good to complete all yum installation steps as root and then do su - metron? I am fine if you feel otherwise. Just that on the environment that I am testing, I do not have sudoers configured for metron user.

This comment has been minimized.

Copy link
@mmiklavc

mmiklavc Apr 17, 2019

Author Contributor

I've updated the docs based on your feedback. How's it look now?

This comment has been minimized.

Copy link
@anandsubbu

anandsubbu Apr 17, 2019

Contributor

+1 looks great!

@asfgit asfgit closed this in b6d8cad Apr 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.