-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
STORM-997: Add proxy user functionality for storm hdfs connector. #692
Conversation
@priyank5485 we need to add this trident as well. |
@harshach thanks for the review. Yes, I will do it. I realized that it needs to be added for trident after creating the pull request. |
@Parth-Brahmbhatt @ptgoetz Can you please review this? |
@priyank5485 and others I am not totally sure that this is a good idea. If HDFS is running secure and Storm is running insecure, with this feature enabled you have now disabled security in HDFS, so anyone who can submit a topology (a.k.a everyone) can pretend to be any user in HDFS. To me I would much rather see the feature that allows this to work with an arbitrary UGI. Or better have a keytab installed on the worker nodes then you just need a config to point to that keytab and the principal you want to use out of it. I am -0 on this change. I am not going to block it, but I would prefer to see it done a different way. |
@revans2 I think what you mentioned above would not happen. I mean the part where you say that storm running in unsecured mode and hdfs in secured mode will disable security. Please correct me if i am wrong. I am new to this. But the way i imagine it will work is that the proxy user functionality code(which is doAs part) will be executed only after the user is logged in to HDFS. In secured mode user will already be validated using keytab. Above that hdfs needs config entries in core-site.xml which is needed for the proxy user functionality to work. Please look at the configurations section at http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Superusers.html |
@priyank5485 You are correct that the name-node needs to be configured to explicitly allow super users and/or groups and optionally restrict those to specific hosts. My issue is not with the code but with how it is intended to be used which is why I am -0. I will not block this going in, but I will not support it either. From the description of STORM-997
To me that sounds like I have an "guest" account on my computer that everyone logs in to. Some of the users don't want to be "guest" so instead of giving them all a separate login and password. I will grant "guest" sudo privileges so they can become any user. The issue is the chain of attach. If storm is "non-secure" anyone with access to the nimbus thrift port can run arbitrary code on the storm cluster. If you configure HDFS so the "storm" user can become anyone on HDFS anyone with access to the nimbus thrift port now has complete access to all of HDFS. You have effectively made HDFS as secure as your storm cluster which by definition is "non-secure". To really make this correct you either want to make storm secure (give each user their own login and password to your computer, and have them stop using "guest") or at a minimum severely restrict what the storm user can do on HDFS. To do that I would think you could make AutoTGT work with an insecure storm cluster. This would not be that hard, we just have to let it look someplace else for the TGT, instead of relying on the StormClient section of the jaas.conf. Or update AutoHDFS to work in insecure mode and have some configuration to restrict which delegation tokens it will fetch for users of this cluster. To be sure that works you have to be sure "nimbus" is a different user from "storm", or at least HDFS will only allow super user access from nimbus. This is still not great because anyone with access to ZK could now steal either the TGT or the delegation tokens, but you have only made some very specific HDFS accounts "non-secure" instead of all of HDFS. |
We are closing stale Pull Requests to make the list more manageable. Please re-open any Pull Request that has been closed in error. Closes apache#608 Closes apache#639 Closes apache#640 Closes apache#648 Closes apache#662 Closes apache#668 Closes apache#692 Closes apache#705 Closes apache#724 Closes apache#728 Closes apache#730 Closes apache#753 Closes apache#803 Closes apache#854 Closes apache#922 Closes apache#986 Closes apache#992 Closes apache#1019 Closes apache#1040 Closes apache#1041 Closes apache#1043 Closes apache#1046 Closes apache#1051 Closes apache#1078 Closes apache#1146 Closes apache#1164 Closes apache#1165 Closes apache#1178 Closes apache#1213 Closes apache#1225 Closes apache#1258 Closes apache#1259 Closes apache#1268 Closes apache#1272 Closes apache#1277 Closes apache#1278 Closes apache#1288 Closes apache#1296 Closes apache#1328 Closes apache#1342 Closes apache#1353 Closes apache#1370 Closes apache#1376 Closes apache#1391 Closes apache#1395 Closes apache#1399 Closes apache#1406 Closes apache#1410 Closes apache#1422 Closes apache#1427 Closes apache#1443 Closes apache#1462 Closes apache#1468 Closes apache#1483 Closes apache#1506 Closes apache#1509 Closes apache#1515 Closes apache#1520 Closes apache#1521 Closes apache#1525 Closes apache#1527 Closes apache#1544 Closes apache#1550 Closes apache#1566 Closes apache#1569 Closes apache#1570 Closes apache#1575 Closes apache#1580 Closes apache#1584 Closes apache#1591 Closes apache#1600 Closes apache#1611 Closes apache#1613 Closes apache#1639 Closes apache#1703 Closes apache#1711 Closes apache#1719 Closes apache#1737 Closes apache#1760 Closes apache#1767 Closes apache#1768 Closes apache#1785 Closes apache#1799 Closes apache#1822 Closes apache#1824 Closes apache#1844 Closes apache#1874 Closes apache#1918 Closes apache#1928 Closes apache#1937 Closes apache#1942 Closes apache#1951 Closes apache#1957 Closes apache#1963 Closes apache#1964 Closes apache#1965 Closes apache#1967 Closes apache#1968 Closes apache#1971 Closes apache#1985 Closes apache#1986 Closes apache#1998 Closes apache#2031 Closes apache#2032 Closes apache#2071 Closes apache#2076 Closes apache#2108 Closes apache#2119 Closes apache#2128 Closes apache#2142 Closes apache#2174 Closes apache#2206 Closes apache#2297 Closes apache#2322 Closes apache#2332 Closes apache#2341 Closes apache#2377 Closes apache#2414 Closes apache#2469
We are closing stale Pull Requests to make the list more manageable. Please re-open any Pull Request that has been closed in error. Closes apache#608 Closes apache#639 Closes apache#640 Closes apache#648 Closes apache#662 Closes apache#668 Closes apache#692 Closes apache#705 Closes apache#724 Closes apache#728 Closes apache#730 Closes apache#753 Closes apache#803 Closes apache#854 Closes apache#922 Closes apache#986 Closes apache#992 Closes apache#1019 Closes apache#1040 Closes apache#1041 Closes apache#1043 Closes apache#1046 Closes apache#1051 Closes apache#1078 Closes apache#1146 Closes apache#1164 Closes apache#1165 Closes apache#1178 Closes apache#1213 Closes apache#1225 Closes apache#1258 Closes apache#1259 Closes apache#1268 Closes apache#1272 Closes apache#1277 Closes apache#1278 Closes apache#1288 Closes apache#1296 Closes apache#1328 Closes apache#1342 Closes apache#1353 Closes apache#1370 Closes apache#1376 Closes apache#1391 Closes apache#1395 Closes apache#1399 Closes apache#1406 Closes apache#1410 Closes apache#1422 Closes apache#1427 Closes apache#1443 Closes apache#1462 Closes apache#1468 Closes apache#1483 Closes apache#1506 Closes apache#1509 Closes apache#1515 Closes apache#1520 Closes apache#1521 Closes apache#1525 Closes apache#1527 Closes apache#1544 Closes apache#1550 Closes apache#1566 Closes apache#1569 Closes apache#1570 Closes apache#1575 Closes apache#1580 Closes apache#1584 Closes apache#1591 Closes apache#1600 Closes apache#1611 Closes apache#1613 Closes apache#1639 Closes apache#1703 Closes apache#1711 Closes apache#1719 Closes apache#1737 Closes apache#1760 Closes apache#1767 Closes apache#1768 Closes apache#1785 Closes apache#1799 Closes apache#1822 Closes apache#1824 Closes apache#1844 Closes apache#1874 Closes apache#1918 Closes apache#1928 Closes apache#1937 Closes apache#1942 Closes apache#1951 Closes apache#1957 Closes apache#1963 Closes apache#1964 Closes apache#1965 Closes apache#1967 Closes apache#1968 Closes apache#1971 Closes apache#1985 Closes apache#1986 Closes apache#1998 Closes apache#2031 Closes apache#2032 Closes apache#2071 Closes apache#2076 Closes apache#2108 Closes apache#2119 Closes apache#2128 Closes apache#2142 Closes apache#2174 Closes apache#2206 Closes apache#2297 Closes apache#2322 Closes apache#2332 Closes apache#2341 Closes apache#2377 Closes apache#2414 Closes apache#2469
No description provided.