Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reuse DataTransferAuthorizable in DataTransferResource #971

Closed
wants to merge 1 commit into from
Closed

Reuse DataTransferAuthorizable in DataTransferResource #971

wants to merge 1 commit into from

Conversation

mcgilman
Copy link
Contributor

NIFI-2704:

  • Re-using the DataTransferAuthorizable in the DataTransferResource.
  • Removing use of the DataTransferResource when obtaining site to site details as it performs additional unnecessary checks.
  • Code clean up.

@mcgilman
NIFI-2704:
3340e46 
- Re-using the DataTransferAuthorizable in the DataTransferResource.
- Removing use of the DataTransferResource when obtaining site to site details as it performs additional unnecessary checks.
- Code clean up.
@ijokarumawak
Copy link
Member

Reviewing.

@ijokarumawak
Copy link
Member

Accessibility test of NiFi.Q based on Policies on NiFi.P

NiFi.Q (RPG) <--pulls data from-- NiFi.P (output-port)

I've done following test with output port. I did the same test for input port, too, got the same behavior.

'retrieve site-to-site details' policy

  1. OK: Without NiFi.Q group: RPG can't retrieve site-to-site info [1]
  2. OK: With NiFi.Q group: RPG can retrieve site-to-site info, but since NiFi.Q is not allowed to access NiFi.P's port, it can't see any port [2]

'send data via site-to-site' policy

  1. OK: Without NiFi.Q group: NiFi.Q can't see any port [2]
  2. OK: With NiFi.Q group: NiFi.Q can see the output port [3]
  3. OK: By enabling transmission, NiFi.Q can receive data from the output port [4]
  4. What if the NiFi.Q group is removed from this policy at this point? NiFi.Q started receiving 403, but no indication on the UI, other than the read warning mark on the relationship [5] Adding NiFi.Q group to the policy recovers data transfer. [6]
  5. How about RAW socket? "User not authorized" error is shown on bulletin board, output port reports error, too [7]

Result

Confirmed that this PR relaxes SiteToSiteResource authorization check, and addresses policy setting issue reported by NIFI-2550.

By doing step, 4 and 5, I noticed there's a difference on the UI in terms of how NiFi UI reports auth error to end user. I will submit another JIRA for this.

Evidences:

[1]
Policies on NiFi.P
image

2016-08-31 01:56:39,479 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine) GET https://0.p.nifi.aws.mine:8443/nifi-api/site-to-site (source ip: 172.31.28.33)
2016-08-31 01:56:39,479 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine
2016-08-31 01:56:39,479 INFO [NiFi Web Server-292] o.a.n.w.a.c.AccessDeniedExceptionMapper L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine does not have permission to access the requested resource. Returning Forbidden response.

RPG on NiFi.Q
image

[2]
image
image

2016-08-31 01:59:29,223 INFO [NiFi Web Server-289] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine) GET https://0.p.nifi.aws.mine:8443/nifi-api/site-to-site (source ip: 172.31.28.33)
2016-08-31 01:59:29,223 INFO [NiFi Web Server-289] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine
2016-08-31 01:59:29,230 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (<L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine><L=0.p.nifi.aws.mine, C=US, CN=0.p.nifi.aws.mine>) GET https://0.p.nifi.aws.mine:8443/nifi-api/site-to-site (source ip: 172.31.28.12)
2016-08-31 01:59:29,231 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine

NiFi.Q can access the port
image

[3]
image

[4]
image

2016-08-31 02:08:03,474 INFO [NiFi Web Server-335] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine) POST https://0.p.nifi.aws.mine:8443/nifi-api/data-transfer/output-ports/f46f3046-c250-1f1b-0000-0000029f9794/transactions (source ip: 172.31.28.33)
2016-08-31 02:08:03,479 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine) GET https://0.p.nifi.aws.mine:8443/nifi-api/data-transfer/output-ports/f46f3046-c250-1f1b-0000-0000029f9794/transactions/f6caf1f3-846c-42f6-8084-51f1883c07b2/flow-files (source ip: 172.31.28.33)
2016-08-31 02:08:05,841 INFO [NiFi Web Server-443] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine) DELETE https://0.p.nifi.aws.mine:8443/nifi-api/data-transfer/output-ports/f46f3046-c250-1f1b-0000-0000029f9794/transactions/f6caf1f3-846c-42f6-8084-51f1883c07b2 (source ip: 172.31.28.33)
2016-08-31 02:08:05,841 INFO [NiFi Web Server-443] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine

[5]
image

image

NiFi.P's log

2016-08-31 02:22:26,449 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine) POST https://0.p.nifi.aws.mine:8443/nifi-api/
data-transfer/output-ports/f46f3046-c250-1f1b-0000-0000029f9794/transactions (source ip: 172.31.28.33)
2016-08-31 02:22:26,449 INFO [NiFi Web Server-292] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine
2016-08-31 02:22:26,449 INFO [NiFi Web Server-292] o.a.n.w.a.c.AccessDeniedExceptionMapper L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine does not have permission to access the requested resource. Return
ing Forbidden response.

NiFi.Q's log

2016-08-31 02:22:26,445 DEBUG [Timer-Driven Process Thread-3] o.a.n.r.util.SiteToSiteRestApiClient initiateTransaction responseCode=403
2016-08-31 02:22:26,445 DEBUG [Timer-Driven Process Thread-3] o.a.n.r.util.SiteToSiteRestApiClient readResponse responseMessage=Unable to perform the desired action due to insufficient permissions. Contact the system administrator.

[6]
image

[7]
RAW socket transfer protocol
image

2016-08-31 02:44:12,080 ERROR [Timer-Driven Process Thread-7] o.a.nifi.remote.StandardRemoteGroupPort
org.apache.nifi.remote.exception.HandshakeException: Received unexpected response User Not Authorized: StandardRootGroupPort[id=f46f3046-c250-1f1b-0000-0000029f9794] authorization failed for user L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine because Access is denied
        at org.apache.nifi.remote.protocol.socket.SocketClientProtocol.handshake(SocketClientProtocol.java:179) ~[nifi-site-to-site-client-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]
        at org.apache.nifi.remote.protocol.socket.SocketClientProtocol.handshake(SocketClientProtocol.java:105) ~[nifi-site-to-site-client-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT]

image

2016-08-31 02:48:16,499 WARN [Site-to-Site Worker Thread-46] o.a.nifi.remote.StandardRootGroupPort StandardRootGroupPort[id=f46f3046-c250-1f1b-0000-0000029f9794] authorization failed for user L=0.q.nifi.aws.mine, C=US, CN=0.q.nifi.aws.mine because Access is denied

@ijokarumawak
Copy link
Member

@mcgilman Thanks for the fix. I've reviewed the code and did a test described above comment. This PR works as expected.

+1 for the change

However, I couldn't test a situation that multiple user identities being authorized as a chain. Which doesn't happen with 1.0.0 so far, since data transfer is a direct Peer to Peer communication.

@ijokarumawak
Copy link
Member

Submitted another issue which was found while testing this PR. NIFI-2718: HTTP Site-to-Site doesn't report port auth failure well, compared to RAW

@asfgit asfgit closed this in 9e10371 Aug 31, 2016
@mcgilman
Copy link
Contributor Author

Thanks @ijokarumawak. I've merged this into master following your +1.

@mcgilman mcgilman deleted the NIFI-2704 branch September 15, 2016 13:41
JPercivall pushed a commit to JPercivall/nifi that referenced this pull request Dec 14, 2016
@mcgilman @JPercivall
NIFI-2704:
bde1c24 
- Re-using the DataTransferAuthorizable in the DataTransferResource.
- Removing use of the DataTransferResource when obtaining site to site details as it performs additional unnecessary checks.
- Code clean up.
- This closes apache#971.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@mcgilman @ijokarumawak