Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix timing issue with initializing Gateway authentication #1255

Closed
CarsonCook opened this issue Mar 12, 2021 · 1 comment · Fixed by #1301
Closed

Fix timing issue with initializing Gateway authentication #1255

CarsonCook opened this issue Mar 12, 2021 · 1 comment · Fixed by #1301
Assignees
Labels
bug Verified defect in functionality Priority: High
Milestone

Comments

@CarsonCook
Copy link
Contributor

CarsonCook commented Mar 12, 2021

Describe the bug
When the Discovery Service is registering zOSMF while the Gateway is starting, there is a timing issue with authenticating. The Gateway waits one minute before polling to see if zOSMF is registered, in this one minute, a user can send a login request to the Gateway. This login request will executing thinking zOSMF doesn't support JWTs, because the zOSMF authenticate endpoint doesn't return 401 (as it is not yet registered). Thus, the Gateway attempts to use the jwtKeyAlias to create a JWT. At this point the jwt secret hasn't been loaded, because the Gateway is waiting for zOSMF to be registered with the Discovery Service to check if the jwt secret should be loaded. So the login request uses a null jwt key argument, throwing an error.

This is a timing problem where zOSMF can be registered with the Discovery Service, but the Gateway does not yet know this and hasn't loaded the jwt key secret. A login request comes in and is routed to zOSMF because it is registered, but the jwt key secret is null as it hasn't been loaded.

Steps to Reproduce

  1. Easiest to see this behaviour running locally with breakpoints, so put a breakpoint in JwtSecurityInitializer where waitUntilZosmfIsUp is called - currently line 96.
  2. Start discovery service, gateway (debug mode), and mock zosmf with AuthenticateApar applied.
  3. Wait until the gateway breakpoint is tripped, and then allow it to pass
  4. Send a login request to the gateway. This may return Service Unavailable as zOSMF has not yet registered to the discovery service. Keep trying to send login requests.
  5. See a response to your login request with IllegalArgumentException: Key argument cannot be null
  6. Wait some time (~1 minute), and try to send another login request. Once the gateway has polled zOSMF, the login request will succeed

Expected behavior
Options:

  1. Send service unavailable while the GW is waiting on zOSMF, even if zOSMF is registered with the discovery service, and only let requests through to zOSMF once the gateway is done checking if zOSMF supports JWT
  2. Don't have GW finish starting until it is finished checking zOSMF
  3. If user sets APIML_SECURITY_ZOSMF_JWT_AUTOCONFIGURATION_MODE=LTPA, immediately load jwt secret. However, if the mode is set to AUTO there will still be a timing issue
  4. Adjust GW polling so that it uses a registered discovery client and checks on every registration event, rather than checking every minute
  5. (Bad option) rely on documentation and logs to tell users to simply wait

https://github.com/zowe/api-layer/wiki/Issue-management

@CarsonCook CarsonCook added bug Verified defect in functionality new New issue that has not been worked on yet labels Mar 12, 2021
@CarsonCook CarsonCook changed the title Enhance AUTO option for zosmfJwtAutoconfiguration Fix timing issue with initializing Gateway authentication Mar 12, 2021
@CarsonCook
Copy link
Contributor Author

Another possible solution would be reducing the polling time down from one minute. Though this just addresses the symptom, not the cause, and would still leave the door open for future timing issues.

@balhar-jakub balhar-jakub added 21PI1 Priority: High and removed new New issue that has not been worked on yet labels Mar 17, 2021
@CarsonCook CarsonCook self-assigned this Mar 18, 2021
@anton-brezina anton-brezina added this to the 21PI1S4 milestone Mar 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Verified defect in functionality Priority: High
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants