Skip to content

KNOX-2773 - Log replay buffer size with info level#604

Merged
smolnar82 merged 1 commit intoapache:masterfrom
MrtnBalazs:KNOX-2773
Jul 7, 2022
Merged

KNOX-2773 - Log replay buffer size with info level#604
smolnar82 merged 1 commit intoapache:masterfrom
MrtnBalazs:KNOX-2773

Conversation

@MrtnBalazs
Copy link
Contributor

What changes were proposed in this pull request?

The replayBufferSize log level was changed from debug to info, to prevent confusion on what value was used, since it can be configured in two places (topology file and service.xml). Now the client can see what value was configured as replayBufferSize in the log after the first request after deploying a topology.

How was this patch tested?

I tested it manually, using the following curl command
curl -k -u tom:tom-password https://localhost:8443/gateway/sandbox/hive

  1. No configuration in either file
    The log output was:
2022-07-07 09:27:30,761 ff7f8623-703b-4118-9d83-7a73d5f37231 INFO  knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom
2022-07-07 09:27:34,515 ff7f8623-703b-4118-9d83-7a73d5f37231 INFO  knox.gateway (DefaultDispatch.java:setReplayBufferSizeInBytes(121)) - replayBufferSize is set to -1 for HIVE
  1. Configured in service.xml
    Configuration:
<param>
    <name>replayBufferSize</name>
    <value>11</value>
</param>

Log output:

022-07-07 09:29:46,028 d15fb564-9185-420c-91da-3c31734e1a9c INFO  knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom
2022-07-07 09:29:54,590 d15fb564-9185-420c-91da-3c31734e1a9c INFO  knox.gateway (DefaultDispatch.java:setReplayBufferSizeInBytes(121)) - replayBufferSize is set to 11,264 for HIVE
  1. Configured in service.xml and in the topology file
    Service.xml configuration:
<param>
    <name>replayBufferSize</name>
    <value>11</value>
</param>

Topology configuration:

<param>
    <name>replayBufferSize</name>
    <value>5</value>
</param>

Log output:

022-07-07 09:31:30,155 df38456e-3b9c-4f29-baca-768a323f4bbe INFO  knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom
2022-07-07 09:31:33,457 df38456e-3b9c-4f29-baca-768a323f4bbe INFO  knox.gateway (DefaultDispatch.java:setReplayBufferSizeInBytes(121)) - replayBufferSize is set to 5,120 for HIVE

We can see that the topology file overrides the service.xml if they are configured differently.

@MrtnBalazs
Copy link
Contributor Author

@zeroflag @smolnar82

@zeroflag zeroflag requested review from smolnar82 and zeroflag July 7, 2022 08:05
Copy link
Contributor

@smolnar82 smolnar82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zeroflag
Copy link
Contributor

zeroflag commented Jul 7, 2022

Code change looks good. Did you manage to trace back the caller of this function in the code? It should originate from an init() or something equivalent. We need to make sure this is not called repeatably but only once.

@smolnar82
Copy link
Contributor

Thanks, @MrtnBalazs for the detailed description above!

@smolnar82
Copy link
Contributor

Code change looks good. Did you manage to trace back the caller of this function in the code? It should originate from an init() something equivalent. We need to make sure this is not called repeatably but only once.

Maybe invoking the same curl command 2-3 times could prove that, right?

@zeroflag
Copy link
Contributor

zeroflag commented Jul 7, 2022

Maybe invoking the same curl command 2-3 times could prove that, right?

There might be another code path. It would be good to see it in the code level as well.

@MrtnBalazs
Copy link
Contributor Author

I tried invoking the same curl command multiple times, and it logged only the first time. I could'n t trace it back to an init() function.

@zeroflag
Copy link
Contributor

zeroflag commented Jul 7, 2022

I tried invoking the same curl command multiple times, and it logged only the first time. I could'n t trace it back to an init() function.

Yes it's non trivial. I think this can be a good exercise to find out how that works.

@MrtnBalazs
Copy link
Contributor Author

MrtnBalazs commented Jul 7, 2022

I managed to trace it back to GatewayDispatchFilter.java init() method, which overrides AbstractGatewayFilter-s init() method which comes from the javax.servlet.Filter interface.
The documentation of this interfaces init() method says:

Called by the web container to indicate to a filter that it is being placed into service.
The servlet container calls the init method exactly once after instantiating the filter. The init method must complete successfully before the filter is asked to do any filtering work.

Copy link
Contributor

@zeroflag zeroflag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@smolnar82 smolnar82 merged commit 32ecc76 into apache:master Jul 7, 2022
stoty pushed a commit to stoty/knox that referenced this pull request May 14, 2024
…yBufferSize log (apache#604)

Change-Id: I92fee4fccc49cbdb1d36260c5c6d9f03b8caf1cb
stoty pushed a commit to stoty/knox that referenced this pull request May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments