{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":112788577,"defaultBranch":"master","name":"pulsar","ownerLogin":"dlg99","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2017-12-01T21:27:46.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/8622884?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1719359880.0","currentOid":""},"activityList":{"items":[{"before":"cf611b2f0e46563b64d69fe85c3dab6f70a709a6","after":"6a4cff824f18b83f74094e02c18786d403f5abe1","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-27T23:15:15.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"Upgrade/Downgrade test","shortMessageHtmlLink":"Upgrade/Downgrade test"}},{"before":"cf3ec67d1bc395c6e44e610ccf77a1e3fc776354","after":"cf611b2f0e46563b64d69fe85c3dab6f70a709a6","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-27T22:59:24.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"fix handling of shared container","shortMessageHtmlLink":"fix handling of shared container"}},{"before":"cf851f65d5b07ed8c45ea199102ca34effd1c1ee","after":"cf3ec67d1bc395c6e44e610ccf77a1e3fc776354","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-27T21:38:56.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":".","shortMessageHtmlLink":"."}},{"before":"36526a60f9210ac21239018c675f193db2994609","after":"cf851f65d5b07ed8c45ea199102ca34effd1c1ee","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-27T20:37:46.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"cleanup storage container before run","shortMessageHtmlLink":"cleanup storage container before run"}},{"before":"3b654889ce7d255070dfb554cd7d96b4c56b14eb","after":"36526a60f9210ac21239018c675f193db2994609","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-27T05:03:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"trying storage container","shortMessageHtmlLink":"trying storage container"}},{"before":"2570ab8d6f811f2540962a8bdddf53453f1c0a80","after":"3b654889ce7d255070dfb554cd7d96b4c56b14eb","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-26T18:45:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"checkstyle","shortMessageHtmlLink":"checkstyle"}},{"before":"837bedf6f6e23e87d689e927b3e8389df10a66b6","after":"2570ab8d6f811f2540962a8bdddf53453f1c0a80","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-26T18:15:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"trying more","shortMessageHtmlLink":"trying more"}},{"before":"a74507105227675042534b2fdb89ad0fc36ea042","after":"837bedf6f6e23e87d689e927b3e8389df10a66b6","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-26T17:57:17.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"another try","shortMessageHtmlLink":"another try"}},{"before":"ac2aef5e63089eee1239f843b559ad2fcfa474ae","after":"a74507105227675042534b2fdb89ad0fc36ea042","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-26T16:48:29.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"trying to make CI work by using themp dir instead fo classdir","shortMessageHtmlLink":"trying to make CI work by using themp dir instead fo classdir"}},{"before":"7c8b14a519be3ea3a08811276d86481311ea2a13","after":"ac2aef5e63089eee1239f843b559ad2fcfa474ae","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-26T01:57:48.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"minus old zk test","shortMessageHtmlLink":"minus old zk test"}},{"before":"58dcddc643bf3d32aee4b00160206b8c055c7099","after":"7c8b14a519be3ea3a08811276d86481311ea2a13","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-26T00:21:29.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"Add to CI","shortMessageHtmlLink":"Add to CI"}},{"before":"82025b8ea104937f933b45ede188fd196fa212a0","after":"2da4ee8b54aa1e15d501b57cd4c476186aff92eb","ref":"refs/heads/master","pushedAt":"2024-06-25T23:58:12.000Z","pushType":"push","commitsCount":107,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"[revert] \"[improve][broker] Optimize `ConcurrentOpenLongPairRangeSet` by RoaringBitmap (#22908)\" (#22968)","shortMessageHtmlLink":"[revert] \"[improve][broker] Optimize ConcurrentOpenLongPairRangeSet…"}},{"before":null,"after":"58dcddc643bf3d32aee4b00160206b8c055c7099","ref":"refs/heads/downgrade-test","pushedAt":"2024-06-25T23:58:00.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"Upgrade/Downgrade test","shortMessageHtmlLink":"Upgrade/Downgrade test"}},{"before":"7cb7da8d60a69fc939267ea00719f666b5ef66ff","after":"7bb5e15ab446c77b4adac8dabbfd1650dac00eb4","ref":"refs/heads/cpick-pr271","pushedAt":"2024-05-29T23:02:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"CR feedback: use better executor","shortMessageHtmlLink":"CR feedback: use better executor"}},{"before":"794ae7de6a6082166a613336493aa560c59dfbde","after":"6fd14cc9eba21d90756538640951c299a76ccf80","ref":"refs/heads/cpick/cursor-large-state","pushedAt":"2024-05-29T21:52:26.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"fix boken test after merge/resolve","shortMessageHtmlLink":"fix boken test after merge/resolve"}},{"before":null,"after":"794ae7de6a6082166a613336493aa560c59dfbde","ref":"refs/heads/cpick/cursor-large-state","pushedAt":"2024-05-28T23:02:47.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"try ledger recovery from previous entries in case of corrupt/missing footer of the chunked data (#282)\n\n(cherry picked from commit 6e72ecbba11325ba17f78a903ec7298bbe937b03)","shortMessageHtmlLink":"try ledger recovery from previous entries in case of corrupt/missing …"}},{"before":null,"after":"7cb7da8d60a69fc939267ea00719f666b5ef66ff","ref":"refs/heads/cpick-pr271","pushedAt":"2024-05-28T17:16:45.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"Do not process acks in the Netty thread (#271)\n\n(cherry picked from commit 5ed7dd102577bb9d029e71042b20c760fa1a4ea5)","shortMessageHtmlLink":"Do not process acks in the Netty thread (apache#271)"}},{"before":"990b8d0334c75255e25899df869887711059cb33","after":"82025b8ea104937f933b45ede188fd196fa212a0","ref":"refs/heads/master","pushedAt":"2024-05-28T17:05:28.000Z","pushType":"push","commitsCount":148,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"[improve][broker] avoid creating new objects when intercepting (#22790)","shortMessageHtmlLink":"[improve][broker] avoid creating new objects when intercepting (apach…"}},{"before":null,"after":"c1ee50d744dced1b718561dd6d08e243c6953cd2","ref":"refs/heads/cursor-recovery","pushedAt":"2024-05-23T18:53:41.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"try ledger recovery from previous entries in cas eof corrupt/missing footer of the chunked data","shortMessageHtmlLink":"try ledger recovery from previous entries in cas eof corrupt/missing …"}},{"before":null,"after":"de5fde304b4a7741fbd892325845c7afe8d98de1","ref":"refs/heads/apache-nonpersistent_keyhash","pushedAt":"2024-04-18T18:15:16.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"reduce stickyHash calculations ofr non-persistent topics","shortMessageHtmlLink":"reduce stickyHash calculations ofr non-persistent topics"}},{"before":"b50e8802a5224dd68832e263e7046650771a1a4e","after":"990b8d0334c75255e25899df869887711059cb33","ref":"refs/heads/master","pushedAt":"2024-04-18T17:34:02.000Z","pushType":"push","commitsCount":1013,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"[improve][build] Upgrade OWASP Dependency check version to 9.1.0 (#22530)","shortMessageHtmlLink":"[improve][build] Upgrade OWASP Dependency check version to 9.1.0 (apa…"}},{"before":null,"after":"99197c3fd8550c5b8a5d4028667903c35707e1a5","ref":"refs/heads/kca-commitrecord-master","pushedAt":"2023-04-17T21:59:16.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"license header","shortMessageHtmlLink":"license header"}},{"before":"075b625993746a6c2d72343cb969845aed2cd28a","after":"b50e8802a5224dd68832e263e7046650771a1a4e","ref":"refs/heads/master","pushedAt":"2023-04-17T21:35:04.000Z","pushType":"push","commitsCount":18,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"[improve] [broker] Close temporary open ledger in BookkeeperBucketSnapshotStorage (#20111)","shortMessageHtmlLink":"[improve] [broker] Close temporary open ledger in BookkeeperBucketSna…"}},{"before":"71fb2a9ef6e936c45b40064fa3218e99738a8f5b","after":"a0ff03ef5fdc29b18c713596fafb1d5315d26ee9","ref":"refs/heads/kca-commitrecord_ds","pushedAt":"2023-04-17T20:51:28.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"KCA: handle kafka sources that use commitRecord","shortMessageHtmlLink":"KCA: handle kafka sources that use commitRecord"}},{"before":null,"after":"71fb2a9ef6e936c45b40064fa3218e99738a8f5b","ref":"refs/heads/kca-commitrecord_ds","pushedAt":"2023-04-15T00:51:07.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"KCA: handle kafka sources that use commitRecord","shortMessageHtmlLink":"KCA: handle kafka sources that use commitRecord"}},{"before":null,"after":"54bb3a7b382685fa253ffb7a2f861e63015f32ed","ref":"refs/heads/kca-commitrecord","pushedAt":"2023-04-15T00:46:22.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"KCA: handle kafka sources that use commitRecord","shortMessageHtmlLink":"KCA: handle kafka sources that use commitRecord"}},{"before":"79e17ba5e09910ce4e4a63a99f0dac90604918a5","after":"db4350dd9867287ea516692087f81ffa9304f61f","ref":"refs/heads/vertx-upgrade-432","pushedAt":"2023-04-11T22:48:38.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"upgrade to 4.3.8, as in 4.16.0 release, not to version in PR upgrading vertx","shortMessageHtmlLink":"upgrade to 4.3.8, as in 4.16.0 release, not to version in PR upgradin…"}},{"before":"075b625993746a6c2d72343cb969845aed2cd28a","after":"79e17ba5e09910ce4e4a63a99f0dac90604918a5","ref":"refs/heads/vertx-upgrade-432","pushedAt":"2023-04-11T21:11:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"Upgrade Vertx to match BK's version (otherwise BK does not work)","shortMessageHtmlLink":"Upgrade Vertx to match BK's version (otherwise BK does not work)"}},{"before":null,"after":"075b625993746a6c2d72343cb969845aed2cd28a","ref":"refs/heads/vertx-upgrade-432","pushedAt":"2023-04-11T21:09:25.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"[fix][proxy] Refresh auth data if ProxyLookupRequests (#20067)\n\nFixes: https://github.com/apache/pulsar/issues/10816\r\nPIP: #19771\r\nSupersedes: https://github.com/apache/pulsar/pull/19026\r\nDepends on: https://github.com/apache/pulsar/pull/20062\r\n\r\n### Motivation\r\n\r\nThe Pulsar Proxy does not properly handle authentication data refresh when in state `ProxyLookupRequests`. The consequence is described in https://github.com/apache/pulsar/issues/10816. Essentially, the problem is that the proxy caches stale authentication data and sends it to the broker leading to connection failures.\r\n\r\nhttps://github.com/apache/pulsar/pull/17831 attempted to fix the underlying problem, but it missed an important edge cases. Specifically, it missed the case that the `ConnectionPool` will have multiple connections when a lookup gets redirected. As such, the following problem exists (and is fixed by this PR):\r\n\r\n1. Client opens connection to perform lookups.\r\n2. Proxy connects to broker 1 to get the topic ownership info.\r\n3. Time passes.\r\n4. Client does an additional lookup, and this topic is on a newly created broker 2. In this case, the proxy opens a new connection with the stale client auth data.\r\n5. Broker 2 rejects the connection because it fails with expired authentication.\r\n\r\n### Modifications\r\n\r\n* Remove some of the implementation from https://github.com/apache/pulsar/pull/17831. This new implementation still allows a broker to challenge the client through the proxy, but notably, it limits the number of challenges sent to the client. Further, the proxy does not challenge the client when the auth data is not expired.\r\n* Introduce authentication refresh in the proxy so that the proxy challenges the client any time the auth data is expired.\r\n* Update the `ProxyClientCnx` to get the `clientAuthData` from the `ProxyConnection` to ensure that it gets new authentication data.\r\n* Add clock skew to the `AuthenticationProviderToken`. This is necessary to make some of the testing not flaky and it will also be necessary for users to configure in their clusters.\r\n\r\n### Verifying this change\r\n\r\nThe `ProxyRefreshAuthTest` covers the existing behavior and I expanded it to cover the edge case described above.\r\n\r\nAdditionally, testing this part of the code will be much easier to test once we implement #19624.\r\n\r\n### Documentation\r\n\r\n- [x] `doc-not-needed`\r\n\r\n### Matching PR in forked repository\r\n\r\nPR in forked repository: the relevant tests pass locally, so I am going to skip the forked tests.","shortMessageHtmlLink":"[fix][proxy] Refresh auth data if ProxyLookupRequests (apache#20067)"}},{"before":"1a6c28dd0072de05a544dbc9243bfbe6bccea5db","after":"075b625993746a6c2d72343cb969845aed2cd28a","ref":"refs/heads/master","pushedAt":"2023-04-11T21:02:36.000Z","pushType":"push","commitsCount":34,"pusher":{"login":"dlg99","name":"Andrey Yegorov","path":"/dlg99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8622884?s=80&v=4"},"commit":{"message":"[fix][proxy] Refresh auth data if ProxyLookupRequests (#20067)\n\nFixes: https://github.com/apache/pulsar/issues/10816\r\nPIP: #19771\r\nSupersedes: https://github.com/apache/pulsar/pull/19026\r\nDepends on: https://github.com/apache/pulsar/pull/20062\r\n\r\n### Motivation\r\n\r\nThe Pulsar Proxy does not properly handle authentication data refresh when in state `ProxyLookupRequests`. The consequence is described in https://github.com/apache/pulsar/issues/10816. Essentially, the problem is that the proxy caches stale authentication data and sends it to the broker leading to connection failures.\r\n\r\nhttps://github.com/apache/pulsar/pull/17831 attempted to fix the underlying problem, but it missed an important edge cases. Specifically, it missed the case that the `ConnectionPool` will have multiple connections when a lookup gets redirected. As such, the following problem exists (and is fixed by this PR):\r\n\r\n1. Client opens connection to perform lookups.\r\n2. Proxy connects to broker 1 to get the topic ownership info.\r\n3. Time passes.\r\n4. Client does an additional lookup, and this topic is on a newly created broker 2. In this case, the proxy opens a new connection with the stale client auth data.\r\n5. Broker 2 rejects the connection because it fails with expired authentication.\r\n\r\n### Modifications\r\n\r\n* Remove some of the implementation from https://github.com/apache/pulsar/pull/17831. This new implementation still allows a broker to challenge the client through the proxy, but notably, it limits the number of challenges sent to the client. Further, the proxy does not challenge the client when the auth data is not expired.\r\n* Introduce authentication refresh in the proxy so that the proxy challenges the client any time the auth data is expired.\r\n* Update the `ProxyClientCnx` to get the `clientAuthData` from the `ProxyConnection` to ensure that it gets new authentication data.\r\n* Add clock skew to the `AuthenticationProviderToken`. This is necessary to make some of the testing not flaky and it will also be necessary for users to configure in their clusters.\r\n\r\n### Verifying this change\r\n\r\nThe `ProxyRefreshAuthTest` covers the existing behavior and I expanded it to cover the edge case described above.\r\n\r\nAdditionally, testing this part of the code will be much easier to test once we implement #19624.\r\n\r\n### Documentation\r\n\r\n- [x] `doc-not-needed`\r\n\r\n### Matching PR in forked repository\r\n\r\nPR in forked repository: the relevant tests pass locally, so I am going to skip the forked tests.","shortMessageHtmlLink":"[fix][proxy] Refresh auth data if ProxyLookupRequests (apache#20067)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEcVWV8QA","startCursor":null,"endCursor":null}},"title":"Activity · dlg99/pulsar"}