Messages API v2 #3537

iliax · 2023-03-22T19:19:31Z

Breaking change? (if so, please describe the impact and migration path for existing application instances)

What changes did you make? (Give an overview)

Is there anything you'd like reviewers to focus on?

How Has This Been Tested? (put an "x" (case-sensitive!) next to an item)

Checklist (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation (e.g. ENVIRONMENT VARIABLES)
My changes generate no new warnings (e.g. Sonar is happy)
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged

Check out Contributing and Code of Conduct

A picture of a cute animal (not mandatory but encouraged)

…4_messagesApiV2

sonarcloud · 2023-03-23T10:51:30Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

…4_messagesApiV2 � Conflicts: � kafka-ui-api/src/main/java/com/provectus/kafka/ui/controller/MessagesController.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/TailingEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/MessagesService.java � kafka-ui-api/src/test/java/com/provectus/kafka/ui/service/RecordEmitterTest.java

sonarcloud · 2023-04-24T19:59:25Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

Haarolean

Looks clean, as a suggestion let's ask frontend folks to take a look at the API, if it's understandable enough for them

Haarolean · 2023-07-10T16:15:37Z

kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/PollingCursorsStorage.java

+  public static final int MAX_SIZE = 10_000;
+
+  private final Cache<String, Cursor> cursorsCache = CacheBuilder.newBuilder()
+      .maximumSize(MAX_SIZE)


Shouldn't we add expireAfterAccess and/or expireAfterWrite?

it is hard to define good properties for that. I think default policy (delete oldest when max size is reached) should work fine.

Haarolean · 2023-07-10T18:35:02Z

kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/OffsetsInfo.java

@@ -56,4 +57,8 @@ public boolean assignedPartitionsFullyPolled() {
    return true;
  }

+  public Set<TopicPartition> allTargetPartitions() {
+    return Sets.union(nonEmptyPartitions, emptyPartitions);


wouldn't it better to calculate this in the constructor instead of doing so on every each method call?

it only called in once and very light to created, so I guess its better to create it lazily

Haarolean · 2023-07-10T18:36:57Z

kafka-ui-api/src/main/java/com/provectus/kafka/ui/model/ConsumerPosition.java

+                               @Nullable Long timestamp,
+                               @Nullable Offsets offsets) {
+
+  // one of properties will be null


should we assert that either is null?

…4_messagesApiV2 � Conflicts: � kafka-ui-api/src/main/java/com/provectus/kafka/ui/controller/MessagesController.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/MessageFilters.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/MessagesService.java � kafka-ui-api/src/test/java/com/provectus/kafka/ui/service/MessagesServiceTest.java

…4_messagesApiV2 � Conflicts: � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/AbstractEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/ConsumingStats.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/ForwardRecordEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/TailingEmitter.java � kafka-ui-api/src/test/java/com/provectus/kafka/ui/service/RecordEmitterTest.java

…4_messagesApiV2 � Conflicts: � kafka-ui-api/src/main/java/com/provectus/kafka/ui/controller/MessagesController.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/AbstractEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/BackwardRecordEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/ConsumingStats.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/ForwardRecordEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/MessagesProcessing.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/OffsetsInfo.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/SeekOperations.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/TailingEmitter.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/serdes/ConsumerRecordDeserializer.java � kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/MessagesService.java � kafka-ui-api/src/test/java/com/provectus/kafka/ui/service/RecordEmitterTest.java

sonarcloud · 2023-08-21T13:45:15Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

iliax · 2023-04-19T11:12:36Z

kafka-ui-contract/src/main/resources/swagger/kafka-ui-api.yaml

+            type: string
+        - name: offs
+          in: query
+          description: partition offsets to read from / to. Format is "p1:offset1,p2:offset2,...".


iliax · 2023-07-11T08:04:35Z

kafka-ui-api/src/main/java/com/provectus/kafka/ui/service/PollingCursorsStorage.java

+  public static final int MAX_SIZE = 10_000;
+
+  private final Cache<String, Cursor> cursorsCache = CacheBuilder.newBuilder()
+      .maximumSize(MAX_SIZE)


it is hard to define good properties for that. I think default policy (delete oldest when max size is reached) should work fine.

iliax · 2023-07-11T08:04:45Z

kafka-ui-api/src/main/java/com/provectus/kafka/ui/model/ConsumerPosition.java

+                               @Nullable Long timestamp,
+                               @Nullable Offsets offsets) {
+
+  // one of properties will be null


iliax · 2023-07-11T08:06:04Z

kafka-ui-api/src/main/java/com/provectus/kafka/ui/emitter/OffsetsInfo.java

@@ -56,4 +57,8 @@ public boolean assignedPartitionsFullyPolled() {
    return true;
  }

+  public Set<TopicPartition> allTargetPartitions() {
+    return Sets.union(nonEmptyPartitions, emptyPartitions);


it only called in once and very light to created, so I guess its better to create it lazily

iliax added 4 commits March 21, 2023 11:50

wip

a279c67

wip

4be46ec

Merge branch 'master' of github.com:provectus/kafka-ui into ISSUE-350…

2f67817

…4_messagesApiV2

wip

306c1fb

iliax requested review from a team and Haarolean as code owners March 22, 2023 19:19

github-actions bot added the status/triage Issues pending maintainers triage label Mar 22, 2023

iliax added 5 commits March 22, 2023 23:21

wip

35c63b8

wip

7f2f161

wip

23157b7

wip

f5d282e

wip

d6244b9

iliax changed the title ~~Messages API v2~~ WIP: Messages API v2 Mar 23, 2023

Merge branch 'master' into ISSUE-3504_messagesApiV2

498a099

iliax linked an issue Mar 24, 2023 that may be closed by this pull request

Messages v2: API overhaul #3504

Open

Haarolean mentioned this pull request Apr 6, 2023

Messages: SEEK ERROR with all partitions selected #3525

Closed

iliax added 4 commits April 24, 2023 11:45

api updates

8f21722

new tests

153745e

new tests

f05e8bb

iliax changed the title ~~WIP: Messages API v2~~ Messages API v2 Apr 24, 2023

minor improvements

911bbc8

Haarolean approved these changes Jul 10, 2023

View reviewed changes

Haarolean added type/enhancement En enhancement to an already existing feature scope/backend type/refactoring area/messages and removed status/triage Issues pending maintainers triage labels Jul 10, 2023

Haarolean added the impact/changelog A PR with changes which should be addressed in the changelog explicitly label Jul 10, 2023

iliax and others added 8 commits July 11, 2023 12:03

PR comments fx

b5a52fc

master merge

e04928c

master merge

c845786

Merge branch 'master' into ISSUE-3504_messagesApiV2

32af672

merged with master

d2e6e6a

Haarolean added the status/pending-frontend label Aug 30, 2023

iliax commented Jan 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Messages API v2 #3537

Messages API v2 #3537

iliax commented Mar 22, 2023 •

edited by Haarolean

Loading

sonarcloud bot commented Mar 23, 2023

sonarcloud bot commented Apr 24, 2023

Haarolean left a comment

Haarolean Jul 10, 2023

iliax Jul 11, 2023

Haarolean Jul 10, 2023

iliax Jul 11, 2023

Haarolean Jul 10, 2023

iliax Jul 11, 2023

sonarcloud bot commented Aug 21, 2023

iliax Apr 19, 2023

iliax Jul 11, 2023

iliax Jul 11, 2023

iliax Jul 11, 2023

Messages API v2 #3537

Are you sure you want to change the base?

Messages API v2 #3537

Conversation

iliax commented Mar 22, 2023 • edited by Haarolean Loading

sonarcloud bot commented Mar 23, 2023

sonarcloud bot commented Apr 24, 2023

Haarolean left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarcloud bot commented Aug 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iliax commented Mar 22, 2023 •

edited by Haarolean

Loading