Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Real-time json index #6408

Merged
merged 1 commit into from Jan 7, 2021
Merged

Conversation

Jackie-Jiang
Copy link
Contributor

Description

Add json index support to mutable (real-time) segments
Add a new quickstart for real-time json index on meetup RSVP events

Example query: Find events that are in the fitness group
select json_extract_scalar(event, '$.event_name', 'STRING') from meetupRsvp where json_match("group", 'group_topics.topic_name=''Fitness''') limit 10

@codecov-io
Copy link

codecov-io commented Jan 5, 2021

Codecov Report

Merging #6408 (4bcdbce) into master (1beaab5) will decrease coverage by 1.42%.
The diff coverage is 56.80%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6408      +/-   ##
==========================================
- Coverage   66.44%   65.02%   -1.43%     
==========================================
  Files        1075     1319     +244     
  Lines       54773    64301    +9528     
  Branches     8168     9361    +1193     
==========================================
+ Hits        36396    41810    +5414     
- Misses      15700    19512    +3812     
- Partials     2677     2979     +302     
Flag Coverage Δ
unittests 65.02% <56.80%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...e/pinot/broker/api/resources/PinotBrokerDebug.java 0.00% <0.00%> (-79.32%) ⬇️
...ot/broker/broker/AllowAllAccessControlFactory.java 71.42% <ø> (-28.58%) ⬇️
.../helix/BrokerUserDefinedMessageHandlerFactory.java 33.96% <0.00%> (-32.71%) ⬇️
...ker/routing/instanceselector/InstanceSelector.java 100.00% <ø> (ø)
...ava/org/apache/pinot/client/AbstractResultSet.java 66.66% <0.00%> (+9.52%) ⬆️
.../main/java/org/apache/pinot/client/Connection.java 35.55% <0.00%> (-13.29%) ⬇️
...inot/client/JsonAsyncHttpPinotClientTransport.java 10.90% <0.00%> (-51.10%) ⬇️
...not/common/assignment/InstancePartitionsUtils.java 73.80% <ø> (+0.63%) ⬆️
...common/config/tuner/NoOpTableTableConfigTuner.java 100.00% <ø> (ø)
...ot/common/config/tuner/RealTimeAutoIndexTuner.java 100.00% <ø> (ø)
... and 1153 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e0398b...4bcdbce. Read the comment docs.

@@ -233,8 +233,8 @@ public void deleteSegmentFile() {
private final SegmentVersion _segmentVersion;
private final SegmentBuildTimeLeaseExtender _leaseExtender;
private SegmentBuildDescriptor _segmentBuildDescriptor;
private StreamConsumerFactory _streamConsumerFactory;
private StreamPartitionMsgOffsetFactory _streamPartitionMsgOffsetFactory;
private final StreamConsumerFactory _streamConsumerFactory;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need final for these ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, seems these are changed by the IDE auto-formatter

public void onOpen(Session session, EndpointConfig config) {
session.addMessageHandler(new MessageHandler.Whole<String>() {
@Override
public void onMessage(String message) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we extend MeetupRsvpStream and override a method to generate the msg to produce?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, let me add that

@Jackie-Jiang Jackie-Jiang merged commit e11bcce into apache:master Jan 7, 2021
@Jackie-Jiang Jackie-Jiang deleted the json_realtime branch January 7, 2021 02:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants