Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

could indexr realtime ingeste can define the JSON format which fit the other json format. #6

Closed
AisTop opened this issue Jan 9, 2017 · 2 comments

Comments

@AisTop
Copy link

AisTop commented Jan 9, 2017

the example json format is {"tag": "a,b", "date": 20160702, "d1": "mac", "m1": 100, "m2": 48224, "m3": 0.76}

but my json format is
{
"date": "20161203",
"tag": "auth_data",
"list": [
{...},{...} ]
}
if i want to change the parse logic which class i could write。

@flowbehappy
Copy link
Collaborator

flowbehappy commented Jan 9, 2017

@AisTop

Currently IndexR does't support plugin. So you have to modify the source code and recompile if you need to do this.

Here are some hints:
UTF8Row.java: The rows in memory, what you need to transform into from your own event format.
Fetcher.java: The input source interface of rows.
UTF8JsonRowCreator: Implements the logic of transforming events into rows. This is the class you may need to replaced
UTF8JsonDeserializer: A fast json parser, currently only works for one level json. You probably can not use it according to your json format.
Kafka08Fetcher: Kafka 0.8 fetcher implementation. This class fetch messages from Kafka in byte[] and transform them into UTF8Row by an UTF8JsonRowCreator. Check Kafka08Fetcher#parseUTF8Row for detail.

Feel free to ask if you have any further questions. Hope you good luck :)

@AisTop
Copy link
Author

AisTop commented Jan 9, 2017

think you !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants