-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kinesis Elasticsearch Sink: cache the mapping of field name to field type #2090
Comments
My benchmark was flawed because it didn't take JIT optimization into account - the actual performance increase is a lot smaller. Still, I think this design change is an improvement. |
fblundun
changed the title
Kinesis Elasticsearch Sink: optimize transformation of TSV to JSON
Kinesis Elasticsearch Sink: cache the mapping of field name to field type
Oct 2, 2015
fblundun
added a commit
that referenced
this issue
Oct 2, 2015
Agree! |
fblundun
added a commit
that referenced
this issue
Oct 13, 2015
fblundun
added a commit
that referenced
this issue
Oct 16, 2015
fblundun
added a commit
that referenced
this issue
Oct 29, 2015
fblundun
referenced
this issue
Dec 9, 2015
fblundun
added a commit
that referenced
this issue
Feb 12, 2016
fblundun
added a commit
that referenced
this issue
Feb 15, 2016
fblundun
added a commit
that referenced
this issue
Feb 29, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The way this currently works, for every event, for every field, the code checks based on the name of the field how that field should be handled using the converter function. This involves checking whether the string name of the field is a member of various sets. It would be faster to precompute and cache the correspondence between field names and handler functions. (Crude benchmarking appears to show that this roughly doubles the speed of the
jsonifyGoodEvent
function.)The text was updated successfully, but these errors were encountered: