Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Stream API to KRisIO #144

Merged
merged 1 commit into from
Apr 3, 2023

Conversation

pawelpasterz
Copy link

Changes:
When parsing on large RIS files (40k+ records) I've noticed large memory consumption. I think KRisIO could expose API with java stream to enable users to utilise stream on their own.

Copy link
Owner

@ursjoss ursjoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pawelpasterz

@Throws(IOException::class)
public fun processToStream(reader: Reader): Stream<RisRecord> = runBlocking(Dispatchers.IO) {
val lineFlow = BufferedReader(reader).lineSequence().asFlow()
Stream.builder<RisRecord>().also { recordsStream ->
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO also semantically does not fit that well. I prefer to use apply in this case. Since I have a few other minor changes (small refactoring in the test, adding the new functionality to the guide, minor cleanup I noticed here but for stuff I introduced earlier), I will merge this PR as is and follow up with my own.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks! I'll try to keep your convention better next time

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You followed my conventions so well that you also copied my grammar mistakes from before :-) No, seriously, there's no complaint about you not following my conventions, as most changes in my follow up PR are fixing my own mistakes from before.

Only Regarding the scoping functions. To me the semantics of apply is about initialization or configurationof an object, whereas also uses the lambda to do something with the object after it has been used. To me apply is more fitting than also in our use case, as we prefill the builder before calling the build method. But I guess that's also opinionated.

Thanks for your contribution.

@ursjoss ursjoss merged commit 9ef25f2 into ursjoss:main Apr 3, 2023
ursjoss added a commit that referenced this pull request Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants