Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support I/O stream #204

Closed
chenfei0928 opened this issue Aug 30, 2018 · 19 comments
Closed

Support I/O stream #204

chenfei0928 opened this issue Aug 30, 2018 · 19 comments

Comments

@chenfei0928
Copy link

chenfei0928 commented Aug 30, 2018

If I deserialize a json from file system, deserialize method require me full read the file to a string.
Can it support read and write json content from I/O stream, like Gson done?

@sandwwraith
Copy link
Member

sandwwraith commented Aug 30, 2018

We are planning a migration to kotlin multiplatform IO library – https://github.com/Kotlin/kotlinx-io . JSON parser would support both strings and kotlinx.io inputs.

charleskorn added a commit to batect/batect that referenced this issue Jan 22, 2020
… become healthy with Docker 19.03.5.

Docker 19.03.5 no longer sends new line characters between events, so
we can't use that as a reliable way to read the first event from the
event stream.

So instead, we have to keep reading characters, one at a time, until
we get a valid JSON object.

Once Kotlin/kotlinx.serialization#204 (or
Kotlin/kotlinx.serialization#253) is
implemented, we can make this slightly nicer by just directly decoding
from a stream.

This resolves #393.
@gajicm93
Copy link

gajicm93 commented Apr 9, 2020

This is quite a bummer because I know for a fact that Android can crash when a very large JSON payload is converted to a String at once.

When can we expect a streaming parse? Thanks.

@jcraane
Copy link

jcraane commented Jul 9, 2020

We indeed face this issue when per persist large json (the apps data) which sometimes triggers OutOfMemory.

@sdeleuze
Copy link

sdeleuze commented Oct 6, 2020

We are interested by that one for Spring Framework support, beware that for Spring MVC we typically deal with InputStream / OutputStream while on Spring WebFlux side we typically deal with java.nio.ByteBuffer, potentially in combination with the upcoming JSON streaming support (#1073).

@epicstar
Copy link

This is quite a bummer because I know for a fact that Android can crash when a very large JSON payload is converted to a String at once.

Yes, this should be considered a stopper for all Android applications.

@mobiletoly
Copy link

Any news or roadmap?

@sandwwraith
Copy link
Member

It's planned for 1.1 or 1.2

@ln-12
Copy link

ln-12 commented Feb 9, 2021

Seems like the feature didn't make into 1.1.0-RC. Currently, it's the only reason why I would not feel confident shipping it in a production app.

@MolotovCherry
Copy link

Really sad, I was looking forward to this being supported

@sdeleuze
Copy link

sdeleuze commented Apr 9, 2021

Could you please add support for that in 1.2 in order to allow Spring to use it in Spring Boot 2.5 ?

@slavonnet
Copy link

slavonnet commented May 12, 2021

+100500 votes! How read big json without buffer, flow or streams?

@chris-hatton
Copy link

chris-hatton commented Jun 26, 2021

I'm developing an application with kotlinx.serialization and came to a point where I really want to implement a Flow<T> of objects from Server to Client, based on the HTTP return stream. It seems unfeasible to do this at the moment due to lack of stream processing support with kotlinx.serialization. This is the biggest weakness I can see with this otherwise great library...

Others have mentioned resource usage; but I wanted to bring another use-case reason to support streaming. Thanks for considering.

@chris-hatton
Copy link

If kotlinx.io has been put on hold; could maintaining developers of kotlinx.serialization consider a separate effort to expose a streaming API? I have seen suggestions that parsing is modelled around a stream internally, but for some reason this isn't exposed.

@sandwwraith
Copy link
Member

I have seen suggestions that parsing is modelled around a stream internally

that (wasn't) true

We currently have progress on working implementation over Java's InputStream/OutputStream (#1569 ) You can take a look and tell if that fits your needs.

@qwwdfsad
Copy link
Collaborator

Fixed in 1.3.0-RC

@slavonnet
Copy link

random EOF exceptions on heavy loads.

@sandwwraith
Copy link
Member

@slavonnet Can you please create a separate issue with a reproducer or at least a description of conditions under which exception occurs?

@martinbonnin
Copy link
Contributor

Is there a separate issue for multiplatform streaming ?

@sandwwraith
Copy link
Member

@martinbonnin I've cerated one for you: #1714

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests