-
-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[cbor] Support writing Map (Object) with finite length #3
Comments
(modified title as #1 already covers case of known-length arrays) |
I'm facing the same issue, which lead to have some unnecessary additional bytes to the serialized data. |
As per example, usage would be via Streaming API (generator), and supporting that should be a bit easier than general-purpose databind. At databind, But one big problem is that there is no applicable method use: |
Thanks @cowtowncoder. Do you see any possible workaround at this point? |
Not really. You can by-pass encoder with |
Thanks @cowtowncoder , actually, I've found out by reading the source code that the method to write a Map with a fixed size exists: |
Ah. Well spotted. The reason I missed that was that I had thought of adding method in |
I'm having a problem with this feature. |
@quannh0308 I don't think this can work as general But I think it will be possible to add necessary method in
and then eventually make What will not be easy or perhaps even possible, unfortunately, will be doing the same for POJOs: problem is that things like But I think that ability to at least force this via Tree Model (it is, after all, always possible to convert from POJO to Tree Model, then serialize that) would make this possible in general case and by users who really need to force length prefix. I do think however that requiring use of length-prefix is a bad idea and implementations should not require it. |
Hi cowtowncoder, I wanted to use finite length Object serialization through ObjectMapper. I m doing
What I understood from your conversations is, the support is added since 3.0. But it looks like it hasn't been made in Jackson-databind package which means I cannot use ObjectMapper to get the finite length support. Can you comment on it. And second question is, do you think there should be a feature or flag to enable disable this functionality instead of making own assumption? If flag needs to be added, where will be added. Would like to know next steps on supporting finite length maps. Thanks for your hardwork on supporting the feature in 3.0. |
@rangaataws 3.0 has not been released, and will not yet for quite some time. Current releases are 2.x, latest being 2.10.0.
so that databind could start calling it in 2.11. Now... as to feature to add... perhaps. I think the way it would work is simply that databind would call method that passes length, when known -- it is not always practical to try to calculate it, for example when entries are filtered -- and then dataformat would decide to use it (or not). As to next steps: to be completely honest, I don't think I will have time to work on this in near future. |
Thanks for the quick response. If it's a lot of work, then I may use CborGenerator directly instead of using it via databind (although I like databind because it exposes clean interface by hiding internal implementation). I was looking at the 2.10 version of JsonGenerator where the method is implemented like below. I don't see size is used for writing the fixed length of the object. Maybe I m missing something.
Unrelated to this, do you think fixed length serialization adds value? I think the providing size may be helpful for languages like C where we have to allocate the memory. |
Right, that part is missing. Just needs to be wired. That would then go in 2.11, along with unit tests to ensure working operation. This issue is still open and not implemented. But aside from encoding length-prefix, generator will also need to keep track that exactly correct number of values will be written, to avoid producing corrupt content (claiming to write N entries, writing some other number). I think existing Array prefix handling code could show guidance. |
Thanks. In short,the latest released (2.10) version doesn't have support for fixed length which means I have to wait for 2.11 :) - Any estimated release date you know of ? reg. length prefix, you are right. Calculating the number of keys in the object is extra logic that could go wrong due to access specifiers and suppressable annotations what not. On the other hand, if customer needs to write the logic, they would have to do the same so I feel the size calculation logic can go into Generator to simplify customer serialization logic. Thanks. |
On calculating: what I mean is both databind side (which like you said can be complex / impractical), but also format (generator) side which must keep track of real value writes, not just what was alleged on As to 2.11: plan to get first release candidates by Dec 2019, release Jan 2020. |
Any updates on this? For now I simply manually write through generator to ease the pain, but I see no reason why serializing a simple Map (of Maps) should not be able to add lengths. |
Contributions welcome! As to why it is not trivial to implement: Jackson generators are by default fully streaming, incremental, which means that length-prefix use requires separate and additional, possibly nested buffering; adding both overhead and complexity. |
(from https://github.com/FasterXML/jackson-dataformat-cbor/issues/11)
By @mbaril:
It would be great to add the possibility to encode array and map with finite length.
The text was updated successfully, but these errors were encountered: