-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is encoding Double or Float supported? #20
Comments
It seems I introduced a bug encoding/decoding It is an easy fix. I will have a patch ready in a couple of hours. Thank you for reporting. |
I have fixed the floating-point problem and added some tests to check correct behavior. The fixes can be found in the develop branch. Please, give it a try and if it solves your problems, I will promote the fix to master and release a new patch version. |
Oh, and I forgot to mention that I have renamed |
Thanks @dehesa. The following statement now works:
but the following does not:
|
Uhm, ok. That is interesting. Could you provide a small example with the behavior you are expecting, such as the one I have in the tests. It will help me out debugging. |
Sorry, am not sure if my Codable experience allows me to replicate without the container that I'm using within the init(from: Decoder). I'm trying to decode an enum property in the parent struct (see below) by decoding each field individually. I'd really like to be able to decode the Codable enum like I can with JSON, but the new container index seems to get mixed up and always fails. I'm essentially serializing an enum which has one case with an associated value and the other does not (its essentially a simplified Result type). Here's how I'd expect it to work (but doesn't):
Here's what I've had to resort to because of the container index becomes problematic when moving to the embedded enum, but it also doesn't work because of the decodeIfPresent(Double.self) always throws.
I'm expecting (and encoding to) the following string records Sorry if this is fundamentally the wrong approach here, but its what I'm coming from the experience I have coding/decoding JSON with Codable. |
Alright. Let me take a look at it. I have a further question, though. Is the CSV schema closed? I mean, is that row definition final? |
For this simple problem then yes, that's doable. I'm basically working on a dummy project to learn some aspects of SwiftUI plotting, Combine and Coding CSV values. The real project I plan to use all this on, however, is managing an assortment time series that are sampled at various frequencies. So column positions are important to determine if there is a datum at any particular sample time. Time series CSV is generally columnar data and in the exports I'll have to process I won't have control of the collapsing of columns in particular rows, they'll simply be empty. So, for the dummy project yes its easy to adapt. But the dummy project has a pattern I've chosen specifically as its the construct I'll have to support when I get past learning/prototyping (I'm actually a believer in a different second system effect, the first time I don't know what I'm doing and it ends up ugly, unmaintainable, and often inefficient. When I do it for the second time I have the knowledge to do it right and am not left with refactoring artifacts I wish weren't there...so thats why I'm constraining this work a bit more than necessary...) |
I fixed a small behavior on For your specific problem, you actually have several solutions (be sure to download the latest from develop branch):
|
Let me know if that works for you and then I will upstream the patch to master. Again, thank you for reporting. |
Yes, I can decodeIfPresent() successfully now. One last question, I encodeRow() one result at a time (because they're a live time series) thus am using lazy. After each encoding I endEncoding() and then write the data with outputStream.writeBytes(). I've learned that I have to re-create a new encoder for the next datum after I've endEncoding() on the last live time series datum. Is there a manner in which I can reuse an encoder or is creating a new encoder cheap enough on every live datum that is collected? |
That is not ideal. Creating a new encoder is not "very expensive", but you are creating and deallocating Why don't you just store the lazy encoder somewhere (it is a reference type) and reuse when a new row arrives? // 1. Create lazy encoder and store it somewhere.
self.cacheEncoder = CSVEncoder {
$0.bufferingStrategy = .sequential
}.lazy(into: /* Whatever you need */)
// 2. Then, every time a new row arrives.
try self.cacheEncoder.encodeRow(customStructure)
// 3. Once you stop the live subscription, call
try self.cacheEncoder.endEncoding()
// there is also a defined deinit, which will `endEncoding()` for you (if you haven't called it) when the reference is deallocated. |
I did try to store the encoder as a class property, but endEncoding on each datum so the file is tail-able, for example. This caused the next datum encoding to fail. For a long running process with a live feed, the data persistence is paramount, so I keep my output stream tied to a file open and continuously append to it, and then close the output stream on deinit, which could be never (or at server reboot, or when files roll). Its somewhat akin to using CSV encoding for a logger, I guess, which may be a bit corrupt. But data loggers do that by nature, and CSV is a natural export format for columnar data of time series data. Would an option to flush the encoder instead of end it be viable here? The other option is for the client code do so on a Timer, letting the buffer grow and end encoding then create a new encoder again. I think this may simply be a case where encoding is not worth the effort beyond:
|
If I understood everything correctly, I believe you have three options to satisfy your requirements.
|
ok, thanks the details and the help here. Those details were not gleamed from your readme page when I first attempted to integrate the library. I may send you a PR for the README to expand on usage so its easier for first time users who may be new to CSV coding; what's there now seems to have some unstated assumptions/knowledge that I certainly didn't have. I'll admit, I had to trap errors to figure out what I needed to do or was doing wrong, which is not an efficient means and even still wasn't/am not really using it correctly without what you thus shared here. |
I agree there might be some assumptions in the README about how to use the library. I have spent so much time on it that some knowledge is second nature to me. I would be delighted if you send PRs my way; having a different view on how to express things is always valuable. I will close this issue now, but feel free to open new ones if you have any more questions or you would like to improve something. If you decide to get involved in the project, the following things help:
|
When I change the floatStrategy as .convert I get an fatal error in this code:
So with either strategy I either get the thrown error or a fatal error if a valid Double is attempted to be encoded. Is this expected or is there another configuration I'm missing?
Similarly, when I try to decode a double, I get an error thrown, but when I decode it as a string and convert the string to a double in my structs init(from: Decoder) I process the field correctly.
The text was updated successfully, but these errors were encountered: