Improve JSON serialization performance on Linux #718
Conversation
| _ = group.wait(timeout: .distantFuture) // forever | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Thanks for including a performance test.
I'd like to split it out into some other place. The unit test is to verify correctness in automation. As this lacks an assert, it wouldn't even really catch any regressions in reality.
There was a problem hiding this comment.
I wasn't sure where Foundation performance tests belong. Would the Swift Benchmark Suite be a better place? I'm happy to remove it from this PR and submit a separate one.
There was a problem hiding this comment.
We don't really have the infrastructure yet, but we should set something up.
I think we should start a discussion on swift-dev mailing list and see if the benchmark suite is the right place or not. If not, we can create our own as part of the project.
| let bufferLength = count+1 // Allow space for null terminator | ||
| var utf8: [CChar] = Array<CChar>(repeating: 0, count: bufferLength) | ||
| if !jsonStr.getCString(&utf8, maxLength: bufferLength, encoding: .utf8) { | ||
| fatalError("Failed to generate a CString from a String") |
There was a problem hiding this comment.
I realize we had a force-unwrap here in the previous code, but why would getCString fail here? If it did, is there a way to recover instead of asserting?
There was a problem hiding this comment.
Good question. The API documentation for Swift String.getCString does not specify what the return value means, but I gather from the implementation that it calls through to NSString.getCString, the documentation for which says that false would be returned if the buffer is too small, or an encoding error occurs.
In this case, we sized the buffer appropriately, and because UTF8 can represent all Unicode code points, we shouldn't encounter an encoding error. Perhaps it would be safe to ignore the return value in this case?
There was a problem hiding this comment.
If we're sure it should not happen, then asserting on that is the right answer instead of ignoring.
This reverts commit 867109e.
|
@parkera I've removed the performance test from this PR, and I posted to the swift-corelibs-dev mailing list re. where it belongs. |
|
@swift-ci please test and merge |
|
Let's get this into master and then I'll check on the Swift 3.0.2 question. |
This change improves the performance of serializing objects to a Data, by using String concatenation to build the result, and then convert to a Data once. This change was developed by @shmuelk and myself.
We may want to revisit this in the future as the performance of appending to Data improves.
Note, we initially used
jsonStr.cString(using: .utf8)to create the Data, but in testing this seemed to leak memory (perhaps this is a bug): IBM-Swift/SwiftyJSON#22I've provided a microbenchmark to demonstrate the effect of these changes:
Before:
After:
The above results were collected on Ubuntu 16.04. I obtained similar results on OS X.