-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce json_object_new_string_{ext,noalloc}(). #620
Introduce json_object_new_string_{ext,noalloc}(). #620
Conversation
361525b
to
db2331c
Compare
db2331c
to
51ef3e5
Compare
This function helps to reduce code duplication, as it can be easily extended for different ways how a new json string-object will be created from a given string buffer.
On some low memory systems, calling malloc (by the use of strdup) may cause memory fragmentation, which eventually results in out of memory situations. Using the json_object_new_string_noalloc() function can be a help to solve such issues.
This is a short test to make sure the json_object_new_string_noalloc() function works as expected.
51ef3e5
to
0e43aab
Compare
memcpy(&dstbuf, &s, sizeof(char *)); | ||
/* string buffer must be free'd here | ||
to avoid memory leaks */ | ||
free(dstbuf); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can't guarantee that calling free() will be valid. Just because you don't want to strdup the string doesn't mean that it's memory from malloc. It might be global memory, or local variables that you happen to know won't go out of scope before the jso does, or a chunk of memory from an allocator other than malloc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To actually support this, we'll need a different way to indicate whether the string is stored directly or not. Perhaps, since c_string.len is signed, we could use negative lengths to mean it's stored directly? hmm...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can't guarantee that calling free() will be valid. Just because you don't want to strdup the string doesn't mean that it's memory from malloc. It might be global memory, or local variables that you happen to know won't go out of scope before the jso does, or a chunk of memory from an allocator other than malloc.
Then we would hit the same problem when the jso gets destroyed, which would call free()
on the string buffer through json_object_string_delete()
as well…
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've marked all changes, that I have made localy, as resolved to keep track of them. I'll push them as soon as we have found a solution here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well sure, so the code in json_object_string_delete would need to do if (jso->o.c_string.len < 0)
instead of if (jso->o.c_string.len >= LEN_DIRECT_STRING_DATA)
, and every other place that currently references LEN_DIRECT_STRING_DATA would need some kind of change. There are 7 of those total.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it be better to have a parameter which specifies a func ptr to the free()
function that corresponds to the undelying memory-allocator? And maybe NULL
to use the system default free()
? For stack-allocated objects we could offer a noop-function as well…
That seems better than relying on users to specify a negative value for the buffer length…
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You misunderstand me. I'm not saying that the caller would specify a negative value, it would be the json-c code that would use a negative value if it decides to store the string directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
...and in the no-strdup case we would never copy the string into direct storage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i.e.:
...
if ( !(flags & JSON_C_NEW_STRING_NO_STRDUP) &&
(len < LEN_DIRECT_STRING_DATA))
{
...
jso->o.c_string.len = -len;
return jso;
}
...
Any interest in revisiting this? If so, feel free to re-open this PR or create a new one against the most recent code (ideally with previous comments addressed). |
This PR is basically a working version of #476.
The
json_object_new_string_ext()
function helps to reduce code duplication, as it can be easily extended for different ways how a new json string-object will be created from a given string buffer.On some low memory systems, calling malloc (by the use of strdup) may cause memory fragmentation, which eventually results in out of memory situations.
Using the
json_object_new_string_noalloc()
function can help to solve such issues.test_string_noalloc
is a short test to make sure thejson_object_new_string_noalloc()
function works as expected.